OAuth governance was built for fixed-purpose SaaS apps. AI agents break that model.
When a user authorizes AI to Google Workspace, the grant event is indistinguishable from any other OAuth connection: same log format, same risk signals. But the behavior isn't the same. A traditional app acts within predictable scope. An AI agent's behavior is decided at inference time, driven by prompts your security team never sees.

Your SSPM sees the grant. It doesn't see what happens after.

Material's OAuth Remediation Agent monitors real-time activity post-grant, classifies risk by observed API behavior, and auto-revokes tokens the moment something deviates, without blocking legitimate AI adoption.

This is the first part of a three-part series on Dr. Katie Paxton-Fear’s excellent presentation, Hack Before You Launch. If you would like to stay up to date with the other articles, check out _secpro’s sister publication, cyber_ai.

Artificial intelligence has transformed software development at a remarkable pace. Tasks that once required experienced developers, months of planning, and significant financial investment can now be completed by individuals with little or no formal programming background. By describing a desired outcome in natural language, users can generate websites, databases, internal tools, and customer-facing applications in a matter of hours.

This phenomenon, often referred to as “vibe coding”, has lowered the barriers to software creation more dramatically than any previous technological shift. Entrepreneurs can test ideas without hiring development teams. Internal business units can build their own solutions rather than waiting for IT departments. Hobbyists can experiment with concepts that would previously have remained little more than sketches on paper.

Register for Forward, June 9-11, for a once-in-a-lifetime conversation with Anthropic CISO, Jason Clinton, about why cyber resilience isn’t just a best practice anymore. It’s survival.

Plus, hear from AWS, Microsoft, CrowdStrike, Cognizant, Alaska Airlines, The Home Depot, Piper Sandler, and dozens more. Then John Cena takes the mainstage with his personal blueprint for overcoming the impossible.

100+ speakers. 50+ sessions. Zero reasons to miss.

Yet while AI has made software development more accessible, it has not eliminated the challenges that accompany software deployment. Security remains one of the most significant of those challenges. During the recent “Hack Before You Launch” workshop, cybersecurity researcher Dr. Katie Paxton-Fear explored the growing disconnect between building applications and securing them, demonstrating how AI-generated software can quickly accumulate vulnerabilities despite appearing fully functional. Readers interested in the workshop itself can view the original event description and learning objectives here: Hack Before You Launch event page

The workshop’s central message was not that AI-generated code is inherently dangerous. Rather, it was that functionality and security are fundamentally different objectives: an application can successfully perform every task it was designed to accomplish while still exposing sensitive data, permitting unauthorised access, or creating opportunities for attackers. These categories of weakness align closely with the industry-standard OWASP Top 10 Web Application Security Risks, which remains one of the most widely used frameworks for evaluating application security.

This distinction is particularly important because AI development tools are often evaluated on their ability to produce visible results. Users judge success by whether a feature works, whether a page loads correctly, or whether a workflow behaves as expected. Attackers evaluate software differently. They are interested not in intended behaviour but in unintended behaviour. Their goal is to discover what an application permits beyond its design specifications.

To illustrate this challenge, the workshop examined the development of a simple AI-generated application. The initial requirements were straightforward: create a fantasy-themed shop for a tabletop role-playing game, generate stock lists and pricing, and provide functionality that would allow users to share the information with players. The resulting application successfully fulfilled its requirements. However, once security testing began, vulnerabilities quickly emerged. For organisations deploying AI systems, the OWASP Top 10 for LLM Applications 2025 provides a useful framework for understanding these emerging threats.

This outcome should not be surprising. Modern applications depend upon layers of libraries, frameworks, APIs, authentication services, and cloud infrastructure. Even experienced developers can struggle to maintain visibility over every component in a growing system. For users relying heavily on AI-generated code, that visibility may be even more limited. The application behaves as expected, but the underlying architecture often remains largely opaque to the person who created it.

Dr. Paxton-Fear noted that security issues multiplied as the demonstration project became more complex. Early vulnerabilities were addressed through software updates and dependency management, but additional weaknesses emerged in areas such as authorisation controls, business logic, and object-level access controls. As features were added and the AI system lost awareness of the broader context of the application, new risks continued to appear.

This reflects a broader challenge facing AI-assisted development. Large language models excel at producing code that satisfies immediate requirements. This challenge becomes particularly relevant as applications incorporate AI agents, external tools, APIs, and autonomous workflows, all of which expand the potential attack surface. OWASP GenAI Security Project. They are considerably less effective at maintaining a holistic understanding of an evolving application over time. Security weaknesses frequently arise not because the AI intentionally creates them, but because the complexity of the project exceeds the context available to the model at any given moment.

One of the most striking observations from the workshop was that many vulnerabilities are effectively invisible to the people building these applications. Traditional software developers generally possess at least a conceptual understanding of the technologies supporting their applications. They know which libraries are installed, which services communicate with one another, and where critical security decisions are made. Vibe coders often interact primarily with prompts and outputs. Their focus is on solving a business problem rather than understanding the architecture required to deliver the solution.

This difference in perspective has important security implications. According to Dr. Paxton-Fear’s analysis, only a small subset of common vulnerabilities can be directly attributed to actions taken by the vibe coder. Issues such as authorisation failures and business logic flaws may result from requirements provided by the user. However, many other risks originate from decisions made by the AI itself. These include vulnerable dependencies, exposed secrets, insufficient rate limiting, information leakage through debugging features, and various forms of injection vulnerability.

The challenge is compounded by the speed at which AI enables development. Rapid iteration allows applications to move from concept to deployment in record time, but security reviews do not always keep pace. The same tools that accelerate innovation can also accelerate the accumulation of technical and security debt. In many cases, vulnerabilities are not discovered until after an application has already been deployed or adopted by users.

For organisations considering the role of AI in software development, the lesson is not that these tools should be avoided. The productivity benefits are too significant to ignore, and the technology is already becoming deeply embedded within development workflows. Instead, organisations must recognise that AI changes who can build software without changing the underlying realities of software security.

A working application is not necessarily a secure application. Functionality demonstrates that software performs its intended task. Security requires a separate process of validation, testing, monitoring, and maintenance. Organisations looking to formalise that process may find the OWASP Top 10 Project and the broader OWASP GenAI Security Project useful starting points. As AI-generated applications become more common, understanding this distinction may prove to be one of the most important cybersecurity challenges facing businesses over the next decade.

• AI has dramatically reduced the barriers to software development, allowing non-developers to create functional applications.

• Functionality and security are separate concerns; software can work exactly as intended while still being vulnerable.

• As AI-generated applications increase in complexity, security weaknesses often become more numerous and more difficult to identify.

• Many vulnerabilities originate not from deliberate user actions but from limitations in how AI systems manage context across large projects.

• Vibe coders frequently lack visibility into the underlying technologies that make up modern applications, creating security blind spots.

• Organisations should focus on building security processes around AI-assisted development rather than attempting to prevent its use entirely.

• Security testing must become a standard part of the development lifecycle for AI-generated applications.

• “Hack Before You Launch” workshop materials, Dr. Katie Paxton-Fear

• OWASP Top 10 Web Application Security Risks: The industry-standard list of common web application vulnerabilities.

• OWASP Top 10 for LLM Applications 2025: Security risks specific to AI-powered systems and generative AI applications.

• OWASP GenAI Security Project: Guidance, tools, and community resources focused on securing generative AI systems.

• OWASP Top 10 GitHub Repository: Source material and supporting documentation for the OWASP Top 10 project.

Copyright (C) 2025 Packt Publishing. All rights reserved. 

Our mailing address is: 

Packt Publishing, Grosvenor House, 

11 St Paul's Square, Birmingham, 

West Midlands, B3 1RB, United Kingdom 

Want to change how you receive these emails? 

You can update your preferences or unsubscribe.