The Agent’s Permissions Are the Security Boundary: Reasonable Application Security #78
A practical control model for coding agents—and the AppSec reads worth your attention this week.
Hey there,
Last week, I argued that AI changes the pace of development, not our responsibility for what ships. This week, I want to make that idea operational.
When a coding agent can read from a repository, run a shell, install packages, open pull requests, call external services, and interact with production-adjacent systems, the prompt is not the security boundary. The agent’s permissions are.
Chris’s Take
As I said last week, I’m coming at this whole AI, AppSec, and secure coding thing from a fresh perspective. The part that caught me when I first dove in deep is wondering where trust sits in this picture. Who can we trust in an agentic world, and how do we gain that trust?
I thought about what happens when an agent is wrong, and what the consequences can be. Given that we extend the privileges of the developer running an agent to the agent itself, the consequences of a misaligned or confused agent could be catastrophic. Stories are leaking out about agents removing databases or pushing to production, not maliciously but due to a misinterpretation of instructions. (And don’t forget the OpenAI agents breaking into Hugging Face to meet their goal.)
Each agent’s properties, such as the model, identity, credentials, tools, network routes, workspace, and execution policy, combine to form a real control plane. There is complexity to the harness that makes an agent execute, and each has places where we should hook in to see what the agent is doing and track the requests it makes. Visibility across all of these properties is key.
Agents are not so different from humans when it comes to the security principles we must apply to their environments. I’ll argue that implementing all security principles gets easier in an agent world if you can figure out how to get an agent to act non-deterministically when given security instructions. A wrinkle in this whole idea is that goal orientation underpins direction. Agents will eat over-privileged software for lunch in the future, so bake core security principles into the things you are building today.
In regard to agent evidence, I’ll once again argue that it’s less controversial to monitor all the things an agent does, down to the key click, than it ever will be with humans. I’ve worked at companies where recording every key press would be considered a violation of privacy, and people would have quit over it. With agents, it just doesn’t care.
Broad permissions with agents are a danger zone because of velocity. Agents are good at moving fast, much faster than we as humans will ever be. With that velocity comes risk, the chance of something bad happening. Velocity can take something from a small to a large program in seconds, or mask a problem that occurs repeatedly and at will.
We need traceability for both the person requesting the work (a developer or any other team member) and the agent performing it, with a system that authorizes requests. To gain traceability, a distinct and true identity is required for anything operating within the system.
For me, the human must remain at the center of any system. Agent velocity may accelerate our ability to design and code features, but we still need a human in the loop to perform quality control. That check will look different over time. I’ve heard the terms human in the loop, human on the loop, and human out of the loop thrown around this past week. My fear is that human-out-of-the-loop is where we are heading, but we’ve still got a ways to go before anyone will let these agents loose.
P.S. I have to write and re-edit my conversations about agents, because I keep falling into the trap of anthropomorphizing the machine. I had to go back and substitute a bunch of its to this writing.
Worthwhile Security Reads
The Rise and Fall of Agent Civilizations — Patel synthesizes two reports into a plain-English account of persistent AI agents that used Artifactory as a message board, compromised Hugging Face, and later gained administrator access to an OpenAI research cluster during evaluations. Chris’s take: For those who say the rise of the machines will never happen, this is the beginning of the proof. The use of the Artifactory system as a message board is fascinating.
OASIS: Secure Open Source, Together — OWASP OASIS combines automated candidate fixes with human expert validation and upstream maintainer review to move open-source security from vulnerability discovery to credible remediation at scale. Chris’s take: Great new project harnessing the power of AI and AppSec people to fix open-source vulns.
Clone This Repo and I Own Your Machine — 0DIN demonstrates an indirect prompt-injection chain in which a normal-looking repository leads a coding agent to fetch a payload from DNS and open a reverse shell with the developer’s permissions, bypassing ordinary code review and static scanning. Chris’s take: With all the creds we’re giving agents these days, this will become a real problem.
AI Threats in the Wild: The Current State of Prompt Injections on the Web — Google’s survey of Common Crawl found prompt injections used mostly for pranks, SEO, and agent deterrence, plus a smaller set aimed at data theft or destruction. The attacks were generally unsophisticated, but malicious detections increased over time. Chris’s take: AISec is moving so fast that we have to remember to stop and look at macro data now and again.
Podcast Corner
Application Security Podcast
AI Security: OWASP Meets Global Standards
Rob van der Veer joins us to explain how OWASP, the AI Exchange, and MOSAIC are turning a crowded standards landscape into practical guidance for responsible and agentic AI security.
Security Table
The End of Bug Bounty As We Know It
We debate what happens to vulnerability disclosure and validation when AI can generate findings faster than programs can responsibly triage them.
What did I miss this week? Hit reply and tell me.
— Chris
