Reasonable Application Security

The Claude Code Telemetry Switch Wasn’t the Whole Story: Reasonable Application Security #80

One telemetry path went where configured. Another still left the environment. Settings express intent; measuring the exits reveals what happened.

Hey there,

I was testing Claude Code telemetry in what I thought was an isolated environment when I noticed a connection that shouldn’t have been there. The OpenTelemetry exporter was pointed at localhost, but the process was also contacting Datadog. That led me to a question that matters far beyond one tool: when a setting says telemetry is configured, what exactly does it control?

Chris’s Take

I was running an experiment on September 6, 2026, in what I thought was an isolated Claude Code test environment. The configured OpenTelemetry exporter sent metrics to localhost, yet another connection appeared to go to Datadog. When a setting says telemetry is configured, what exactly does it control?

Two things were in play in this environment. The first was the opted-in OpenTelemetry. Using Claude Code’s default settings, I didn’t have any telemetry-disabling setting flags loaded. The second was the Datadog log shipping path. When I saw this traffic leaving the environment, I thought, “Uh oh, something is loose in my environment, something that I did not start.” A setting shows where an exporter should send data; it doesn’t prove that the process has no other network clients.

I was using Claude Code 2.1.236 for the beginning of the test on macOS. I varied the CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC flag on and off while using the same prompt. I saw traffic going to Datadog when the disable flag was unset, but not when it was set. DNS was the piece of data that showed me the traffic was going to Datadog. I then re-ran the test on my Linux machine.

Across five runs on my Linux box, I saw one POST per run, 74–89 records, 126–154 KB, 19 event types, and 246 distinct field names. The content contained platform and client details, feature flags, token/cost measurements, process resources, and tool success/failure metadata. No prompts, source code, file paths, usernames, hostnames, email addresses, or Git remotes appeared in the exercised paths.

I learned that CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 is inherited across supported shell launch paths. That said, a privileged process can relaunch without the variable, and future versions of Claude Code can change this behavior. So far, my takeaway is that a default setting expresses preference, an independent observation records what happened, and network policy can constrain what is possible.

My primary takeaway from this experiment is that we need to have better visibility into what agents are doing. We must build an inventory of metrics, logs, traces, crash reporting, and feature flags. We need a way to observe the agent’s outbound DNS queries and network connections and record this information for further analysis. To make the results of this experiment actionable, use a managed settings.json for your developer’s environments, and disable telemetry for everyone.

Turns out, I didn’t discover anything. Telemetry is part of how products work. What caused me to lose my breath for a second was the thought that an agent was working in my environment and had gone rogue in a way that I couldn’t explain. After the experiment, I realized everything was fine, and nothing was rogue. But the current state of the agentic world had me guessing for a minute or two. The Claude Code configuration named the information's destination, while observation revealed the Datadog path, and payload inspection prevented an unsupported conclusion.

Agent visibility is key, and I believe the only reliable way to know what leaves is to measure the exits.

Worthwhile Security Reads

Five reads on the security failures that emerge when AI agents can interpret untrusted instructions, install dependencies, generate code, and spend real money at machine speed.

  1. Hackers ran up a $600,000 AI bill after swiping API keys — A vibe-coded application failed open, exposed an AI API key, and allowed attackers to consume $600,000 in model credits over three weeks without detection. Chris’s take: And they say vibe-coded is dangerous, eh? Vibe coding requires a security architecture layer in which these decisions are set by a pattern or require an escalation path. Monitoring of environments is crucial for understanding when you’re spending money.

  2. Claude, Codex, and Hermes installed unowned code inside corporate networks — Coding agents followed references to nonexistent dependencies and installed researcher-controlled packages through trusted package managers inside corporate environments. Chris’s take: Audit your llms.txt, internal registry proxy, and documents to ensure that everything listed there really exists.

  3. IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests — Researchers hid malicious instructions across issue text, comments, and PDFs; 66.5% of test cases crossed both model- and agent-level guardrails in Cursor, Claude Code, and Codex Desktop on GPT-5.3/5.4 Codex and Sonnet 4.6. Chris’s take: Everything these days is a trust boundary, and whenever crossing a boundary, it’s time to laser focus on the crossing.

  4. Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the Wild — Across 304,362 verified AI-authored commits, more than 15% introduced at least one issue; AI changes introduced about 1.5 times as many security issues as they fixed, and nearly a quarter of tracked issues persisted. Chris’s take: I think there is a path forward for AI-authored commits. My process now is to use multiple models for code generation and review, and then let them battle it out over blocking and should-fix issues.

Podcast Corner

Application Security Podcast

Your AppSec Bottleneck Is a People Problem
Lisi Hocke explains how psychological safety, cognitive load, power sources, and community can make security champions programs sustainable—and why putting people first removes AppSec bottlenecks.

WatchListen

Security Table

When AI Escapes the Sandbox
The squad examines an AI model that escapes its test environment and publishes a malicious package to PyPI, then debates reward hacking, AI ethics, and the controls needed to contain agents.

WatchListen

Where to Find Chris

I’ll be at OWASP Global AppSec USA in San Francisco, November 5–6. I’m moderating a keynote debate and recording a live episode of The Application Security Podcast from the conference. If you’re there, come say hello—and join us for the debate and live recording.

What did I miss this week? Hit reply and tell me.

— Chris