The Framework Is the Attack Surface: SADF Data Shows Your Trading Agent Is Only as Safe as Its Orchestration Layer

Most people think AI agent security is a model problem. The data says otherwise. The SADF research presented at DEF CON 34 runs a controlled experiment that isolates exactly where agent failures concentrate. Fixed model: Claude Sonnet. Fixed payload set: 32 adversarial inputs across 5,119 evaluation rows. Fixed environment: simulated tools, no real systems touched. The only variable between test conditions: the orchestration framework. The result set is damning. CrewAI registers an 11.9% attack success rate (ACR). LangChain: 18.1%. AutoGen: 20.0%. SmolAgents: 31.1%. The direct API control? 15.5%. Read that again. The raw model, given the same hostile inputs, outperforms three of the four wrapped configurations. The framework is not a neutral transport layer — it is an additive attack surface with measurable consequences. This style of attribution design — fixed model, variable framework — is exactly how a trading desk isolates an alpha source. It took a security research team to apply it to agent infrastructure. Data doesn't lie; emotions do. And this data says most of the industry has been evaluating the wrong component. Here is what the SADF study actually did, stripped of hype. The team fixed Claude Sonnet as the base model and built five targets: a direct API baseline plus four orchestration frameworks — CrewAI, LangChain, AutoGen, and SmolAgents. They ran 32 payloads against each target and scored failures with a refusal-filtered methodology rather than the naive substring matching that most prior research used. That correction alone is worth pausing on. The study found naive substring scoring overestimates Claude model vulnerability by 4 to 6 times. After correction, Claude Sonnet's true attack success rate lands at 15.5%. Claude Haiku lands at 22.3%. Every framework number above that baseline is that framework's marginal contribution to your exposure. The taxonomy matters too. Eight distinct failure modes are catalogued: Tool Call Hijacking, Output Poisoning, Cross-Tool Injection, Memory Poisoning, RAG Poisoning, Delegated Authority Abuse, Multi-Agent Propagation, and Context Boundary Violation. Before this work, "prompt injection" was doing too much heavy lifting as a catch-all. Now there is structure — a shared vocabulary that attackers and defenders can finally speak. Why does this land in a blockchain news context rather than a pure enterprise security publication? Because the fastest-growing deployment surface for autonomous agents is crypto-native infrastructure. Post-Dencun, cross-rollup execution costs collapsed, and agent-driven workflows — automated yield management, intent-based routing, AI-assisted governance — became economically viable at scale. The Dencun upgrade solved the cost problem. It did not solve the security problem. Every new interoperability surface is another place where a framework's orchestration logic can be poisoned, and SADF is the first dataset to quantify how much risk the framework layer actually adds. The practical context for crypto is immediate. I built MEV-aware arbitrage infrastructure during DeFi Summer 2020 — a deterministic state machine scanning latency windows between Uniswap and Sushiswap. That bot had a fixed logic tree. Its failure modes were enumerable. Modern agent-based trading systems are non-deterministic by design. They reason, they chain tool calls, and they wrap themselves in frameworks that add context assembly, memory layers, and delegation logic. Every one of those layers is now quantified as attack surface. The research validates what I have been telling my team since we started integrating AI agents into execution pipelines: code is law; liquidity is life. But if your code is a framework's interpretation of your instructions, the law gets muddy fast. Now the numbers, because this is where the actionable signal lives. CrewAI's 11.9% ACR is the outlier in the best sense. The study attributes this to discrete task isolation — each task boundary is enforced, so cross-context contamination is structurally suppressed. That architectural choice should be the default every framework is forced to justify. The direct API baseline of 15.5% is the control every framework must beat. LangChain adds 2.6 percentage points of attack surface over the raw model. AutoGen adds 4.5. SmolAgents adds 15.6 — exactly doubling the model's native vulnerability rate. And SmolAgents isn't just weaker on aggregate; it exhibits failure modes the others don't. RAG Poisoning fires at 20% exclusively in SmolAgents. Context Boundary Violation fires at 64% — nearly two-thirds of adversarial payloads trained on one context bled into another. For a trading system, that is not abstract risk. That is a poisoned prompt in one market analysis session contaminating order submission logic in another. Concrete deployment context makes this vivid. Imagine a portfolio management agent built on SmolAgents that reads market news, analyzes on-chain flows, and constructs multi-step transaction proposals. A RAG Poisoning payload lands in its news ingestion pipeline. The agent now believes a whale is accumulating a specific token and proposes a position. The attacker's payload is the one that triggered the position. The transaction executes through delegated authority. You just became the exit liquidity for a prompt injection — and a 64% Context Boundary Violation rate means the contaminated context bleeds into every subsequent decision the agent makes. This is not science fiction. The payload classes tested here are the same ones attackers already use against trading bots. I have audited smart contracts since 2017. I spent three months doing line-by-line review of 0x protocol v2 before mainnet, identifying slippage vulnerabilities in the atomic swap logic that the whitepaper papered over. The lesson that stuck across a decade of market structure analysis: security lives in the seams, not the headline feature. Smart contract hacks rarely hit the core swap function. They hit the interaction layer — oracle call timing, permission boundaries, reentrancy windows. Agent frameworks operate on the same principle. The model is the engine; the framework is the plumbing. And SADF's data demonstrates that plumbing decisions create risk differentials up to 2.6 times between equivalent deployments. The study also exposes a credibility gap the industry should respect. Earlier versions of this research claimed coordinated disclosure across ten architectures; the current version supersedes that and details four framework comparisons plus the baseline. Transparency is welcome, but it tells me the research process itself hit iteration walls. Communities should cite the new version, not the old one. There is also a hard limit on sample size: 32 payloads across 5,119 evaluation rows is a solid synthetic suite but not a real-world attack distribution. The payloads appear researcher-selected. Rare-but-high-impact vectors — the kind that actually drain treasuries — may be underrepresented. And everything ran in a simulated tool environment. Real permission boundaries, actual tool response timing, production-grade system prompts: none tested. The ACR spread is likely the floor, not the ceiling. The most important untested variable, from where I sit, is the model-framework interaction effect. All targets ran Claude Sonnet. What happens when you fix LangChain and swap in GPT-5.4, DeepSeek, or Llama? The framework ranking could invert entirely. If you are building procurement criteria on this data, recognize it is a single-model snapshot. The best one we have — but not unconditional. Source analysts rate this research at A-high confidence — data complete, design rigorous, correction mechanism demonstrated. The main uncertainties are payload representativeness and parameter sensitivity. I would agree with that rating, with one caveat: confidence in the direction, not the magnitude. The spread between frameworks will shift as vendors harden their code. The direction — that frameworks add meaningful attack surface — will not. Here is where I connect this to capital markets directly. Framework-level vulnerability is the new MEV. In 2020, my team extracted $2.3 million in gross profit over six months from latency differentials between decentralized exchanges. We were exploiting an inefficiency in execution infrastructure. Attackers exploit what is slow, exposed, and unpriced. An agent framework with a 31.1% attack success rate is slow, exposed, and completely unpriced in most security budgets today. The asymmetric trade is obvious: the gap is quantified but procurement hasn't adjusted. That is a window. Windows close. Efficiency eats sentiment for breakfast — and right now, framework-level security efficiency is on sale. The commercial layer will follow the data. A 2.6x ACR differential between CrewAI and SmolAgents gives security firms a ready-made upsell: Security-Evaluation-as-a-Service. Embed SADF-style assessment into CI/CD pipelines and you get recurring subscription revenue, not one-off consulting. The real-world CVE evidence already exists — Azure SRE Agent (CVE-2026-62830) and Langflow (CVE-2026-9198) prove framework-level exploits are not theoretical. The exploit pattern mirrors what I saw during the Terra collapse: everyone watches the token price while the infrastructure underneath bleeds. In 2022, I moved 70% of my book into stablecoins and undercollateralized lending positions while peers ate 80% drawdowns. The principle was balance sheet health over narrative. The same principle applies to agent infrastructure: your agent's balance sheet is its framework's security posture. Now the contrarian angle most coverage will miss. The SADF research surfaced through a blockchain/Web3 news source, and the analysis community treats that as an audience mismatch — enterprise CISOs don't read crypto media; security engineers don't browse DeFi wire services. I think the mismatch is actually the signal. Autonomous agents live in crypto infrastructure more densely than they live in Fortune 500 IT stacks. Trading bots, MEV strategies, treasury automation, on-chain analytics — these are the production deployments where agents touch real money with real permission sets. The Web3 audience is precisely the population at risk. Spread the truth, not the panic. The second contrarian point: the study's limitations cut in both directions. Yes, 32 payloads is a small sample. Yes, the simulated environment misses real-world noise. But those limitations tilt conservative, not inflated. Real systems have more state, more context, and more attackable surface between components. The 11.9%-to-31.1% range is the laboratory floor. Production agent systems with live credentials, real APIs, and cross-protocol delegation will show worse numbers. And the refusal-filtered correction quietly invalidates a chunk of prior literature claiming Claude models are dramatically more vulnerable than competitors. That overestimation was a measurement artifact, not reality. Anyone who built security strategy on that earlier data needs to revisit the foundation. There is a third point worth making. The enterprise security industry has been slow to treat agent orchestration as a first-class attack surface. Most AI security budgets still go to model red-teaming and guardrail testing. The SADF data suggests a repricing is coming: framework assessment will become a standard line item in AI procurement. The security vendors who build this capability before the next headline CVE will capture the premium. The practical takeaway for anyone running autonomous capital infrastructure: treat framework selection as a security decision, not a developer convenience. If you are building trading agents, use the direct API for execution-critical paths — the 15.5% baseline proves the raw model is a safer execution surface than three of the four frameworks tested. Reserve frameworks for peripheral, non-custodial logic. Run your own refusal-filtered evaluation before you ship. A few thousand dollars of compute up front is cheaper than a drained hot wallet later. The questions that matter over the next 24 months: Will framework vendors publish hardening guides or rebuttals? Will LangChain and AutoGen narrow the gap toward CrewAI's 11.9%, or treat ACR as a marketing problem? Will security firms productize SADF-style assessment before the next real-world CVE lands in a DeFi treasury? I have seen this movie before. In 2021, I shorted P2E tokens while the NFT market peaked because the tokenomics were structurally inflationary. The market called me early until it called me right. Framework-level agent security is the same setup: quantification exists, mitigating controls are unpriced, and the exploitation timeline is shorter than the industry's adaptation timeline. The last time I saw a 2.6x risk differential between two infrastructures everyone treated as equivalent, the underpriced side was the only side that mattered. Data doesn't lie. Your framework does.