AWS Demonstrates Trajectory-Aware Post-Training for Security Agents

At [un]prompted 2026 Aaron Brown, Agentic AI Builder at AWS, presented a talk titled Trajectory-Aware Post-Training of Open-Weight Models for Security Agents and published an open-source pipeline that operationalizes post-training on model trajectories for cybersecurity tasks. The presentation emphasizes practical steps: environment instrumentation, trajectory capture, dataset curation, post-training (fine-tuning) on multi-step behavior, and specialized evaluation for agentic security scenarios.

The core idea is trajectory-aware post-training, which moves beyond single-turn fine-tuning by conditioning model updates on sequences of agent-environment interactions (trajectories). Brown framed this as a post-training stage applied to open-weight SLMs-models with accessible parameters that practitioners can adapt. The pipeline components described and demonstrated include:

• •Environment instrumentation to log state, actions, observations, and metadata for agent runs
• •Dataset curation to convert raw interaction logs into trajectory examples and context windows suitable for sequence modeling
• •Post-training procedures that reframe the objective to predict next actions or next observations within trajectory contexts rather than isolated completions
• •Evaluation harnesses and red-team scenarios to measure robustness, correctness, and safety of agentic behaviors

The implementation is explicitly open-source and includes reproducible environment setup, sample datasets, and evaluation scripts. Brown highlighted practical engineering considerations: deterministic logging for reproducibility, bounding context windows to avoid catastrophic forgetting during post-training, and integrating runtime monitors to intercept unsafe or undesirable agent actions.

This talk intersects two growing trends in applied AI security. First, the move from model-centric prompts to agentic, stateful systems makes sequential behavior the primary evaluation surface. Second, availability of open-weight models democratizes experimentation but increases both defensive capability and dual-use risk. By providing a documented pipeline, the work closes a gap between conceptual agent research and operational security tooling. For practitioners, the important shift is methodological: treat agent behavior as first-class training data and adopt trajectory-centric objectives instead of relying solely on prompt engineering or single-turn fine-tuning like classic supervised fine-tuning and RLHF.

Implementing trajectory-aware post-training changes how you collect data, how you instrument environments, and how you evaluate models under adversarial conditions. The pipeline lowers friction for reproducible research and for integrating agentic models into security workflows, such as automated vulnerability hunting, simulated attack/defense exercises, and blue-team automation. It also forces teams to reckon with monitoring, access controls, and governance because the same techniques that improve defensive agents can amplify offensive capabilities when misused.

Expect community forks that extend the pipeline to additional simulators and to structured environment APIs (e.g., CTF-style envs, network simulators). Look for benchmarking suites that compare trajectory-aware post-training against offline RL, behavioral cloning, and sequence-level RLHF on security tasks. Operational teams should evaluate access policies for open-weight models and integrate runtime safeguards into any production agent pipeline.