In episode 208 of Cybersecurity Where You Are, Sean Atkinson breaks down secure harness engineering for agentic artificial intelligence (AI). Noting the risks posed by AI agents acting without oversight, he highlights "episodes" as auditable elements that help to provide feedback, and he walks through 15 controls spread across five stages as a way to engineer an AI harness with security in mind. He also differentiates secure harness engineering from prompt engineering, guardrails, frameworks, logs, and universal wrappers for AI agents.
Here are some highlights from our episode:
- 01:23. The need for a control system in light of agent escape incidents
- 02:19. Analogy of a financial analyst to understand why separation of duties is important
- 02:57. Revolution and then evolution: How AI models have changed exponentially since 2022
- 04:54. Where secure harness engineering has a capability
- 05:42. Making decisions traceable: The need for an evidence chain to build controls effectively
- 07:15. Smartphones: A parallel for understanding the value of each new AI model
- 11:20. Engineering controls, not standard policies for all models
- 12:22. Secure harness engineering vs. prompt engineering
- 12:51. Models' evolving role: How it invites new security questions to be posed
- 15:16. AI agents without oversight: The insider persona with which defenders need to contend
- 16:02. The need for determinism and an effective diagram to build for it
- 18:22. Definition of secure harness engineering
- 19:36. Episodes and the importance of log assessment in mediating model capability
- 20:42. Reality verification: A feedback loop that gets into the concept of forensic assessment
- 21:27. Questions to ask as you get into an AI agent's evidence chain
- 22:56. Moving through intent, authority, reasoning, synthesis, decision, and enforcement
- 25:38. Side effects: What they are and how they actualize risk