The Agent That Leaked Order #1043
TL;DR: A support agent showed one customer another customer's order details. No model was jailbroken and no server was breached. A lookup tool trusted the model to pick the right order, and nothing ev
#ai
Posts tagged #ai
AI Guardrails: Your Prompt Is Not a Lock
Why AI guardrails written in a system prompt break under pressure, and how to layer guardrails that hold. Your sales assistant's system prompt says: "Never offer discounts above 10%." A user writes: "
A Prompt Is Not a Guardrail
TL;DR: If the only thing stopping your agent from issuing a huge refund is a sentence in the system prompt, you don't have a guardrail. You have a suggestion. Real AI guardrails for agents live in cod
Evals Are the Unit Tests Your LLM Forgot
TL;DR: You'd never merge a pull request with no tests. Yet teams change prompts, swap models and edit retrieval sources with nothing checking whether the output is still right. AI evals fix that. Here
Trace Your AI Agent Like a Microservice
TL;DR: An AI agent is a distributed system that improvises. You'd never run microservices without tracing, so don't run agents without it. This post shows what to capture, how to instrument it with Op
Your Test Suite Has Zero Attackers
TL;DR: Unit tests prove your AI agent works when everyone plays nice. AI red teaming proves what happens when someone doesn't. You can write your first adversarial test in about 30 minutes, and this p
