01
Rare events
Attacks are rare. How often an alert is right is set by the base rate, not by how good the detector looks on a balanced test set.
Free course · for data scientists · in preparation
DefendingAI agents
Agent security is a statistics problem. Build every defence yourself on a laptop, then learn what it becomes at fifty million calls a day.
Calls10,000Alerts101Attacks1Real0.98%
01The problem
You already know the hard part.
The central problems of securing AI agents have statistical twins. The course supplies the security half.
02
Operating points
Every threshold spends a false-alarm budget. You choose where to spend it, and measure it where the system acts.
03
Distribution shift
A detector tested on prose meets JSON, logs, diffs and SQL in production, and behaves differently.
04
Counterfactuals
Will a new policy help? Replay it over last month's traffic and run it in shadow before you enforce it.
02Try it
Four sliders. One uncomfortable number.
A detector that catches 99% of attacks and flags 1% of harmless calls sounds excellent. Move the sliders and see how many of its alerts are real.
Alerts that are real attacks
1%Alerts per day
Real attacks among them
Real attackFalse alarm
03What you build
Every component, with your own hands.
No security product, commercial or open-source, is used or taught. You build a miniature agent gateway with general-purpose open tools, one component per module.
- M01A small tool-calling agent, and the attack that breaks it
- M03A gateway that intercepts every tool call before it runs
- M04Prompt-injection detectors: patterns and a fine-tuned classifier
- M05Thresholds set from a measured false-positive budget
- M06An evaluation harness on data shaped like real tool calls
- M07A red-team harness with honest coverage reports
- M08A policy engine that allows, refuses, or holds for a person
- M09Behavioural baselines and sequence detectors
- M10A model loader that checks before it trusts
- M11Personal-data detection, redaction and erasure
- M12A signed, tamper-evident log, and replay of a policy change
- M13A policy learner with a shadow stage
04Scale
Build it small. Reason about it at scale.
Every module ends with what the same component becomes at millions of calls a day, thousands of agents and many teams, worked out from your own measurements.
On your laptop
40msYour detector scores one tool call in 40 milliseconds. You measured it, you understand every line, and it runs in 8 GB of memory with no GPU.
At 50 million calls a day
23coresThe same detector keeps about 23 cores busy around the clock. Where does it run? What happens to a call when it is slow or down? Who reviews its alerts?
05Curriculum
Fifteen modules and a capstone.
Each module ends at a measurement you produce and defend, not at a page you have read. About 105 hours, self-paced, in English and Hebrew.
I
Foundations
- 00Orientation: why agent security is a statistics problem
- 01Setting up your lab and your toy agent
- 02Security fundamentals for data people
- 03Anatomy of an agent and its attack surface
II
Detection
- 04Prompt injection, direct and indirect
- 05Detectors are classifiers: base rates, budgets, operating points
- 06Evaluation hygiene and distribution shift
- 07Adversarial robustness and red-teaming
III
Enforcement
- 08From score to decision: policy, holds and humans
- 09Behavioural and sequence anomaly detection
- 10The ML supply chain
- 11Data protection: personal data, redaction, erasure
IV
Operations
- 12Evidence: audit trails, replay and counterfactual evaluation
- 13Learning policy from traffic, safely
- 14Governance, audit and regulation for engineers
CapstoneBuild it, then break someone else's.Your own gateway for a new scenario, a scale memo, and a red-team report on another learner's work.
06Certificate
Free to learn. Paid to certify.
The whole course is open. The only paid part is the final exam, which a person reviews.
Free
The courseNo account needed to read.
- Every lesson, build and test
- English and Hebrew
- A starter kit and automated checks for each module
- Yours to keep and revisit
Paid
Final exam and certificateOptional. Price announced at launch.
- Your capstone, reviewed personally
- Detailed written feedback
- A signed certificate that anyone can verify
- A public page linking your reviewed work, if you choose
07Questions
Frequently asked.
Do I need a security background?
No. The course assumes Python, probability and machine learning, and teaches the security from the ground up.
What do I need to run it?
A laptop with 8 GB of memory, running macOS, Windows (with WSL 2) or Linux. No GPU is required. Installing Docker and setting up the lab is a module of its own.
Which products does it teach?
None. You build every component yourself with open, general-purpose tools, and wherever a tool is chosen the lesson names an alternative.
Is it really free?
Yes. Every lesson, build and automated check is free. Only the final exam, which a person reviews, and the certificate that follows are paid.
In which languages?
English and Hebrew. Code, commands and tools stay in English in both.
When does it open?
It is in preparation. Join the waitlist to hear when the first modules are published.
waitlist
08 Waitlist
One e-mail when the first modules are published, and occasional notes on how the course is going. Write to shimon@defendingaiagents.com; your address is used only for this course, and you can ask to be removed at any time.