It's Monday, 8:47am. You're the new prompt-engineering lead at Sentinel — a medical-scribe SaaS quietly running inside about 300 US clinics. Your predecessor Daria rage-quit two weeks ago and left behind a Notion folder of prompt files with names like `v3_final_FINAL_works_pls_no_touch.md`. Last Wednesday, two transcripts hallucinated a medication dose that was never spoken in the visit. Legal has been in Mateo's office for an hour every morning since. The CTO has a board meeting in 5 working days and the CFO wants cost-per-transcript on a dashboard by Friday. Your only help is Kai — a junior who has read every Anthropic and OpenAI doc twice and still can't tell a working prompt from a confident-looking one. In the next 60 minutes you'll do the one thing Daria never did: write the eval before the prompt. Everything else in this course depends on you internalizing that move now.
You'll walk out able to
- Diagnose a prompt session as 'no signal' even when the model looks confident
- Write an eval set before writing a prompt — with testable, specific cases
- Read a prompt diff and identify which line caused a hallucination regression
- Pick a structured eval-shaped prompt over a vibey eloquent one — and say why
- Choose the reasoning-effort dial that fits cost + latency + quality at once
- Run a spec linter on a real medical-scribe prompt and commit to one improvement
Your first task
Kai is euphoric. Watch the session.
Kai pings you on Slack at 8:51am: 'I think I got the new transcript prompt working — Claude wrote a really solid summary on the first try. Want to see?' He shares his terminal session. Mateo is standing behind him. You read carefully.
What you'll answer
Read Kai's session below. What's the one diagnosis you give him?
Free · no card · we save your progress
Save your progress & earn the certificate
You've seen the opening — keep going free. 7 tasks · auto-graded · finish in 60 min and the certificate is yours. Free forever · no card · cancel any time.
We'll also email you a magic link so you can sign in on your phone. No password, ever.
Already started? Sign in with magic link instead.