The Duck Test for AI Assistants: Why "You Are an Expert ___" Isn't Enough

"If it walks like a duck, looks like a duck, and quacks like a duck, then it's a duck." Unless you built it with a prompt — in which case it only quacks.

If you've set up a Custom GPT, a Claude Project, a Gem, or a bot in a no-code tool, you've written the line: "You are an expert screenwriter." It's the natural first move — and it does less than you'd think. Good news: the fix takes no code. The prompt is your product, and everything below happens inside the tools you already use.

Do this

A build checklist. Work top to bottom; skip nothing.

1. Keep the persona to one line. "You are a patient math tutor." Then stop.
2. Tell it what it's trying to win at, not just its title. ("Win at the student understanding, not at giving the answer.")
3. Ask it for the steps it should take before it answers. Start with skeleton.
4. Define what's good and bad with example. Don't describe them — show them.
5. Say exactly what "done" looks like — the format, the length, whether it should recommend or just list.
6. Attach your real material: knowledge files, raw screenshots, reference docs, past examples, your writing samples.
7. For plain factual lookups, tell it to answer straight, no character.
8. Add a feedback loop: propose, ask how it landed, adjust, go again. If corrected, change the plan — don't defend it.
9. Test on messy, half-formed questions — the ones real users actually type — not clean demo prompts.

That's the whole method. The rest of this post is why each piece matters.

Why it works: the four tests

Split the "expert" you're trying to conjure into four layers. A persona line only covers the first two — the easy ones. The other two are what the checklist above is really building.

1. Looks like a duck — the title

"You are an expert screenwriter" makes your bot claim the role. That's it. Claiming expertise doesn't create it, any more than a name tag makes someone a surgeon. It costs one line, so spend one line — and no more. (Checklist item 1.)

2. Quacks like a duck — the voice

This is the one thing a persona reliably delivers: your bot starts sounding the part, using the right vocabulary and tone. Useful for creative and stylistic work. But here's the trap — researchers have found that piling on an "expert" persona can make answers less accurate, because the model works harder at sounding right than at being right. That's exactly why the checklist tells you to drop the character for simple factual lookups (item 7): keep the costume for judgment and style, take it off when you just need a correct fact.

3. Walks like a duck — the thinking

This is where real quality lives, and a persona does nothing for it. An expert isn't good because of their title; they're good because of how they move through a problem and what they're optimizing for. So you spell both out. Telling your bot what it's trying to win at (item 2) is what lets it make trade-offs — and making trade-offs is what judgment actually is. Writing out the steps (item 3) turns a vague vibe into a repeatable routine. If you can't name the steps, ask the model itself — "What are the five questions a great travel agent asks before planning a trip?" — then curate the answer back into your prompt.

4. Works like a duck — the delivery

The difference between a know-it-all and a genuinely helpful assistant is what it hands you. A bot that only quacks produces this: "There are several great destinations to consider! Italy offers rich history, while Japan blends tradition and modernity…" A bot that works produces this: "Given your October dates and mid-range budget, I'd do 5 nights in Lisbon over Rome — better weather that week, cheaper flights, less crowded. Day-by-day below. Risk: one museum is closed Mondays, so I put it on day 2."

You get the second one by defining "done" and showing an example (items 4 and 5) — a good/bad pair teaches far more than the word "good." Two things make delivery even better than a fancier prompt: give it a pond (item 6 — real material to work from beats beautiful wording working from nothing), and test it like a real user (items 8 and 9). Watch for the tell: does it recommend and do, or does it lecture? If it drifts into options with no opinion, tighten the delivery instructions.

A practical synthesis of current thinking on prompting and AI assistant design — a reusable pattern, not a formal study.