Synthetic Personas: Tireless, Opinionated, and Average

Nothing beats real human feedback on a product. Sitting with someone, listening, watching them actually use the thing you built. So much of what you learn in that room never makes it into words — the hesitation before a click, the eyebrow raise, the workaround they don't even realize they're doing. A transcript can't capture that, and I don't think it ever will.

That said, I've been playing with synthetic personas.

This is by no means new. Bain was writing about it back in 2025, and entire companies — Synthetic Users among them — exist to service the space. The common use case is in lieu of traditional market research, and I don't have a quarrel with the premise: if LLMs are trained to mimic human responses, they can produce reasonable approximations of how a segment might react to something. That makes sense to me.

But that's not where I've found them most valuable, and it's not the thing that surprised me.

Where they earn their keep, for me, is ongoing product feedback. I've built a handful of personas — most of them modeled on actual customers — and I prompt them to review my applications on a regular cadence. When they hit something that breaks their use case or trips up usability, they open a ticket. They never get tired. I can talk to them as often as I want. And the feedback is genuinely useful.

Here's the skeleton of a persona:

And here's a bug report ticket one of them actually filed:

The surprise — the thing I've been happiest about — is that this killed the idea bottleneck. When both ideas and development are cheap, the constraint moves somewhere else, and personas are how I keep the top of that funnel full. I've got AIs sending me GIFs of improvements they've implemented. I've got others pointed at narrow problems — one on UX, one on usability, one on conformance. I can have an agent walk the entire signup flow every single day — or any other flow I care about — and tell me the moment something feels off.

None of this means I trust it blindly, and I think the people selling it sometimes oversell it.

Synthetic personas have real edges. Left alone, they regress toward a plausible mean — the safe, average reaction rather than the sharp one. They're sycophantic by default; you have to work to get an LLM to actually tell you something is bad. And they reflect the model's training priors as much as they reflect your actual users, which is the part worth sitting with: a persona "based on" a customer is still, underneath, the model's idea of that kind of person.

That's exactly why they're good at taste and bad at gems.

In many ways I think about AI-generated feedback the same way I think about market research surveys. It's a great way to read the general taste of a segment. It's useless for finding the hidden gems — the unique, slightly weird insights that actually make a product great. Surveys never found those either.

A great piece of advice I once got: if you're ever confused about what you're supposed to be doing, pick up the phone and call a customer.

I've handed a proxy for that to my agents — but only a proxy. So the real question becomes: how do I know when to trust what the synthetic personas tell me? My rule, roughly, is that they're allowed to set the agenda but not to settle it. When a persona flags something that squares with how I already understand my customers, I move on it. When it flags something that doesn't square — or when the stakes are high enough that being wrong is expensive — I still pick up the phone myself. The synthetic feedback tells me where to look. A real customer tells me whether I'm right.

Fundamentally: if you're doing AI-driven development, you need workflows that encapsulate synthetic personas — not because they replace human feedback, but because they're how you keep a feedback loop running at the speed the rest of your stack now moves. Velocity that scales requires it.

In my next post, I'll get into how I see Dev and Product working together across AI-based workflows — the rapid build-and-feedback loop this all feeds into.