A prototype has to be able to fail
The most valuable thing we build for clients is sometimes the thing we talk them out of building. And the cheapest way to have that conversation is a two-week prototype with one rule: it must touch real data and real users.
Most AI prototypes fail this bar. They're demos on synthetic data, shown to stakeholders who nod politely. They prove the technology works in principle — which was never in doubt — and teach nothing about whether it works here, for these users, on this data. A prototype that can't fail teaches nothing.
Week one tests the risky bet; week two decides
Our two-week format is deliberately constrained. Week one: wire the riskiest assumption to reality. If the bet is "the model can extract structured data from our documents," the prototype processes 200 real documents and we measure accuracy — not on a leaderboard, on their documents. If the bet is "users will trust AI drafts," five real users try it on real work and we watch. Week two: harden just enough to be honest — basic error handling, a simple UI — and run the eval that decides: build, pivot, or kill.
Killing the idea early is a successful outcome
The kill outcome is a success. We've had prototypes reveal that the data was unusable, that users hated the interaction, that the accuracy bar needed was unreachable with current models. Each of those discoveries cost two weeks instead of six months. Clients initially disappointed become the strongest advocates — because we spent their money finding truth, not building slideware.
Real data, a go criterion, and a written post-mortem
Three rules keep it honest: real data (no exceptions), a pre-committed decision criterion (what accuracy/adoption number means "go"), and a written post-mortem either way. The post-mortem for a killed idea is often the most valuable document we produce — it tells the client exactly what would need to change for the idea to work later.
Before committing to a full build, prototype the risk — two weeks of truth beats six months of hope.
Field note
The most valuable killed prototype: a "fully automated" applicant screening tool. Two weeks, real resumes, real recruiters. Result: the model was accurate, but recruiters hated it — they felt it turned hiring into button-pushing and didn't trust what they couldn't interrogate. Kill decision, documented. Six months later the same client greenlit the redesigned version: AI as ranked shortlist with quoted evidence, recruiter decides. It succeeded — because the prototype had discovered the real requirement wasn't accuracy, it was agency. The post-mortem said exactly that, in writing, which is why the second attempt started from the right insight instead of repeating the mistake.
Related service AI products
