Climb one rung at a time
Teams routinely reach for fine-tuning when prompting would do, and for RAG when a better prompt would do. There's a ladder of escalating cost and complexity, and you should climb it one rung at a time — most problems are solved on the lower rungs.
Prompt first, then retrieval
Rung one: prompt engineering. Better instructions, few-shot examples, explicit output schemas. This solves a surprising fraction of "the model isn't good enough" complaints, costs nothing to try, and takes hours. If you haven't spent a serious day on the prompt with evals measuring the result, you haven't earned the right to complain about the model.
Rung two: RAG. When the problem is knowledge — the model doesn't know your docs, your products, your policies — retrieval beats training. It's cheaper, it's updatable (edit a document, not a training run), and it provides citations. Most enterprise AI is knowledge problems wearing capability costumes.
Fine-tune for behavior, not for facts
Rung three: fine-tuning. Reach for it when the problem is behavior, not knowledge: you need a specific tone, format, or reasoning pattern applied consistently across thousands of outputs, and examples can't fit in context. Fine-tuning teaches style and procedure; it's a poor way to teach facts (they go stale, and the model confabulates around them anyway). It also demands eval discipline — a fine-tuned model needs regression testing against everything it used to do, because training shifts behavior in ways prompts don't.
Sometimes change the product instead of the model
Rung four, the one nobody mentions: change the product. Sometimes the right answer isn't a smarter model but a different interaction — constrained choices instead of free text, a review step instead of autonomy, a smaller problem scope. Product design solves whole classes of "AI limitations" without touching the model at all.
Climb in order, measure at each rung, stop when the evals are green. The teams that skip rungs pay for it twice: once in complexity, once when they have to maintain what they built.
We apply this ladder explicitly in our AI consulting engagements — cheapest effective rung first, always.
Field note
A client arrived convinced they needed fine-tuning for contract clause extraction — quoted $80k and three months by another vendor. We spent a day on the prompt with 12 few-shot examples and hit 94% on their eval set; the remaining errors were retrieval misses, fixed by better chunking. Total cost: under a week. Fine-tuning would have "worked" too — eventually, expensively — while locking them into a retraining cycle every time their clause library changed. The ladder isn't about being cheap; it's about not buying complexity you don't need. Climb only when the rung you're on demonstrably fails.
Related service AI products
