Models can see; most products still expect text
Models can now see images, hear audio, and read documents natively — no separate OCR pipeline, no transcription service duct-taped on. The capability jump is real. But most products we've assessed aren't architecturally ready to use it, because they were designed around text in, text out.
Storage, cost, and showing what the model saw
The readiness gaps are concrete. Storage: where do the images live, at what resolution, for how long, under whose consent? A text log is kilobytes; an image trail is gigabytes with privacy implications. Processing: multimodal inference costs multiples of text — which images actually need model vision versus a cheap classifier pre-filter? Most "analyze this image" workloads are 90% obvious cases a small model handles and 10% genuinely hard ones. UI: how do you show the user what the model saw? For text RAG we show cited passages; the multimodal equivalent — highlighting the region of the image the answer came from — is still rare and badly needed for trust.
Use vision where a form field cannot stand in
The use cases that justify the complexity share a trait: the visual information has no good text substitute. Reading a hand-annotated engineering diagram. Assessing damage from photos. Verifying an ID document's security features. If the image could be replaced by a form field, it should be — multimodal is expensive overkill for structured data entry.
Build an image eval set before the feature ships
There's also a testing gap nobody talks about. Text evals are straightforward; multimodal evals need curated image sets with ground truth, and building them is slow, manual work. Teams ship multimodal features with far less eval coverage than their text features, which is exactly backwards given the higher stakes of visual misinterpretation.
Our advice: pilot multimodal on one high-value visual task with a proper eval set, keep the text pipeline as fallback, and measure whether the vision actually beats the cheaper alternative. Sometimes it transforms the product. Sometimes OCR plus a text model was fine all along.
When the visual case is real, we build it properly — eval'd, cost-controlled, and honest about where vision earns its keep.
Field note
The engineering-diagram pilot: field engineers photographed hand-annotated schematics, and the vision model extracted the annotations into structured work orders. The key architectural decision was the pre-filter — a small classifier that routed clean, printed diagrams to the cheap OCR-plus-text pipeline and only sent genuinely hand-annotated ones to the vision model. Result: 85% of volume at a tenth of the cost, with the expensive model reserved for where it earned its keep. And the eval set was 200 photographed diagrams with human-transcribed ground truth — slow to build, worth every hour, because visual misreads are silent in a way text errors aren't.
Related service AI products
