Events are a worst-case setting for a live model
Events are AI's worst-case scenario: hard deadlines, no do-overs, hundreds of moving pieces, and everything happening live. Building Summit Connect — an event platform with AI-assisted operations — taught us things about shipping AI that no lab benchmark ever could.
Reliability and a one-tap interface come first
Lesson one: reliability beats intelligence. An event coordinator doesn't need a brilliant AI; they need one that works at 8pm on event day when the Wi-Fi is dying. We designed every AI feature with offline-tolerant fallbacks and pre-computed outputs for the critical paths — attendee check-in lists, session schedules, speaker briefs generated the night before, not live. The fanciest model is worthless if it needs conditions the venue can't guarantee.
Lesson two: the users are not technical and the moment is not calm. Interfaces for event staff need to work for someone holding a radio in one hand: huge touch targets, one-tap actions, zero configuration. We stripped every AI feature down to a single button with a clear outcome — "generate today's briefing," "draft the delay announcement" — because nobody is writing prompts during a keynote.
Rehearsal is the eval an event can actually run
Lesson three: rehearsal is the eval. You can't A/B test an event. So we rehearsed: full simulated run-throughs with the actual data, the actual staff, the actual timeline. Every AI output got reviewed in rehearsal, which caught the failure modes no eval set would — like the speaker-bio generator that worked perfectly except for speakers with non-Western name orders.
The feature that gets renewed is the invisible one
Lesson four: the highest-value AI at events is invisible. Not a chatbot for attendees — automated reconciliation of registrations vs. check-ins, real-time capacity alerts, post-event reports assembled from a dozen sources. The glamorous features get the demo; the boring ones get the renewal.
Events taught us that production AI is mostly about everything around the model. Summit Connect is where we learned to build for the worst day, not the demo day.
Field note
The rehearsal that earned its keep: full run-through, real data, two days before the event. The speaker-bio generator mangled every speaker with a non-Western name order — flawless on the eval set, which was all Western names. Caught in rehearsal, fixed in an hour, invisible on event day. The other save: pre-computed briefings. Every "AI-generated" artifact the staff needed on event day was generated the night before and cached — the live AI features were conveniences, never dependencies. When the venue Wi-Fi degraded mid-afternoon, nobody noticed, because nothing critical needed the network. Build for the worst day.
Related service AI products
