Model-Agnostic Voice and Chat AI: Provider Routing with Fallback

This is a conversational AI platform for a confidential client. We set up its AI engineering from the start. We wrote its agent runtime, the code that runs its AI agents.

The platform does not depend on one AI provider. One component selects the language model, speech-to-text and text-to-speech providers. All 3 have ordered backup lists.

We chose not to use an off-the-shelf LLM gateway, a service between the app and the model providers. Its main feature is automatic retry on another model. We kept the retry choice in the agent runtime, which sees facts that a gateway cannot see.