Aegis: Telemetry & Fallback API Gateway
Rajat Gade
Published
0
- Built a self-healing API gateway for agentic workflows and LLM-powered applications using Python and FastAPI for reliable AI-driven user interactions despite upstream failures.
- Architected a semantic caching layer utilizing Redis and local embedding models to intercept high-similarity queries, bypassing costly LLM inference entirely and heavily reducing p99 response latency.
- Orchestrated LLM infrastructure to support continuous AI-driven interactions, specifically engineering fallbacks to handle edge cases like rate-limiting and external API downtime.
- Engineered and orchestrated LLM infrastructure, including fallbacks for AI-driven interaction edge cases like rate-limiting and API downtime.
- Instrumented the gateway with rigorous Prometheus and Grafana telemetry to monitor cache hit rates and
system degradation, deploying data quality gates to reject prompt injection anomalies automatically.