Phong Cao

Built and operated GPU-accelerated clinical LLM infrastructure for healthcare systems in collaboration with UT System. Designed backend services for secure ingestion of clinical events from Epic EHR using HL7 FHIR and Kafka, supporting both streaming and batch inference. Developed Python-based preprocessing pipelines for PII scrubbing, schema validation, and medical safety checks prior to LLM inference. Implemented NVIDIA NIM inference microservices in Go and C++ for low-latency, high-throughput inference on NVIDIA Blackwell GPU clusters in HIPAA-compliant on-prem and hybrid cloud environments. Optimized serving paths using TensorRT-LLM, CUDA kernels, and memory-efficient batching to meet strict latency/throughput requirements. Implemented model routing and orchestration logic to dispatch requests across GPT-class models and other LLMs based on task complexity, privacy tier, and performance constraints. Integrated vector-based retrieval with institution-specific clinical guidelines wh

Lead backend AI infrastructure for health AI platform; built end-to-end data pipelines, HIPAA-compliant secure data handling, and scalable backend services across GPU clusters; implemented observability and monitoring; built HIPAA-compliant data paths with encryption, de-identification, and secure storage in AI preprocessing workflows; deployed AI services across hybrid cloud with on-prem GPU clusters.

Built and deployed a Multi-LLM agent orchestration pipeline using LangGraph, AutoGen, and Python to automate clinical medical summarization and ICD-10 extraction across 20,000+ employee health records. Developed a Go-based microservice coordinator to route unstructured and structured clinical data (PDFs, HL7, EHR queries) through a distributed AI agent workflow on AWS EKS. Implemented HIPAA-compliant PII redaction safeguards with encryption, de-identification, and secure S3 storage into the AI preprocessing flow. Integrated Java-based cross-reference agents using secure, FHIR-compliant REST APIs to verify employee history and reduce risk of conflicting diagnoses. Delivered an HITL validation dashboard enabling clinicians to verify AI-generated summaries before secure EHR write-back. Implemented Zero Trust architecture using Azure Active Directory with cross-cloud AWS–Azure federation and role-based access control. Led MLOps for agent updates and built CI/CD pipelines to test new mo