Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

Senior Software Engineer with 10+ years of experience building scalable, distributed systems and cloud-native infrastructure across GCP and Azure. Strong background in high-performance microservices, event-driven architectures, and real-time data pipelines using Go, Python, Java, and C++. Proven track record delivering production-grade platforms that power AI and ML systems, including LLM-integrated workflows, automation frameworks, and backend infrastructure across NVIDIA, Google, and Tesla. Focused on driving reliability, API performance, and cross-functional collaboration in mission-critical environments.…Senior Software Engineer with 10+ years of experience building scalable, distributed systems and cloud-native infrastructure across GCP and Azure. Strong background in high-performance microservices, event-driven architectures, and real-time data pipelines using Go, Python, Java, and C++. Proven track record delivering production-grade platforms that power AI and ML systems, including LLM-integrated workflows, automation frameworks, and backend infrastructure across NVIDIA, Google, and Tesla. Focused on driving reliability, API performance, and cross-functional collaboration in mission-critical environments.

Phong Cao

Full Stack Developer, Back-End Developer, Front-End Developer, +5





Senior Software Engineer with 10+ years of experience building scalable, distributed systems and cloud-native infrastructure across GCP and Azure. Strong background in high-performance microservices, event-driven architectures, and real-time data pipelines using Go, Python, Java, and C++. Proven track record delivering production-grade platforms that power AI and ML systems, including LLM-integrated workflows, automation frameworks, and backend infrastructure across NVIDIA, Google, and Tesla. Focused on driving reliability, API performance, and cross-functional collaboration in mission-critical environments.…Senior Software Engineer with 10+ years of experience building scalable, distributed systems and cloud-native infrastructure across GCP and Azure. Strong background in high-performance microservices, event-driven architectures, and real-time data pipelines using Go, Python, Java, and C++. Proven track record delivering production-grade platforms that power AI and ML systems, including LLM-integrated workflows, automation frameworks, and backend infrastructure across NVIDIA, Google, and Tesla. Focused on driving reliability, API performance, and cross-functional collaboration in mission-critical environments.

Available to hire

Senior Software Engineer with 10+ years of experience building scalable, distributed systems and cloud-native infrastructure across GCP and Azure. Strong background in high-performance microservices, event-driven architectures, and real-time data pipelines using Go, Python, Java, and C++.

Proven track record delivering production-grade platforms that power AI and ML systems, including LLM-integrated workflows, automation frameworks, and backend infrastructure across NVIDIA, Google, and Tesla. Focused on driving reliability, API performance, and cross-functional collaboration in mission-critical environments.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Intermediate

Language

Aragonese

Fluent

Javanese

Advanced

Work Experience

Senior Software Engineer, AI Infrastructure & Backend Systems at NVIDIA

May 1, 2024 - Present

Built and operated GPU-accelerated clinical LLM infrastructure for healthcare systems in collaboration with UT System. Designed backend services for secure ingestion of clinical events from Epic EHR using HL7 FHIR and Kafka, supporting both streaming and batch inference. Developed Python-based preprocessing pipelines for PII scrubbing, schema validation, and medical safety checks prior to LLM inference. Implemented NVIDIA NIM inference microservices in Go and C++ for low-latency, high-throughput inference on NVIDIA Blackwell GPU clusters in HIPAA-compliant on-prem and hybrid cloud environments. Optimized serving paths using TensorRT-LLM, CUDA kernels, and memory-efficient batching to meet strict latency/throughput requirements. Implemented model routing and orchestration logic to dispatch requests across GPT-class models and other LLMs based on task complexity, privacy tier, and performance constraints. Integrated vector-based retrieval with institution-specific clinical guidelines wh

Senior Software Engineer, AI Infra & Backend Systems at NVIDIA

May 1, 2024 - Present

Lead backend AI infrastructure for health AI platform; built end-to-end data pipelines, HIPAA-compliant secure data handling, and scalable backend services across GPU clusters; implemented observability and monitoring; built HIPAA-compliant data paths with encryption, de-identification, and secure storage in AI preprocessing workflows; deployed AI services across hybrid cloud with on-prem GPU clusters.

Senior Software Engineer at Tesla

May 1, 2023 - February 1, 2024

Built and deployed a Multi-LLM agent orchestration pipeline using LangGraph, AutoGen, and Python to automate clinical medical summarization and ICD-10 extraction across 20,000+ employee health records. Developed a Go-based microservice coordinator to route unstructured and structured clinical data (PDFs, HL7, EHR queries) through a distributed AI agent workflow on AWS EKS. Implemented HIPAA-compliant PII redaction safeguards with encryption, de-identification, and secure S3 storage into the AI preprocessing flow. Integrated Java-based cross-reference agents using secure, FHIR-compliant REST APIs to verify employee history and reduce risk of conflicting diagnoses. Delivered an HITL validation dashboard enabling clinicians to verify AI-generated summaries before secure EHR write-back. Implemented Zero Trust architecture using Azure Active Directory with cross-cloud AWS–Azure federation and role-based access control. Led MLOps for agent updates and built CI/CD pipelines to test new mo

Senior Software Engineer, AI Infra & Backend Systems at Google

November 1, 2021 - March 1, 2023

Project: Google Gemini – Multimodal Foundation Model Platform (Gemini II). Expanded TPU pod usage, distributed inference, orchestration, and MLOps; implemented privacy-preserving data handling and integrated with Vertex AI; supported on-prem and cloud deployments across Austin, TX (Nov 2021–Mar 2023).

Senior Software Engineer, AI Infra & Backend Systems at Google

December 1, 2019 - November 30, 2021

Project: Google Gemini – Multimodal Foundation Model Platform. Drove end-to-end ML infrastructure for Gemini across Mountain View, CA (Dec 2019–Nov 2021) and Austin, TX (Nov 2021–Mar 2023). Built large-scale data pipelines, inference orchestration, and Vertex AI integrations for multi-modal LLMs; collaborated with DeepMind/Google Brain teams to enable robust, scalable deployment.

Software Engineer at Arista Networks

May 1, 2015 - December 1, 2019

Developed Cloud Vision platform capabilities for software-defined networking and analytics across hybrid and multi-cloud environments. Designed and implemented high-throughput backend services in Go to ingest, normalize, and process streaming network telemetry from physical and virtual EOS devices across on-prem and public cloud. Built and scaled Network Data Lake (NetDL) pipelines capable of processing millions of state changes per second to enable near real-time visibility for configuration, routing, and health signals without performance degradation. Developed Kubernetes-deployed cloud-native microservices with horizontal autoscaling, fault isolation, and strong consistency guarantees. Implemented policy-driven orchestration workflows translating Cloud Vision UI intent into deterministic device configurations using Python automation and EOS config templates. Integrated Zero Touch Provisioning (ZTP) pipelines for automated device onboarding, configuration, and patching to reduce op

Software Engineer at Google

May 1, 2015 - December 1, 2019

Worked on the Gemini multimodal foundation model platform (DeepMind/Google Brain). Supported the transition from Mountain View to Austin for TPU-based scaling, contributing backend inference and distributed systems required to power Gemini Apps, Workspace, and Vertex AI. Designed and maintained Gemini’s distributed inference pipeline for Pro, Flash, and Ultra models with expert routing (MoE), multimodal input handling, and low-latency global production responses. Built autoscaling, high-throughput model-serving systems using Go, C++, and Kubernetes across TPU pods to provide reliable access to Gemini APIs via Vertex AI with strict latency SLAs. Integrated Gemini into Google Workspace and developer tools, exposing LLM-backed summarization, document generation, and code assistance through secure REST and gRPC interfaces. Orchestrated multimodal data ingestion pipelines (text, audio, code, image) using Dataflow, Pub/Sub, and Cloud Storage. Collaborated with Google Research, Privacy, a

Senior Software Engineer at Google

May 1, 2015 - December 1, 2019

Cloud Vision & Cognitive Networking Platform (Any Cloud Initiative): software-defined networking and analytics platform designed to shift enterprise networks from device-level to network-wide automation; includes real-time telemetry, data processing pipelines, hybrid cloud deployments, and SOC2/HIPAA-like security considerations.