Staff Full-Stack Engineer → ML Inference
Tony Lee
Professional Summary
Staff-level full-stack engineer with 10+ years of building production applications and APIs and leading cross-team architecture. Self-directed GPU labs cover vLLM serving on AWS, Triton kernels, and CUDA Graph replay, with repeated benchmarks, correctness checks, and documented trade-offs.
Work Experience
Staff Software Engineer
Sep 2024 - Present
DTEX Systems
- Designed services across user interfaces, APIs, data, and infrastructure. Packaged frontend services in containers to improve release stability.
- Led an OpenSearch Dashboards migration for the product launch, working with AWS/OpenSearch engineers to resolve deployment and environment issues.
- Automated local development and tests, cutting cycle time by 20%.
Senior Technical Lead
Aug 2019 - Apr 2024
Cisco Systems, Inc.
- Led frontend, backend, API, and integration architecture for Onboarding Experience. Enterprise customers completed onboarding in under 30 minutes.
- Made architecture decisions with teams in the US, Europe, and India. Improved build, release, and test automation to reduce new onboarding bugs.
- Led AngularJS-to-React migration to address security and maintenance issues. Added Cypress end-to-end tests, cutting test creation time by 50%.
Senior Software Engineer
Jan 2019 - Aug 2019
Tico Co., Ltd.
- Cut message loading time by 60% through frontend and API optimization. Reduced codebase size by 30% by reorganizing code into reusable modules.
Co-founder / CTO
Apr 2016 - Oct 2018
Popup Technology Co., Ltd.
- Built the core platform from scratch, led a 5-person engineering team, and shipped features that doubled revenue within 12 months.
ML Inference — Self-Directed (2025–Present)
Repeated benchmarks and correctness checks, including changes that did not improve speed. Results apply to tested hardware and workloads. Reports: tonylee.bio/work.
GPU Model Serving Lab
vLLM + AWS EKS- Deployment: Automated vLLM deployment and load tests on AWS Kubernetes (EKS), using Qwen2.5-0.5B on one NVIDIA T4 per server with monitoring.
- Capacity: Long prompts increased request waits while model processing time stayed steady. Load tests identified capacity limits.
- Response time: 95% of measured responses arrived within 28 seconds, down from 78, by limiting simultaneous requests in the test client. Trade-offs: 8% unserved requests and 7% less text output per second (in tokens).
Triton GPU Performance Lab
NVIDIA A10G + H200- 53% lower median runtime: Replayed recorded GPU operations (CUDA Graphs) in a simulated A10G text-generation step. Combining operations alone did not help.
- 5.9x faster RMSNorm: Used Triton to rescale model values versus separate PyTorch operations on A10G. Confirmed gains on H200; Nsight Compute found A10G limited by GPU memory transfer speed.
- 74% to 89% of cuBLAS speed: Tuned H200 matrix multiplication in 16-bit (FP16) tests, using NVIDIA's GPU math library as the baseline. Discarded slower configurations.
Skills
- Production: Languages / Web
- TypeScript · JavaScript · Python · Go · SQL · React · Node.js · GraphQL / REST
- Production: Delivery / Leadership
- Build / Release Automation · Cypress End-to-End Tests · Performance Optimization · Architecture Design · Cross-Team Leadership
- Lab: GPU / Serving
- PyTorch · Triton GPU Tuning · CUDA Graph Replay · vLLM Serving
- Lab: Testing / Monitoring
- Prompt / Cache Tests · Response Time (Latency) · Processing Rate (Throughput) · k6 Load Tests · Benchmarks · Prometheus / Grafana · GPU Monitoring (DCGM)
- Lab: Cloud / Deployment
- Kubernetes on AWS (EKS) · Terraform · AWS Compute / Networking
Education
M.E. Electrical Engineering
University of California, San Diego
Sep 2006 - Jun 2008
B.S. Electrical Engineering
University of Illinois at Urbana-Champaign
Sep 2003 - Dec 2005