Senior ML Engineer | Kimchi (LLM Inference Optimization)

The Senior ML Engineer will improve production LLM inference across throughput, latency, and KV cache utilization. Kimchi matches workloads to cost-efficient models and serving configurations on customer infrastructure, accounting for traffic patterns, GPU hardware, memory, and quantization. The work spans inference engines such as vLLM, SGLang, and TensorRT-LLM, as well as distributed systems and multi-GPU deployments.

The engineer will profile and optimize serving performance, tune batching and kernels, measure quantization quality, and develop caching and cold-start improvements. The role also includes setting the technical direction for benchmarking and technology choices, then communicating decisions through writeups and reproducible experiments.

Highlights

• Optimize LLM throughput, latency, and KV cache use.
• Work across inference engines, GPUs, and distributed systems.
• Measure quantization tradeoffs on real workloads.
• Lead technical direction for inference optimization.