Software Engineer, GPU Inference

About the role

The role is a senior software engineer focused on productionizing and optimizing Cerebras’s GPU‑based inference stack, integrating custom inference APIs, the vLLM runtime, AMD ROCm software, and rack‑scale GPU infrastructure to deliver reliable, numerically correct, and high‑performance model serving.

About the company

Cerebras Systems builds the world’s largest AI chip—a wafer‑scale engine 56× larger than GPUs—that delivers industry‑leading training and inference speeds, over 10× faster than GPU‑based cloud services. The company partners with leading model labs, global enterprises, AI‑native startups and has a multi‑year partnership with OpenAI to deploy massive inference workloads.

Requirements

• 5+ years of software engineering experience with substantial individual‑contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong programming ability in C++ and Python, including multithreading, concurrency, memory management, and performance‑sensitive software.
• Hands‑on experience with a high‑performance model‑serving framework such as vLLM, SGLang, TensorRT‑LLM, Triton Inference Server, or equivalent.
• Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.
• Experience debugging distributed systems across multiple layers rather than treating the serving framework or accelerator runtime as a black box.
• Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency‑sensitive services in production.
• Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.
• Strong communication and technical leadership skills with a demonstrated ability to drive ambiguous cross‑functional projects to completion.
• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or related discipline, or equivalent practical experience.
• Experience with AMD Instinct accelerators and the ROCm ecosystem (HIP, RCCL, rocprofiler, AMD SMI, AITER, hipBLASLt, Composable Kernel, etc.).
• Deep CUDA experience demonstrating ability to transfer GPU systems knowledge across accelerator platforms.
• Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT‑LLM, or another open‑source ML systems project.
• Experience optimizing prefill‑heavy or disaggregated prefill/decode inference architectures.
• Understanding of KV‑cache transfer, prefix caching, continuous batching, chunked prefill, request scheduling, and memory‑aware admission control.
• Experience with multi‑GPU and multi‑node inference, including tensor parallelism, pipeline parallelism, expert parallelism, RDMA, collective communication, and failure handling.
• Experience optimizing Mixture‑of‑Experts or multimodal models.
• Knowledge of GPU kernel optimization, operator fusion, graph capture, attention kernels, GEMM tuning, and communication/computation overlap.
• Experience with reduced‑precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4, including validation of their numerical and model‑quality effects.
• Experience building numerical‑comparison, determinism, model‑validation, or performance‑regression test systems.
• Experience collaborating directly with accelerator vendors, framework maintainers, or open‑source communities.