Our work covers the full inference path: integrating serving engines with custom hardware, developing high-performance compute kernels, enabling efficient data movement, and driving models from early validation through production. We operate at frontier scale with large distributed models.
This is a ground-up effort with rapidly evolving hardware and software. We are looking for an IC who can write and optimize low-level code for custom hardware, validate model architectures end-to-end, build test and profiling infrastructure, and drive performance across the stack.
Key job responsibilities
– Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
– Implement and validate LLM architectures (decoder-only, mixture-of-experts) end-to-end – from PyTorch model definition through distributed execution on custom hardware.
– Integrate custom accelerator backends into open-source ML serving frameworks (vLLM, PyTorch), including scheduler extensions, memory management, and model parallelism.
– Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
– Profile and optimize inference workloads – identify bottlenecks, instrument critical paths, and drive latency and throughput improvements from simulation through hardware bringup.
– Own features end-to-end: from design through implementation, testing, and integration into the broader software stack.
– Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions.
Basic Qualifications
– Bachelor's degree or equivalent.
– 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience.
– Knowledge of computer architecture, operating systems, and parallel computing.
Preferred Qualifications
– Knowledge of Machine Learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques.
– Knowledge of ML frameworks including JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, and TensorRT.
– Experience in developing and deploying LLMs in production on GPUs, Neuron, TPU or other AI acceleration hardware.
