03What you'll do
Invent and ship optimizations. Design new strategies across every layer of a request – prompt and context compression, tool pruning, file and dev-command trimming, semantic caching, cache-prefix stabilization – and take them all the way to the hot path. Each one has to cut tokens on real traffic without degrading output.
Prove every win. Build the evals and the token/quality instrumentation that separate a genuine saving from a plausible-sounding one. An optimization only ships when the numbers back it.
Live in the caching trade-offs. Provider prompt caches only pay off on byte-stable prefixes, and a naive rewrite can cost more than it saves. Knowing exactly when an optimization actually wins – deterministic transforms, per-session decision pins, semantic dedup – is a lot of the job.
Build and own the gateway that runs them. The real-time proxy that carries production LLM traffic – streaming, retries, backpressure – executing every optimization on the live request path.
Keep it fast and deploy anywhere. Defend a tight latency budget on the hot path, and package the whole thing so it installs cleanly into any customer's cloud – AWS, GCP, Azure, containers, Kubernetes – and upgrades without drama.
Own systems end to end. From first commit to code serving live customer traffic – plus the spend-visibility and governance product built on top – with a direct line to the founders.
You've built and run backend or infrastructure systems in production, owned them end to end, and care about latency, correctness, and observability.
You're comfortable in the request hot path – concurrency, streaming, tail latency, and failure modes are things you reason about by default.
You've shipped to cloud environments and can build something that installs cleanly in someone else's.
You're strong in TypeScript/Node, which is our stack, or fluent enough in a systems language that picking it up is quick.
You already build AI-native – coding agents and LLM tooling are part of how you ship.
You understand, or want to go deep on, LLMs, tokens, and how model APIs actually behave.
