Hardware - San Francisco
About the Team
OpenAI’s Hardware organization develops AI-native silicon and system-level solutions for the unique demands of advanced AI workloads. Building on efforts like Jalapeño.
About the Role
You will build the model runtime within the inference engine that executes complex, frontier models at scale on OpenAI’s custom silicon. The runtime will sit between models running on the hardware and the upper layers of the cluster serving software stack, translating demanding inference workloads into efficient execution while optimizing for throughput, latency, utilization, and reliability.
You will work across model architectureance in production.
In this role, you will:
-
Design and implement the LLM inference runtime for frontier models running on custom silicon.
-
Build schedulingance inference.
-
Develop distributed execution strategies across chips, hosts, and racks, including model partitioning, communication, and synchronization.
-
Optimize end-to-end latency, throughput, memory efficiency, and hardware utilization across diverse model architectures and serving workloads.
-
Partner with kernelance bottlenecks across the stack.
-
Enable new model featuresats, and hardware capabilities in a reliable production runtime.
-
Create profilingance-modeling tools that make runtime behavior measurable and actionable.
-
Debug complex correctnessance, and reliability issues spanning model code, runtime software, communication layers, and hardware.
-
Turn workload insights into clear requirements for future generations of silicon and system architecture.
You might thrive in this role if:
-
Have strong systems programming experience in C++ance-oriented environments.
-
Have built or optimized runtimes, distributed systems, compilers, kernels, model-serving infrastructure, or adjacent systems software.
-
Understand modern LLM inference, including prefill and decode behavior, batching, KV-cache tradeoffs, and model parallelism.
-
Can reason quantitatively about latency, throughput, compute intensity, memory bandwidth, communication, and utilization.
-
Are comfortable profiling and debugging performance across multiple layers of a hardware-software stack.
-
Can design clean abstractions while retaining the low-level control needed to extract performance from specialized hardware.
-
Work effectively across model, systems, compiler, kernel, and hardware teams to drive ambiguous technical problems to closure.
-
Care about production quality, including correctness, observability, reliability, maintainability, and graceful behavior at scale.
To comply with U.S. export control laws and regulations, candidates for this role may need to meet certain legal status requirements as provided in those laws and regulations.
About OpenAI
OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core the full spectrum of humanity.
We are an equal opportunity employeration, or other applicable legally protected characteristic.
Background checks for applicants will be administered in accordance with applicable lawation technology systems and related data security obligations.
To notify OpenAI that you believe this job posting is non-compliant. No response will be provided to inquiries unrelated to job posting compliance.
We are committed to providing reasonable accommodations to applicants with disabilities.
At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
Compensation
$266K – $445K + Offers Equity