'We made a chip and it is fast': OpenAI's Jalapeno shows big gains in speed

OpenAI says its first custom inference chip can deliver more AI work per watt while reducing response times, as it moves towards deploying its own silicon alongside Nvidia and other accelerators

OpenAI said, on Wednesday, its first custom artificial intelligence (AI) inference chip, Jalapeno, has delivered significantly higher performance and energy efficiency than the commercial systems it was tested against, strengthening the company’s push to build more of its AI infrastructure in-house.

 

"Since announcing Jalapeno, our first custom inference chip, we’ve been testing it and the system around it. The results show a major advance: more intelligence from every watt and faster responses, delivering both higher throughput and lower latency in one architecture without sacrificing efficiency," the company said in a post on X.

First results show speed, efficiency gains

According to the company, Jalapeno delivered between 1.5 and 1.9 times more AI work per watt at peak throughput, meaning how much work it can handle in a given period, and 1.7 to 3.6 times lower end-to-end latency, or how quickly it can return a response, across three large language models. For highly interactive workloads, performance was between 2.1 and 4.1 times higher.

 

 

The chip was tested on GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T using InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. OpenAI compared Jalapeno with leading commercially available AI systems across different operating points, from high-throughput serving to highly interactive, low-latency use.

 

On Kimi K2.5 1T, the largest public model in the test, OpenAI reported about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. Jalapeno is rated at 700 watts, although its measured sustained power remained at or below 550 watts during the workloads tested.

 

As AI agents take on more complex tasks, they may need to make several requests in a row, making even small delays add up. OpenAI says Jalapeno is designed to reduce these delays by speeding up both the processing of a request and the generation of a response, while also reducing the time spent moving data around the system.

 

OpenAI has also designed the chip, memory, networking and software as a single system rather than treating the processor as an isolated component. The company describes this as a “full-stack advantage”, allowing it to design models, products, serving software, chips, memory, networking and systems together. The company sees Jalapeno as the beginning of a multigenerational platform, rather than a one-off processor. 

we made a chip and it is fast

— Sam Altman (@sama) August 25, 2026

Designed for faster AI agents

According to OpenAI, producing more useful AI work from the same amount of power and hardware could allow the company to serve more demand, enabling more use cases without costs rising at the same pace.

 

OpenAI plans to begin deploying it within its own computing infrastructure by the end of 2026. Production qualification, software development and performance validation across more models are still underway. The company also said it will continue to widely deploy accelerators from Nvidia and other partners for both training and inference workloads. That suggests Jalapeno is intended to supplement, rather than immediately replace, external hardware.