OpenTPU – An open-source AI accelerator, developed by AI

fsbonetto

After using AI to develop risc-v CPU cores, the same technique was used for developing openTPU. An open source AI inference engine. It's able to run most of the modern models like Qwen 3.5, Gemma 4, and many others. The TPU started able to produce only a few tokens per second and trough a recursive self improvement loop got to 80+ tok/sec on the smallers models.

vatsachak

I feel like there is a lot to be gained from an experienced user pointing an LLM in a tasteful direction.

skybrian

This seems to be running on an FPGA board that costs ~$300? Anyone know more about the hardware?

pcarolan

Really dumb question from a software guy. Why aren't the labs burning their frontier models into chips already? Seems like the performance gains and cost per request would be worth it. That said, I understand neither the economics nor the physical challenges to doing this.

rfgplk

Yep, 99.9% of people are completely oblivious to what LLMs can do. Just wait until the next gen of CPUs/GPUs designed by LLMs start coming out (fyi chip development tools have advanced centuries in the last few months) and you'll start seeing exponential gains in hardware.

xg15

"Recursive self-improvement will kill us all!" Also: Here is our recursive self-improvement hard at work...

athrowaway3z

I haven't really dug into the results yet, but my guess is that a SOTA model has been able to produce an accelerator that runs a model since around December. The obvious next step is to get enough memory throughput to run that SOTA model itself so that it develop its own hardware. But perhaps the more interesting question is this: Can an AI be given a big FPGA and design a model architecture that takes advantage of the fabric being reconfigurable.

bitwize

Colossus is building Colossus II.

srameshc

This post brings me to question "What does it mean to be a software developer in future" ?

AnimalMuppet

Can anyone comment on the performance of this hardware? How does it compare to state of the art, human-designed hardware? Is this actually an improvement? (To get to recursive self-improvement, you first have to improve at all.)

gfalcao

The birth of SkyNet

rcarmo

Well, as long as it doesn't start developing anatomically accurate metal skeletons with red glowing eyes...

mbgerring

> AI is now capable of developing its own inference hardware No, it isn't. A human prompted an LLM to build a software simulation environment for hardware design, enabling an LLM, when prompted by a human, to optimize hardware designs against constraints in the simulation.

deepsun

Bulldozers, excavators and rollers are now capable of building roads.

random__duck

Opened the RTL, looked at the floating point math, learned that apparently you don't need correct floating point operations for LLMs, closed the page.

jonahss

I've been vibecoding an open source hardware AV1 decoder: https://github.com/Jonahss/openav1

jijji

this looks like a start, however, you really need to focus on what OpenAI already did with Jalapeno. [0]. If you could make a true open source inference chip, GPU not CPU based, it would make a real difference and reduce the costs of buying these chip from your design. [0] https://openai.com/index/jalapeno-first-results/

kingcauchy

I'm distinctly reminded of the game Universal Paperclips