{ "archive_path": "archive/1791324472.351906", "base_url": "github.com/FeSens/openTPU", "basename": "openTPU", "bookmarked_date": "2026-10-06 22:07", "canonical": { "archive_org_path": "https://web.archive.org/web/github.com/FeSens/openTPU", "dom_path": "output.html", "favicon_path": "favicon.ico", "git_path": "git/", "google_favicon_path": "https://www.google.com/s2/favicons?domain=github.com", "headers_path": "headers.json", "htmltotext_path": "htmltotext.txt", "index_path": "index.html", "media_path": "media/", "mercury_path": "mercury/content.html", "pdf_path": "output.pdf", "readability_path": "readability/content.html", "screenshot_path": "screenshot.png", "singlefile_path": "singlefile.html", "warc_path": "warc/", "wget_path": "github.com/FeSens/openTPU.html" }, "domain": "github.com", "extension": "", "hash": "1XFHC5K54AJZX891BXVC", "history": { "archive_org": [], "dom": [], "favicon": [ { "cmd": [ "curl", "--silent", "--location", "--compressed", "--max-time", "120", "--output", "favicon.ico", "--user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 ArchiveBox/0.7.4 (+https://github.com/ArchiveBox/ArchiveBox/) curl/curl 8.14.1 (x86_64-pc-linux-gnu)", "https://www.google.com/s2/favicons?domain=github.com" ], "cmd_version": "curl 8.14.1 (x86_64-pc-linux-gnu)", "end_ts": "2026-10-06T22:09:12.064964+00:00", "index_texts": null, "output": "favicon.ico", "pwd": "/data/archive/1791324472.351906", "schema": "ArchiveResult", "start_ts": "2026-10-06T22:09:11.502405+00:00", "status": "succeeded" } ], "git": [], "headers": [ { "cmd": [ "curl", "--silent", "--location", "--compressed", "--head", "--max-time", "120", "--user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 ArchiveBox/0.7.4 (+https://github.com/ArchiveBox/ArchiveBox/) curl/curl 8.14.1 (x86_64-pc-linux-gnu)", "https://github.com/FeSens/openTPU" ], "cmd_version": "curl 8.14.1 (x86_64-pc-linux-gnu)", "end_ts": "2026-10-06T22:09:12.935040+00:00", "index_texts": null, "output": "headers.json", "pwd": "/data/archive/1791324472.351906", "schema": "ArchiveResult", "start_ts": "2026-10-06T22:09:12.074191+00:00", "status": "succeeded" } ], "htmltotext": [], "media": [], "mercury": [ { "cmd": [ "/app/node_modules/@postlight/parser/cli.js", "https://github.com/FeSens/openTPU" ], "cmd_version": "1.0.0", "end_ts": "2026-10-06T22:09:33.153713+00:00", "index_texts": null, "output": "mercury", "pwd": "/data/archive/1791324472.351906", "schema": "ArchiveResult", "start_ts": "2026-10-06T22:09:27.292791+00:00", "status": "succeeded" } ], "pdf": [], "readability": [ { "cmd": [ "/app/node_modules/readability-extractor/readability-extractor", "/tmp/tmpey21oafm", "https://github.com/FeSens/openTPU" ], "cmd_version": "0.0.11", "end_ts": "2026-10-06T22:09:27.287765+00:00", "index_texts": [ "openTPU\nAn open-source AI accelerator, developed by AI.\nopenTPU brings the lessons of auto-arch-tournament\nto AI accelerators. It asks two questions: how far can AI agents go at hardware design, and can\nthey build the chip that runs their own inference?\n\notpu-chat running LFM2.5-230M on the FPGA card (left), with otpu-smi showing the card's\nutilization and DRAM bandwidth (right).\nA place to learn\nopenTPU is also a learning project. The whole accelerator lives in one small monorepo that you\ncan read end to end: the hardware design (SystemVerilog), the instruction set, a bit-exact\nsimulator, a kernel language and its compiler, and the host software that drives a real PCIe\ncard. If you want to understand how an AI accelerator works, from a matmul in Python down to\nthe wires, this is a good place to start.\nResults\nThe design runs ten modern models with their real weights on an Inspur YPCB-00338 card\n(Xilinx Kintex-7 xc7k480t, two DDR3 channels), and the card produces the same tokens as the\nsimulator, bit for bit.\n\n\n\nModel\nWeights\nDecode, device\nDecode, wall\nPrefill, device\nDRAM while decoding\n\n\n\n\nLFM2.5-230M\nint8\n59.0 tok/s\n52.3 tok/s\n295.6 tok/s\n14.5 GB/s (85% of peak)\n\n\nLFM2.5-230M\n4-bit, int8 head\n85.8 tok/s\n82.1 tok/s\n335.4 tok/s\n14.1 GB/s (82%)\n\n\nQwen3-0.6B\nint8\n21.6 tok/s\n21.3 tok/s\n92.1 tok/s\n14.4 GB/s (84%)\n\n\nQwen3-0.6B\n4-bit, int8 head\n31.3 tok/s\n30.7 tok/s\n103.4 tok/s\n13.9 GB/s (82%)\n\n\nQwen3.5-0.8B\nint8\n17.6 tok/s\n16.3 tok/s\n61.4 tok/s\n14.5 GB/s (85%)\n\n\nQwen3.5-0.8B\n4-bit, int8 head\n24.5 tok/s\n23.3 tok/s\n66.7 tok/s\n14.1 GB/s (83%)\n\n\nGemma 4 E2B\n4-bit, int8 head\n10.57 tok/s\n10.53 tok/s\n32.1 tok/s\n15.6 GB/s (92%)\n\n\nGemma 4 E2B\n4-bit, 4-bit head\n12.14 tok/s\n12.09 tok/s\n29.9 tok/s\n15.5 GB/s (91%)\n\n\nLFM2-2.6B\nint8\n6.05 tok/s\n6.03 tok/s\n21.4 tok/s\n16.1 GB/s (94%)\n\n\nLFM2-2.6B\n4-bit, int8 head\n10.96 tok/s\n10.93 tok/s\n20.6 tok/s\n15.8 GB/s (93%)\n\n\nSmolLM3-3B\nint8\n5.00 tok/s\n4.99 tok/s\n21.1 tok/s\n16.0 GB/s (94%)\n\n\nSmolLM3-3B\n4-bit, int8 head\n8.74 tok/s\n8.72 tok/s\n22.8 tok/s\n15.7 GB/s (92%)\n\n\nPhi-4-mini (3.8B)\nint8\n3.99 tok/s\n3.98 tok/s\n13.8 tok/s\n16.0 GB/s (94%)\n\n\nPhi-4-mini (3.8B)\n4-bit, int8 head\n6.56 tok/s\n6.55 tok/s\n15.0 tok/s\n15.8 GB/s (92%)\n\n\nQwen3.5-2B\nint8\n8.02 tok/s\n8.00 tok/s\n38.2 tok/s\n16.0 GB/s (94%)\n\n\nQwen3.5-2B\n4-bit, int8 head\n12.09 tok/s\n12.03 tok/s\n41.7 tok/s\n15.8 GB/s (92%)\n\n\nQwen3.5-4B\n4-bit, int8 head\n5.88 tok/s\n5.87 tok/s\n12.9 tok/s\n15.7 GB/s (92%)\n\n\nGemma 4 E4B\nint8, 4-bit head and down 0-23\n3.78 tok/s\n3.75 tok/s\n14.8 tok/s\n16.0 GB/s (94%)\n\n\n\nMeasured on the card: the first three models on 2026-09-29 with the production image\ndeploy_champ_e698dcd7. LFM2-2.6B, SmolLM3-3B and Phi-4-mini on 2026-09-30, and Qwen3.5-2B and\n4B and Gemma 4 on 2026-10-01, with build B, deploy_fused133c_79c5707a, production since then.\nBuild B decodes LFM2-2.6B, SmolLM3 and Phi-4-mini 8-9% faster than e698dcd7 (Gemma 4 E2B 10%),\nat 91-94% of the DRAM peak instead of 82-87%. Qwen3.5-4B's int8 image is over 4 GiB.\n\nThe image: main e698dcd at 133.33 MHz, one bitstream for all models. It has LiteDRAM\ncontrollers calibrated by a small CPU inside the memory core, a four-column systolic matrix\nunit and the stream engine (docs/stream.md). DDR3-1066, with a 17.1 GB/s\npeak.\nThe host: the card sits in opentpu (Intel Core i7-4790).\nMethod, tools/qual/perf.py: decode is 64 greedy tokens after a 512-token prompt, with the\nhost's argmax in the loop (not streamed). \"Device\" counts only the cycles the accelerator runs;\n\"wall\" adds the host. Prefill is the 512-token prompt, on the device.\nDRAM traffic comes from the card's own counters while it runs.\nGemma 4 E2B keeps its per-layer embedding tables on the card (3.5-3.6 GiB images;\ndocs/gemma4.md); in int8 it does not fit. It matches Hugging Face's greedy\ntokens on three prompts with either head. E4B's table (2.95 GB) stays on the host, which\ncopies one 11 KB row into the card per token; its image is 3.96 GiB, int8 with the head and\nthe first 24 layers' down projections in 4-bit (docs/gemma4_e4b.md).\nIn the card's own decode loop (the card picking every token) Gemma 4 decodes faster: E2B\n11.01 / 12.73 tok/s (int8 / 4-bit head), E4B 3.83 tok/s, on the device.\nEvery configuration matches the simulator token for token, per-position and with the\nresident decode program. More detail in docs/board.md.\n\nWith the logits streamed back while the card runs (tools/decode_profile.py, 96 tokens), 4-bit\ndecode is faster, in device / wall tok/s:\n\nLFM2: 89.5 / 84.5;\nQwen3: 33.7 / 33.3;\nQwen3.5: 24.6 / 24.2;\nLFM2-2.6B: 11.07 / 11.02 (build B);\nSmolLM3-3B: 8.92 / 8.89 (build B);\nPhi-4-mini: 6.69 / 6.67 (build B).\n\nThe previous production image, se-cand3, was built with the Xilinx MIG, a two-column matrix\nunit and a 120.755 MHz clock. Measured the same way, the new image:\n\ndecode: within 2.3% of se-cand3's in every configuration. Decode is bound by DRAM, and\nLiteDRAM reads at 82-85% of the DDR3 peak, as the MIG did.\nprefill: 1.3x (Qwen3.5) to 2.0x (LFM2 4-bit) faster.\ncalibration: when the image starts, the core's CPU calibrates both DDR3 channels in 12 s,\nwith no host involvement.\n\nThe earlier images and their numbers are in docs/board.md, section 5.\nMixture-of-experts models bigger than the card's 4 GiB run with their experts streamed from host\nstorage (docs/offload.md, section 10). The card routes each token and computes\nevery expert, and it keeps the experts in per-layer slots in its DRAM. The host only copies\nmissing experts from a pool file into those slots, at the link's rate (section 10.1). Measured\non 2026-10-01 with build B (79c5707a), the card's own decode loop picking every token, 4-bit\nexperts, int8 head:\n\nLFM2.5-8B-A1B (8.5B parameters, 1.7B active): 10.6 tok/s over 160 tokens. 98.5% of expert\nuses hit the slots, and 5.2 MB streamed per token.\nQwen3.5-35B-A3B (34.7B parameters, 3.0B active): 3.95 tok/s, with Hugging Face's 16\ngreedy tokens. 62% of expert uses hit, and 153 MB streamed per token at 1.41 GB/s over PCIe\n(section 10.3).\nBoth match the simulator bit for bit.\n\n4-bit weights (docs/quant.md) use FP4 values with two-level block scales, 4.25\nbits per weight, and keep the LM head in int8 for accuracy. They cut the bytes per token by about\na third and raise decode speed by 40% (Qwen3.5) to 45% (Qwen3, LFM2), at a measurable cost in\nperplexity that docs/quant.md reports per model.\nThe host is nearly out of the way. For LFM2 and Qwen3 the card runs one decode program compiled\nonce, which reads the position from a register and looks up its own embedding and RoPE rows, and\nthe logits stream back while the card is still running: the host adds 0.17 to 0.30 ms per\ntoken on omarchy (0.45 to 1.3 ms on opentpu).\nQwen3.5 runs the same way for decode; its prefill still compiles each chunk's program on the host,\nahead of the card.\nHow it works\n Kernels in ol mlp, attention, full model layers\n | @ol.jit\n Language + compiler layouts, affine loop addressing, fusion\n |\n ISA 8 x 32-bit words per instruction\n |\n ISA simulator <======> RTL same bits, checked by the tests\n (Python) (SystemVerilog)\n | Vivado bitstream\n FPGA card Kintex-7 xc7k480t\n | PCIe\n Host otpu-chat, otpu-smi, otpu-lens\n\nThe machine is deliberately simple. A sequencer issues one instruction per cycle to a few\nunits: DMA moves data, the matrix unit multiplies int8 weights streamed from DRAM, the vector\nunit does fp32 math, and a quantizer turns results back into int8. There is no cache and no\nhidden scheduling: every data movement is an instruction, so a trace shows exactly where the\ncycles go. docs/isa.md describes the whole instruction set.\nA kernel looks like this:\nfrom opentpu import language as ol\n\n@ol.jit\ndef mlp(h, gamma, w_gate, w_up, w_down, out, eps): # simplified; see kernels/mlp.py\n x = ol.load(h)\n xs = ol.quantize(rmsnorm(x, ol.load(gamma), eps))\n g = ol.dot(xs, w_gate)\n u = ol.dot(xs, w_up)\n a = ol.all_gather(silu(g) * u)\n y = ol.all_gather(ol.dot(a, w_down))\n if ol.program_id() == 0:\n ol.store(out, x + y)\nBecause every data movement is an instruction, a trace of a run explains its speed. Lens, the\nprofiler, records a run from the RTL, the simulator or the card and opens it in the browser,\nwith a roofline, a timeline and per-instruction tables (docs/lens.md).\n\nLens replaying part of a Qwen3 decode step. Colours show what each unit is doing in each\ncycle: busy, waiting on DRAM, or waiting on another instruction.\nTry it\nEverything except the card runs on a laptop.\npip install -e .\npip install pytest torch transformers\npython3 -m pytest -q # RTL tests also need Verilator 5\n\nhf download LiquidAI/LFM2.5-230M --local-dir models/LFM2.5-230M\notpu-chat --model lfm2 --backend isa # chat on the simulator\nWith a card, build the bitstream (make bit in boards/ypcb-00338), load\nit over JTAG, then run sudo otpu-setup and otpu-chat --backend board.\ndocs/board.md walks through the bring-up.\n\n\n\nCommand\nWhat it does\n\n\n\n\notpu-chat\nchat with Qwen3-0.6B, LFM2.5-230M (--model lfm2), Qwen3.5-0.8B (--model qwen35), LFM2-2.6B (lfm2-2.6b), SmolLM3-3B (smollm3), Phi-4-mini (phi4-mini) or Qwen3.5-2B / 4B (qwen35-2b, qwen35-4b)\n\n\notpu-smi\ntemperature, power, DRAM bandwidth and per-unit utilization\n\n\notpu-lens\nrecord a run and open it in the profiler\n\n\notpu-selftest, otpu-diag\ncheck that the card works\n\n\n\nWhere to start reading\n\ndocs/isa.md: the instruction set. Everything else is built on it.\nopentpu/kernels and docs/compiler.md: how a kernel\nbecomes instructions.\nopentpu/isasim.py: the simulator, which is the spec.\nrtl/: the hardware, starting from rtl/top/otpu_top.sv.\ndocs/lfm2.md, docs/qwen35.md, docs/llama.md,\ndocs/benchmarks.md: whole models and where their cycles go.\ndocs/board.md: the physical card, from clocks to PCIe.\n\nWhat's next\n\nThe last few percent of DRAM. Decode is bound by DRAM efficiency: it reads 82 to 85% of the\nDDR3-1066 peak. Work on the LiteDRAM path's efficiency is under way.\nTiming margin and area. The design closes 133.33 MHz, the clock at which the 128-byte port\nmatches the two DDR3 channels, but only just (WNS +0.032 ns). A tournament of Vivado runs keeps\nworking on its margin and area. Decode is bound by DRAM, so a faster clock mostly helps prefill.\nFaster prefill. The four-column systolic matrix unit is in the production image; prefill is\nstill limited by the matrix unit's multiply rate.\n\nContributing\nIssues and pull requests are welcome, and most of the work needs only Python and Verilator, not\nan FPGA. Changes to the ISA, the simulator or the RTL must keep python3 -m pytest -q passing,\nand performance claims should say how they were measured.\nLicense\nApache License 2.0. See LICENSE." ], "output": "readability", "pwd": "/data/archive/1791324472.351906", "schema": "ArchiveResult", "start_ts": "2026-10-06T22:09:23.194976+00:00", "status": "succeeded" } ], "screenshot": [], "singlefile": [], "title": [ { "cmd": [ "curl", "--silent", "--location", "--compressed", "--max-time", "120", "--user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 ArchiveBox/0.7.4 (+https://github.com/ArchiveBox/ArchiveBox/) curl/curl 8.14.1 (x86_64-pc-linux-gnu)", "https://github.com/FeSens/openTPU" ], "cmd_version": "curl 8.14.1 (x86_64-pc-linux-gnu)", "end_ts": "2026-10-06T22:09:23.178160+00:00", "index_texts": null, "output": "GitHub - FeSens/openTPU: An open-source AI accelerator, developed by AI: RTL, ISA, simulator, compiler and profiler in one repo. Runs Qwen3, LFM2.5 and Qwen3.5 on a Kintex-7 PCIe card. \u00b7 GitHub", "pwd": "/data/archive/1791324472.351906", "schema": "ArchiveResult", "start_ts": "2026-10-06T22:09:23.101871+00:00", "status": "succeeded" } ], "wget": [ { "cmd": [ "wget", "--no-verbose", "--adjust-extension", "--convert-links", "--force-directories", "--span-hosts", "--no-parent", "-e", "robots=off", "--reject=js,mjs,woff,woff2,ttf,otf,eot,mp4,m4v,mov,webm,ogv,mkv,avi,flv", "--timeout=120", "--restrict-file-names=windows", "--page-requisites", "--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 ArchiveBox/0.7.4 (+https://github.com/ArchiveBox/ArchiveBox/) wget/GNU Wget 1.21.3", "--compression=auto", "--timestamping", "https://github.com/FeSens/openTPU" ], "cmd_version": "GNU Wget 1.21.3", "end_ts": "2026-10-06T22:09:23.088163+00:00", "index_texts": null, "output": "github.com/FeSens/openTPU.html", "pwd": "/data/archive/1791324472.351906", "schema": "ArchiveResult", "start_ts": "2026-10-06T22:09:12.944317+00:00", "status": "succeeded" } ] }, "icons": null, "is_archived": true, "is_static": false, "latest": { "archive_org": null, "dom": null, "favicon": "favicon.ico", "git": null, "media": null, "pdf": null, "screenshot": null, "singlefile": null, "title": "GitHub - FeSens/openTPU: An open-source AI accelerator, developed by AI: RTL, ISA, simulator, compiler and profiler in one repo. Runs Qwen3, LFM2.5 and Qwen3.5 on a Kintex-7 PCIe card. \u00b7 GitHub", "warc": null, "wget": "github.com/FeSens/openTPU.html" }, "link_dir": "/data/archive/1791324472.351906", "newest_archive_date": "2026-10-06T22:09:27.292791+00:00", "num_failures": 0, "num_outputs": 6, "oldest_archive_date": "2026-10-06T22:09:11.502405+00:00", "path": "/FeSens/openTPU", "schema": "Link", "scheme": "https", "snapshot_id": "5b227c44-7647-42d2-9cf8-88b484c0b7cd", "sources": [ "/data/sources/1791324472-import.txt" ], "tags": null, "tags_str": "", "timestamp": "1791324472.351906", "title": "GitHub - FeSens/openTPU: An open-source AI accelerator, developed by AI: RTL, ISA, simulator, compiler and profiler in one repo. Runs Qwen3, LFM2.5 and Qwen3.5 on a Kintex-7 PCIe card. \u00b7 GitHub", "updated": "2026-10-06T22:09:11.501929+00:00", "updated_date": "2026-10-06 22:09", "url": "https://github.com/FeSens/openTPU" }