Taalas-Style On-Chip Weights on a $250 FPGA: a Language Model at 60k tok/s Featured
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.