VectorFLASH is an edge FPGA LLM accelerator that enables long-context inference up to 128K tokens on a Xilinx Zynq UltraScale+ XCZU15EG. As input lengths grow with RAG and multi-turn agent interaction, the KV cache EMA comes to dominate, exceeding the weight EMA by over 9× at 128K, demanding sub-4-bit compression on both. VectorFLASH applies dual VQ with three key features. First, a Single Alignment Rotation Unit performs Hadamard rotation for online KV VQ using an INT48 butterfly, exploiting the 48-bit DSP48E2 adder via lossless FP-to-INT conversion to cut the DSP usage by 95.5%. Second, an Adaptive-Beta Entropy-Constrained VQ (AB-ECVQ) removes redundant centroids and applies Huffman coding for 2.5-bit weight compression, reducing the total latency by 31.7% with a Huffman Decoding Unit of only 9% resource overhead. Third, a high-hit-rate entry replication with the Codebook Lookup MAC Core balances the PE workload, reducing the latency by 90.1%, while the VQ MAC PE achieves 2.89× higher throughput/LUT than an INT8 MAC. VectorFLASH attains 2.8-to-8.7 Token/s and 197.3 GOPS/W with a peak of 1.48 TOPS at 7.5W, the first edge FPGA realization of dual-VQ long-context LLM inference.
