Presentation
REACT: Rapid Error-Tolerant Activation Compressor for Efficient Transformer Training
DescriptionTraining large language models (LLMs) poses considerable challenges due to long training times, extensive memory requirements, and substantial bandwidth demands. Activation compression is a promising approach to reduce the memory footprint since activation memory accounts for the majority of memory usage in LLM training with large batch sizes and long context lengths. However, because reducing data size increases the amount of information per bit, it may become sensitive to a fault, which could adversely affect the model convergence and accuracy. In this work, we demonstrate that compression can enhance robustness to bit errors and present REACT, a Rapid Error-tolerant Activation Compressor for efficient Transformer training, which not only reduces memory usage but also enhances error tolerance. Using base-delta encoding, which stores only the differences between data, it minimize data size while significantly reducing the number of error-vulnerable bits. Additionally, an error-limiting encoding scheme is incorporated to address remaining vulnerable bits. Experimental results show that REACT maintains accuracy under bit error rate 1000 times higher while achieving a 4 times reduction in activation memory, with negligible performance overhead.
Event Type
Research Manuscript
TimeTuesday, July 2811:42am - 11:54am PDT
LocationMtg Room 101A