Presentation
Achieving Intra- and Inter-Operator Communication Lower Bounds in Near-Memory Processing
DescriptionNear-memory processing (NMP) is a promising way to overcome the memory wall in large language models (LLMs). However, dataflow optimization in NMP is fundamentally constrained, as existing analyses cannot efficiently handle the new distributed vault/channel organization. We propose the Remote-Access-Free (RAF) dataflow, which uses tensor rotation to abstract this distributed organization and eliminate all intra-operator remote access. On top of RAF, we apply analytical optimization to minimize intra-operator local access, thereby achieving the intra-operator communication lower bound. We then introduce data partitioning that removes inter-operator remote access and enable operator fusion to minimize inter-operator local access, so that the inter-operator communication lower bound is also reached. Experimental results show that RAF reduces energy by 50.4%, 39.0%, and 37.8%, and delivers speedups of 3.98×, 1.72×, and 1.57× over IANUS, H^2LLM, and OptiPIM, respectively.
Event Type
Research Manuscript
TimeTuesday, July 2810:56am - 11:10am PDT
LocationMtg Room 203AB