Presentation
Late Breaking Results: Influential Data Selection for LLM-Based RTL Generation
DescriptionScarcity and noise in open-source datasets severely limit Large Language Models (LLMs) in Register Transfer Level (RTL) design. To address this, we propose a targeted data selection framework using Low-rank Gradient Similarity Search (LESS). By leveraging gradient-based influence estimation, LESS filters detrimental data by selecting training examples that align with the target task's gradient trajectory. Experiments show that fine-tuning on just 5% of LESS-selected data matches full-dataset training performance. Furthermore, using LESS-selected data for second-stage fine-tuning outperforms fully trained models, whereas random selection degrades them. Prioritizing data quality over quantity thus offers a promising path to state-of-the-art automated hardware design.
Event Type
Work in Progress
TimeMonday, July 275:11pm - 5:12pm PDT
LocationExhibit Hall