Are Large Language Models Good Neural Architecture Generators for Edge?

Abstract

Neural Architecture Search (NAS) is an incredibly indispensable technique in edge computing scenarios where computational resources are limited. However, the vastness of search spaces presents a formidable obstacle, hindering efficient exploration and identification of optimal architectures. In response, our study introduces an innovative paradigm that characterizes hardware and deep learning architectures, addressing the urgent demand for streamlined and swift architectural design processes. Our approach pioneers the utilization of a fine-tuned Large Language Model (LLM) to navigate the intricate terrain of architectural space. By harnessing the innate capabilities of LLMs, traditionally associated with natural language processing tasks, we extend their application domain to the realm of neural architecture design. Empirical assessments conducted on benchmark datasets such as CIFAR-10 and ImageNet highlight the effectiveness of our fine-tuned LLM-based methodology. Notably, our findings reveal performance levels on par with state-of-the-art (SOTA) NAS techniques, achieving unparalleled speed and completing architectural searches within seconds.