A research team led by Professors Zhao Weisheng and Zhang Yue from the School of Integrated Circuit Science and Engineering at Beihang University has made significant progress in high-performance AI chip design, proposing an array partition multi-macro CIM (APM-CIM) system with dynamicsparse approximation for convolutional neural network (CNN) inference on edge applications.

The findings have been published in IEEE Journal of Solid-State Circuits (JSSC), a leading international journal in integrated circuit design, under the title "APM-CIM: An Array Partition Multi-macro CIM System with Dynamic Sparse Approximation for Neural Network Edge Applications."

Computing-in-memory (CIM) has been widely investigated as a solution to the von Neumann bottleneck to reduce data transfer between memory and processor. However, existing CIM macros for neural network (NN) edge applications still suffer from high latency and energy consumption due to the correlation between data precision and the number of multiplication and accumulation (MAC) cycles. Moreover, modern CNN models contain massive parameters that are distributed across multiple CIM macros. While multi-macro architectures enable large-scale deployment, they incur inefficient bandwidth utilization during input loading and frequent inter-module communication, both of which cause significant performance degradation.

Innovation features of proposed APM-CIM for challenges in CNN

To address these issues, the team presents an array partition multi-macro CIM (APM-CIM) system with dynamic sparse approximation. First, a novel CIM macro incorporating double-input and sparse approximation mechanism is proposed to halve the number of computation cycles and skip insignificant operands during MAC. Second, an array partition mapping scheme for weights is proposed to reduce the bandwidth requirement of a CIM macro, thereby maximizing cache bandwidth utilization to implement high parallelism computation. Finally, data and task managers are designed to minimize the performance degradation of the CIM macro at the system level.

Through cross-level co-design of circuit, architecture, and system, the fabricated 40-nm CMOS chip achieves a system energy efficiency of 67.63 TOPS/W, which reduces the performance loss ratio by 3.7× and is 10.4× higher than state-of-the-art CIM-based systems. Using the VGG model on the CIFAR-10 dataset, the system delivers an accuracy of 89.95% at a power consumption of 34.0 mW.

This work was a collaboration among the School of Integrated Circuit Science and Engineering at Beihang University, the Hangzhou International Innovation Institute of Beihang University, and the National Key Laboratory of Spintronics, under the guidance of Professor Zhao Weisheng. Doctoral student Zhang Bojun from the School of Integrated Circuit Science and Engineering and Associate Researcher Wang Jinkai from the Hangzhou International Innovation Institute are co-first authors of the paper. Associate Researcher Wang Jinkai and Professor Zhang Yue are co-corresponding authors. The work was supported by the National Key Research and Development Program and the Zhejiang Provincial Natural Science Foundation, among other projects.

Article link: https://ieeexplore.ieee.org/abstract/document/11649852

Editor: Lyu Xingyun