Light ｜ Matrix Multiplication Photon Accelerator and Its Application

Written by Cheng Junwei (Huazhong University of Science and Technology)

In recent years, with the increasing demand for data processing by artificial intelligence and the fifth generation communication, the data computing capacity and computing power consumption have increased rapidly. The clock frequency of traditional electronic methods is generally limited to several GHz, which can no longer meet the needs of ultra-high speed and low delay mass data processing. On the other hand, with the failure of Moore’s Law, it becomes more and more difficult to improve their performance and energy efficiency by relying on semiconductor electronic technology.

Due to the continuous and substantial increase of information capacity, in the foreseeable future, the general electronic processor is difficult to be competent for high-complexity artificial intelligence and signal processing tasks. butmatrix calculationIt is one of the most widely used and essential information processing tools in science and engineering.

Most signal processing, such as discrete Fourier transform and convolution operation, can be attributed to matrix calculation. The main feature of neural network algorithm, especially represented by deep learning, is that it contains heavy matrix calculation. Taking convolutional neural network as an example, its matrix calculation overhead can account for 80% or even more than 90% of the total overhead. Accelerating and optimizing matrix calculation can greatly improve the calculation efficiency and power consumption of signal processing and artificial intelligence.

Photonic devices have ultra-large bandwidth and ultra-low power consumption, the frequency of light can reach 100 THz, and there are many free physical dimensions, which makes photonic computing one of the most competitive candidates in high-capacity and low-delay matrix information processing in the post-Moore era. In recent years, photon matrix multiplication has developed rapidly and is widely used in the fields of photon acceleration such as optical signal processing, artificial intelligence and photonic neural network. A large number of applications based on matrix multiplication show great potential and opportunities in the field of photon accelerators.

Fig. 1 conceptual diagram of matrix multiplication photon accelerator

Recently, fromHuazhong University of Science and TechnologyThe research team, andChinese University of Hong Kong, University of Shanghai for Science and Technology, Zhejiang UniversityandXizhi technologyA number of research scholars have cooperated to "Photonic matrix multiplication lights up photonic accelerator and beyond"For the topic, in　Light: Science & Applications　A summary article on the theme of matrix multiplication photon accelerator was published in.

Photon matrix-vector multiplication classification

At present, the mainstream photon matrix-vector multiplication (MVM) mainly includes three categories, namely, matrix calculation based on (single/multiple) plane optical conversion (PLC), matrix calculation based on Mach Zehnder interferometer (MZI) network, and matrix calculation based on wavelength division multiplexing (WDM). Among them, the matrix calculation of plane transformation is divided into single-plane matrix calculation (SPLC) and multi-plane matrix calculation (MPLC), both of which belong to coherent calculation, and the reported input vector length can reach the order of 357 and 490,000 respectively. The output vector length of MZI and WDM methods is generally less than 100, which is mainly used to integrate photonic matrix computing chips. These methods mainly calculate the matrix based on the spatial dimension or wavelength dimension of photons, and can also combine multiple dimensions of photons to construct ultra-high-capacity photon tensor core.

Fig. 2 Classification of Photon Matrix Multiplication

Accelerate application of photon matrix-vector multiplication

Photon matrix multiplication network itself can be used as a general linear photon loop for photon signal processing. In recent years, MVM has become a powerful tool for various photon signal processing methods. MPLC-MVM benefits from its ability of large-scale matrix calculation, which can manage a large number of patterns, can be used as a general pattern classifier, and the scale of its operating patterns can reach hundreds. It can also be further extended to realize the simultaneous control of multiple classical physical dimensions of photons, and can be used in applications such as spatial imaging encryption. MZI-MVM is easy to integrate, and because of the high speed of phase shifter, it can realize the automatic configuration of MZI grid function, and can be used as an adaptive mode processing device to realize free uploading, downloading, multiplexing and demultiplexing of multiple modes and channel descrambling. WDM-MVM takes up less space and is easier to configure the transmission matrix. It has been applied to programmable pulse shaping, micro-ring weight library and signal component analysis.

Figure 3 Application of Photon Matrix Multiplication

Artificial intelligence technology has been widely used in various electronic industries, such as speech recognition and image processing based on deep learning. As the basic component of neural network, matrix computing takes up most of the computing tasks, for example, the computation of GoogleNet and OverFeat models exceeds 80%. Improving the performance of the matrix is one of the most effective ways to accelerate the neural network. Compared with electric calculation, optical calculation is poor in data storage and flow control, and the low efficiency of optical nonlinearity limits the application of nonlinear calculation such as activation function. Then, through wavelength, mode and polarization multiplexing technology, optical methods have obvious advantages in large-scale parallel computing, and have extremely high data modulation speed and low delay. Therefore, photonic network is very suitable for matrix calculation. The combination of optical computing and artificial intelligence is expected to realize intelligent photon processor and photon accelerator. In recent years, artificial intelligence technology has also developed rapidly in the field of optics. All kinds of photon matrix calculations have been proved to be able to replace the linear part of neural network algorithm, and its single core has been proved that the calculation capacity can exceed 11TOPS, the delay of photon matrix calculation is generally in the order of picoseconds, the energy consumption of a single multiplication and addition operation is in the order of defocus, and the signal modulation rate can be as high as 100 GHz. Compared with electronic computing, it has obvious advantages in speed, delay and power consumption.

Challenges and prospects

At present, there is still a huge gap between photonic matrix calculation and electronic calculation. In order to solve this problem, one of the direct and effective solutions is to manufacture large-scale photonic integrated circuits. Similar to integrated circuits, the improvement of manufacturing technology provides an opportunity to realize photonic integrated circuit chips with larger scale and higher integration density. In addition, by using multiple free dimensions of photons, such as modes, wavelengths, etc., optical devices can perform a large number of parallel calculations, which can be performed in a physical photon calculation core. At the same time, we can expand the network scale by optimizing photonic devices, such as spectrum reuse strategy, higher modulation speed and lower power consumption modulator, low loss waveguide, hybrid integration and so on.

　

Matrix calculation and activation function are two basic operating elements of neural network model. Photon matrix calculation has obvious advantages over electronic methods in signal rate, delay, calculation density and power consumption, but the photon activation function is still immature. At present, it is mainly divided into optical-electrical-optical conversion method and all-optical method. How to realize the activation function with low power consumption and high response rate in the future is still a difficult problem.

Before all-optical artificial neural network matures, especially before optical nonlinear effect and all-optical cascade maturity, photoelectric hybrid artificial intelligence computing is still a more practical and competitive candidate architecture for deep artificial neural network. Therefore, it is one of the core research paths of photonic artificial intelligence to develop an efficient and dedicated optoelectronic hybrid artificial intelligence hardware chip system. In the future, the bottom hardware of accelerator can be realized based on photon matrix calculation and electronic control, and various algorithms suitable for this hardware can be developed. Finally, these algorithms can be flexibly called in the user layer to realize various accelerated applications, such as channel descrambler and image recognition.

Figure 4 Photoelectric Hybrid AI Computing Chip Architecture

Thesis information

Zhou, H., Dong, J., Cheng, J. et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci Appl 11, 30 (2022).

https://doi.org/10.1038/s41377-022-00717-8

Professor Dong Jianji from Huazhong University of Science and Technology is the corresponding author of the paper, and Associate Professor Zhou Hailong is the first author of the paper. Co-authors of the thesis include Dr. Huang Chaoran from the Chinese University of Hong Kong, Dr. Shen Yichen from Xizhi Technology, Professor Zhang Qiming and Academician Gu Min from Shanghai University of Science and Technology, Dr. Qian Chao, Professor Chen Hongsheng and Professor Ruan Zhichao from Zhejiang University, and Dr. Cheng Junwei, Dr. Dong Wenchan and Professor Zhang Xinliang from Huazhong University of Science and Technology.

Read the original text