In situ training of an in-sensor artificial neural network based on ferroelectric photosensors
Abstract In-sensor computing has emerged as an ultrafast and low-power technique for next-generation machine vision. However, in situ training of in-sensor computing systems remains challenging due to the demands for both high-performance devices and efficient programming schemes. Here, we experimen...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-024-55508-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841544449863188480 |
|---|---|
| author | Haipeng Lin Jiali Ou Zhen Fan Xiaobing Yan Wenjie Hu Boyuan Cui Jikang Xu Wenjie Li Zhiwei Chen Biao Yang Kun Liu Linyuan Mo Meixia Li Xubing Lu Guofu Zhou Xingsen Gao Jun-Ming Liu |
| author_facet | Haipeng Lin Jiali Ou Zhen Fan Xiaobing Yan Wenjie Hu Boyuan Cui Jikang Xu Wenjie Li Zhiwei Chen Biao Yang Kun Liu Linyuan Mo Meixia Li Xubing Lu Guofu Zhou Xingsen Gao Jun-Ming Liu |
| author_sort | Haipeng Lin |
| collection | DOAJ |
| description | Abstract In-sensor computing has emerged as an ultrafast and low-power technique for next-generation machine vision. However, in situ training of in-sensor computing systems remains challenging due to the demands for both high-performance devices and efficient programming schemes. Here, we experimentally demonstrate the in situ training of an in-sensor artificial neural network (ANN) based on ferroelectric photosensors (FE-PSs). Our FE-PS exhibits self-powered, fast (<30 μs), and multilevel (>4 bits) photoresponses, as well as long retention (50 days), high endurance (109), high write speed (100 ns), and small cycle-to-cycle and device-to-device variations (~0.66% and ~2.72%, respectively), all of which are desirable for the in situ training. Additionally, a bi-directional closed-loop programming scheme is developed, achieving a precise and efficient weight update for the FE-PS. Using this programming scheme, an in-sensor ANN based on the FE-PSs is trained in situ to recognize traffic signs for commanding a prototype autonomous vehicle. Moreover, this in-sensor ANN operates 50 times faster than a von Neumann machine vision system. This study paves the way for the development of in-sensor computing systems with in situ training capability, which may find applications in new data-streaming machine vision tasks. |
| format | Article |
| id | doaj-art-969fd969317c426fb4d5e4b8586648ab |
| institution | Kabale University |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-969fd969317c426fb4d5e4b8586648ab2025-01-12T12:32:04ZengNature PortfolioNature Communications2041-17232025-01-0116111210.1038/s41467-024-55508-zIn situ training of an in-sensor artificial neural network based on ferroelectric photosensorsHaipeng Lin0Jiali Ou1Zhen Fan2Xiaobing Yan3Wenjie Hu4Boyuan Cui5Jikang Xu6Wenjie Li7Zhiwei Chen8Biao Yang9Kun Liu10Linyuan Mo11Meixia Li12Xubing Lu13Guofu Zhou14Xingsen Gao15Jun-Ming Liu16Institute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityKey Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, Hebei Key Laboratory of Photo-Electricity Information and Materials, Hebei UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityKey Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, Hebei Key Laboratory of Photo-Electricity Information and Materials, Hebei UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityKey Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, Hebei Key Laboratory of Photo-Electricity Information and Materials, Hebei UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityNational Center for International Research on Green Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityInstitute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal UniversityAbstract In-sensor computing has emerged as an ultrafast and low-power technique for next-generation machine vision. However, in situ training of in-sensor computing systems remains challenging due to the demands for both high-performance devices and efficient programming schemes. Here, we experimentally demonstrate the in situ training of an in-sensor artificial neural network (ANN) based on ferroelectric photosensors (FE-PSs). Our FE-PS exhibits self-powered, fast (<30 μs), and multilevel (>4 bits) photoresponses, as well as long retention (50 days), high endurance (109), high write speed (100 ns), and small cycle-to-cycle and device-to-device variations (~0.66% and ~2.72%, respectively), all of which are desirable for the in situ training. Additionally, a bi-directional closed-loop programming scheme is developed, achieving a precise and efficient weight update for the FE-PS. Using this programming scheme, an in-sensor ANN based on the FE-PSs is trained in situ to recognize traffic signs for commanding a prototype autonomous vehicle. Moreover, this in-sensor ANN operates 50 times faster than a von Neumann machine vision system. This study paves the way for the development of in-sensor computing systems with in situ training capability, which may find applications in new data-streaming machine vision tasks.https://doi.org/10.1038/s41467-024-55508-z |
| spellingShingle | Haipeng Lin Jiali Ou Zhen Fan Xiaobing Yan Wenjie Hu Boyuan Cui Jikang Xu Wenjie Li Zhiwei Chen Biao Yang Kun Liu Linyuan Mo Meixia Li Xubing Lu Guofu Zhou Xingsen Gao Jun-Ming Liu In situ training of an in-sensor artificial neural network based on ferroelectric photosensors Nature Communications |
| title | In situ training of an in-sensor artificial neural network based on ferroelectric photosensors |
| title_full | In situ training of an in-sensor artificial neural network based on ferroelectric photosensors |
| title_fullStr | In situ training of an in-sensor artificial neural network based on ferroelectric photosensors |
| title_full_unstemmed | In situ training of an in-sensor artificial neural network based on ferroelectric photosensors |
| title_short | In situ training of an in-sensor artificial neural network based on ferroelectric photosensors |
| title_sort | in situ training of an in sensor artificial neural network based on ferroelectric photosensors |
| url | https://doi.org/10.1038/s41467-024-55508-z |
| work_keys_str_mv | AT haipenglin insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT jialiou insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT zhenfan insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT xiaobingyan insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT wenjiehu insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT boyuancui insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT jikangxu insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT wenjieli insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT zhiweichen insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT biaoyang insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT kunliu insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT linyuanmo insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT meixiali insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT xubinglu insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT guofuzhou insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT xingsengao insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors AT junmingliu insitutrainingofaninsensorartificialneuralnetworkbasedonferroelectricphotosensors |