Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices
Abstract Multimodal machine learning, as a prospective advancement in artificial intelligence, endeavors to emulate the brain's multimodal learning abilities with the objective to enhance interactions with humans. However, this approach requires simultaneous processing of diverse types of data,...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-12-01
|
| Series: | Advanced Science |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/advs.202406242 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846141342620581888 |
|---|---|
| author | Haonan Sun Haoxiang Tian Yihao Hu Yi Cui Xinrui Chen Minyi Xu Xianfu Wang Tao Zhou |
| author_facet | Haonan Sun Haoxiang Tian Yihao Hu Yi Cui Xinrui Chen Minyi Xu Xianfu Wang Tao Zhou |
| author_sort | Haonan Sun |
| collection | DOAJ |
| description | Abstract Multimodal machine learning, as a prospective advancement in artificial intelligence, endeavors to emulate the brain's multimodal learning abilities with the objective to enhance interactions with humans. However, this approach requires simultaneous processing of diverse types of data, leading to increased model complexity, longer training times, and higher energy consumption. Multimodal neuromorphic devices have the capability to preprocess spatio‐temporal information from various physical signals into unified electrical signals with high information density, thereby enabling more biologically plausible multimodal learning with low complexity and high energy‐efficiency. Here, this work conducts a comparison between the expression of multimodal machine learning and multimodal neuromorphic computing, followed by an overview of the key characteristics associated with multimodal neuromorphic devices. The bio‐plausible operational principles and the multimodal learning abilities of emerging devices are examined, which are classified into heterogeneous and homogeneous multimodal neuromorphic devices. Subsequently, this work provides a detailed description of the multimodal learning capabilities demonstrated by neuromorphic circuits and their respective applications. Finally, this work highlights the limitations and challenges of multimodal neuromorphic computing in order to hopefully provide insight into potential future research directions. |
| format | Article |
| id | doaj-art-43f3ad18cba143b28225efd01b8fc79d |
| institution | Kabale University |
| issn | 2198-3844 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Science |
| spelling | doaj-art-43f3ad18cba143b28225efd01b8fc79d2024-12-04T12:14:54ZengWileyAdvanced Science2198-38442024-12-011145n/an/a10.1002/advs.202406242Bio‐Plausible Multimodal Learning with Emerging Neuromorphic DevicesHaonan Sun0Haoxiang Tian1Yihao Hu2Yi Cui3Xinrui Chen4Minyi Xu5Xianfu Wang6Tao Zhou7School of Automation Engineering University of Electronic Science and Technology of China Chengdu 611731 ChinaState Key Laboratory of Electronic Thin Film and Integrated Devices University of Electronic Science and Technology of China Chengdu 611731 ChinaSchool of Automation Engineering University of Electronic Science and Technology of China Chengdu 611731 ChinaState Key Laboratory of Electronic Thin Film and Integrated Devices University of Electronic Science and Technology of China Chengdu 611731 ChinaState Key Laboratory of Electronic Thin Film and Integrated Devices University of Electronic Science and Technology of China Chengdu 611731 ChinaState Key Laboratory of Electronic Thin Film and Integrated Devices University of Electronic Science and Technology of China Chengdu 611731 ChinaState Key Laboratory of Electronic Thin Film and Integrated Devices University of Electronic Science and Technology of China Chengdu 611731 ChinaSchool of Automation Engineering University of Electronic Science and Technology of China Chengdu 611731 ChinaAbstract Multimodal machine learning, as a prospective advancement in artificial intelligence, endeavors to emulate the brain's multimodal learning abilities with the objective to enhance interactions with humans. However, this approach requires simultaneous processing of diverse types of data, leading to increased model complexity, longer training times, and higher energy consumption. Multimodal neuromorphic devices have the capability to preprocess spatio‐temporal information from various physical signals into unified electrical signals with high information density, thereby enabling more biologically plausible multimodal learning with low complexity and high energy‐efficiency. Here, this work conducts a comparison between the expression of multimodal machine learning and multimodal neuromorphic computing, followed by an overview of the key characteristics associated with multimodal neuromorphic devices. The bio‐plausible operational principles and the multimodal learning abilities of emerging devices are examined, which are classified into heterogeneous and homogeneous multimodal neuromorphic devices. Subsequently, this work provides a detailed description of the multimodal learning capabilities demonstrated by neuromorphic circuits and their respective applications. Finally, this work highlights the limitations and challenges of multimodal neuromorphic computing in order to hopefully provide insight into potential future research directions.https://doi.org/10.1002/advs.202406242multifunctional integrationmultimodal learningmultiterminal deviceneuromorphic computing |
| spellingShingle | Haonan Sun Haoxiang Tian Yihao Hu Yi Cui Xinrui Chen Minyi Xu Xianfu Wang Tao Zhou Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices Advanced Science multifunctional integration multimodal learning multiterminal device neuromorphic computing |
| title | Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices |
| title_full | Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices |
| title_fullStr | Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices |
| title_full_unstemmed | Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices |
| title_short | Bio‐Plausible Multimodal Learning with Emerging Neuromorphic Devices |
| title_sort | bio plausible multimodal learning with emerging neuromorphic devices |
| topic | multifunctional integration multimodal learning multiterminal device neuromorphic computing |
| url | https://doi.org/10.1002/advs.202406242 |
| work_keys_str_mv | AT haonansun bioplausiblemultimodallearningwithemergingneuromorphicdevices AT haoxiangtian bioplausiblemultimodallearningwithemergingneuromorphicdevices AT yihaohu bioplausiblemultimodallearningwithemergingneuromorphicdevices AT yicui bioplausiblemultimodallearningwithemergingneuromorphicdevices AT xinruichen bioplausiblemultimodallearningwithemergingneuromorphicdevices AT minyixu bioplausiblemultimodallearningwithemergingneuromorphicdevices AT xianfuwang bioplausiblemultimodallearningwithemergingneuromorphicdevices AT taozhou bioplausiblemultimodallearningwithemergingneuromorphicdevices |