Connecting dreams with visual brainstorming instruction
Abstract Recent breakthroughs in understanding the human brain have revealed its impressive ability to efficiently process and interpret human thoughts, opening up the possibility of intervening in brain signals. In this paper, we aim to develop a straightforward framework that uses other modalities...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2025-07-01
|
| Series: | Visual Intelligence |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s44267-025-00081-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849238246382370816 |
|---|---|
| author | Yasheng Sun Bohan Li Mingchen Zhuge Deng-Ping Fan Salman Khan Fahad Shahbaz Khan Hideki Koike |
| author_facet | Yasheng Sun Bohan Li Mingchen Zhuge Deng-Ping Fan Salman Khan Fahad Shahbaz Khan Hideki Koike |
| author_sort | Yasheng Sun |
| collection | DOAJ |
| description | Abstract Recent breakthroughs in understanding the human brain have revealed its impressive ability to efficiently process and interpret human thoughts, opening up the possibility of intervening in brain signals. In this paper, we aim to develop a straightforward framework that uses other modalities, such as natural language, to translate the original “dreamland”. We present DreamConnect, employing a dual-stream diffusion framework to manipulate visually stimulated brain signals. By integrating an asynchronous diffusion strategy, our framework establishes an effective interface with human “dreams”, and progressively refines their final image synthesis. Through extensive experiments, we demonstrate the efficacy of our method to accurately direct human brain signals in desired directions, ultimately enabling concept manipulation through direct manipulation of the functional magnetic resonance imaging (fMRI) signals. We hope that this work will motivate the use of brain signals in human-computer interaction applications. |
| format | Article |
| id | doaj-art-b3572b8768fc40a7b6b7c1c331a14b93 |
| institution | Kabale University |
| issn | 2097-3330 2731-9008 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Springer |
| record_format | Article |
| series | Visual Intelligence |
| spelling | doaj-art-b3572b8768fc40a7b6b7c1c331a14b932025-08-20T04:01:42ZengSpringerVisual Intelligence2097-33302731-90082025-07-013111810.1007/s44267-025-00081-2Connecting dreams with visual brainstorming instructionYasheng Sun0Bohan Li1Mingchen Zhuge2Deng-Ping Fan3Salman Khan4Fahad Shahbaz Khan5Hideki Koike6School of Computing, Tokyo Institute of TechnologyCollege of Computer Science, Shanghai Jiao Tong UniversityCenter of Excellence for Generative AI, KAUSTCollege of Computer Science, Nankai UniversityComputer Vision Group, MBZUAIComputer Vision Group, MBZUAISchool of Computing, Tokyo Institute of TechnologyAbstract Recent breakthroughs in understanding the human brain have revealed its impressive ability to efficiently process and interpret human thoughts, opening up the possibility of intervening in brain signals. In this paper, we aim to develop a straightforward framework that uses other modalities, such as natural language, to translate the original “dreamland”. We present DreamConnect, employing a dual-stream diffusion framework to manipulate visually stimulated brain signals. By integrating an asynchronous diffusion strategy, our framework establishes an effective interface with human “dreams”, and progressively refines their final image synthesis. Through extensive experiments, we demonstrate the efficacy of our method to accurately direct human brain signals in desired directions, ultimately enabling concept manipulation through direct manipulation of the functional magnetic resonance imaging (fMRI) signals. We hope that this work will motivate the use of brain signals in human-computer interaction applications.https://doi.org/10.1007/s44267-025-00081-2Functional magnetic resonance imaging (fMRI)Brain-to-image generationDiffusion modelsLarge language model (LLM) |
| spellingShingle | Yasheng Sun Bohan Li Mingchen Zhuge Deng-Ping Fan Salman Khan Fahad Shahbaz Khan Hideki Koike Connecting dreams with visual brainstorming instruction Visual Intelligence Functional magnetic resonance imaging (fMRI) Brain-to-image generation Diffusion models Large language model (LLM) |
| title | Connecting dreams with visual brainstorming instruction |
| title_full | Connecting dreams with visual brainstorming instruction |
| title_fullStr | Connecting dreams with visual brainstorming instruction |
| title_full_unstemmed | Connecting dreams with visual brainstorming instruction |
| title_short | Connecting dreams with visual brainstorming instruction |
| title_sort | connecting dreams with visual brainstorming instruction |
| topic | Functional magnetic resonance imaging (fMRI) Brain-to-image generation Diffusion models Large language model (LLM) |
| url | https://doi.org/10.1007/s44267-025-00081-2 |
| work_keys_str_mv | AT yashengsun connectingdreamswithvisualbrainstorminginstruction AT bohanli connectingdreamswithvisualbrainstorminginstruction AT mingchenzhuge connectingdreamswithvisualbrainstorminginstruction AT dengpingfan connectingdreamswithvisualbrainstorminginstruction AT salmankhan connectingdreamswithvisualbrainstorminginstruction AT fahadshahbazkhan connectingdreamswithvisualbrainstorminginstruction AT hidekikoike connectingdreamswithvisualbrainstorminginstruction |