Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model
IntroductionThe hippocampal formation exhibits complex and context-dependent activity patterns and dynamics, e.g., place cell activity during spatial navigation in rodents or remapping of place fields when the animal switches between contexts. Furthermore, rodents show context-dependent renewal of e...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Computational Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fncom.2024.1462110/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841527850223534080 |
|---|---|
| author | David Kappel Sen Cheng |
| author_facet | David Kappel Sen Cheng |
| author_sort | David Kappel |
| collection | DOAJ |
| description | IntroductionThe hippocampal formation exhibits complex and context-dependent activity patterns and dynamics, e.g., place cell activity during spatial navigation in rodents or remapping of place fields when the animal switches between contexts. Furthermore, rodents show context-dependent renewal of extinguished behavior. However, the link between context-dependent neural codes and context-dependent renewal is not fully understood.MethodsWe use a deep neural network-based reinforcement learning agent to study the learning dynamics that occur during spatial learning and context switching in a simulated ABA extinction and renewal paradigm in a 3D virtual environment.ResultsDespite its simplicity, the network exhibits a number of features typically found in the CA1 and CA3 regions of the hippocampus. A significant proportion of neurons in deeper layers of the network are tuned to a specific spatial position of the agent in the environment—similar to place cells in the hippocampus. These complex spatial representations and dynamics occur spontaneously in the hidden layer of a deep network during learning. These spatial representations exhibit global remapping when the agent is exposed to a new context. The spatial maps are restored when the agent returns to the previous context, accompanied by renewal of the conditioned behavior. Remapping is facilitated by memory replay of experiences during training.DiscussionOur results show that integrated codes that jointly represent spatial and task-relevant contextual variables are the mechanism underlying renewal in a simulated DQN agent. |
| format | Article |
| id | doaj-art-b7ffe7c2cc5b406da44bb4d1c327c19d |
| institution | Kabale University |
| issn | 1662-5188 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computational Neuroscience |
| spelling | doaj-art-b7ffe7c2cc5b406da44bb4d1c327c19d2025-01-15T06:10:50ZengFrontiers Media S.A.Frontiers in Computational Neuroscience1662-51882025-01-011810.3389/fncom.2024.14621101462110Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning modelDavid KappelSen ChengIntroductionThe hippocampal formation exhibits complex and context-dependent activity patterns and dynamics, e.g., place cell activity during spatial navigation in rodents or remapping of place fields when the animal switches between contexts. Furthermore, rodents show context-dependent renewal of extinguished behavior. However, the link between context-dependent neural codes and context-dependent renewal is not fully understood.MethodsWe use a deep neural network-based reinforcement learning agent to study the learning dynamics that occur during spatial learning and context switching in a simulated ABA extinction and renewal paradigm in a 3D virtual environment.ResultsDespite its simplicity, the network exhibits a number of features typically found in the CA1 and CA3 regions of the hippocampus. A significant proportion of neurons in deeper layers of the network are tuned to a specific spatial position of the agent in the environment—similar to place cells in the hippocampus. These complex spatial representations and dynamics occur spontaneously in the hidden layer of a deep network during learning. These spatial representations exhibit global remapping when the agent is exposed to a new context. The spatial maps are restored when the agent returns to the previous context, accompanied by renewal of the conditioned behavior. Remapping is facilitated by memory replay of experiences during training.DiscussionOur results show that integrated codes that jointly represent spatial and task-relevant contextual variables are the mechanism underlying renewal in a simulated DQN agent.https://www.frontiersin.org/articles/10.3389/fncom.2024.1462110/fullhippocampusglobal remappingreinforcement learningextinction learningplace cell |
| spellingShingle | David Kappel Sen Cheng Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model Frontiers in Computational Neuroscience hippocampus global remapping reinforcement learning extinction learning place cell |
| title | Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model |
| title_full | Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model |
| title_fullStr | Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model |
| title_full_unstemmed | Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model |
| title_short | Global remapping emerges as the mechanism for renewal of context-dependent behavior in a reinforcement learning model |
| title_sort | global remapping emerges as the mechanism for renewal of context dependent behavior in a reinforcement learning model |
| topic | hippocampus global remapping reinforcement learning extinction learning place cell |
| url | https://www.frontiersin.org/articles/10.3389/fncom.2024.1462110/full |
| work_keys_str_mv | AT davidkappel globalremappingemergesasthemechanismforrenewalofcontextdependentbehaviorinareinforcementlearningmodel AT sencheng globalremappingemergesasthemechanismforrenewalofcontextdependentbehaviorinareinforcementlearningmodel |