LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system
On the whole, the present microgrid constitutes numerous actors in highly decentralized environments and liberalized electricity markets. The networked microgrid system must be capable of detecting electricity price changes and unknown variations in the presence of rare and extreme events. The netwo...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIMS Press
2025-03-01
|
| Series: | AIMS Electronics and Electrical Engineering |
| Subjects: | |
| Online Access: | https://www.aimspress.com/article/doi/10.3934/electreng.2025009 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849325297931911168 |
|---|---|
| author | Desh Deepak Sharma Ramesh C Bansal |
| author_facet | Desh Deepak Sharma Ramesh C Bansal |
| author_sort | Desh Deepak Sharma |
| collection | DOAJ |
| description | On the whole, the present microgrid constitutes numerous actors in highly decentralized environments and liberalized electricity markets. The networked microgrid system must be capable of detecting electricity price changes and unknown variations in the presence of rare and extreme events. The networked microgrid system comprised of interconnected microgrids must be adaptive and resilient to undesirable environmental conditions such as the occurrence of different kinds of faults and interruptions in the main grid supply. The uncertainties and stochasticity in the load and distributed generation are considered. In this study, we propose resilient energy trading incorporating DC-OPF, which takes generator failures and line outages (topology change) into account. This paper proposes a design of Long Short-Term Memory (LSTM) - soft actor-critic (SAC) reinforcement learning for the development of a platform to obtain resilient peer-to-peer energy trading in networked microgrid systems during extreme events. A Markov Decision Process (MDP) is used to develop the reinforcement learning-based resilient energy trade process that includes the state transition probability and a grid resilience factor for networked microgrid systems. LSTM-SAC continuously refines policies in real-time, thus ensuring optimal trading strategies in rapidly changing energy markets. The LSTM networks have been used to estimate the optimal Q-values in soft actor-critic reinforcement learning. This learning mechanism takes care of the out-of-range estimates of Q-values while reducing the gradient problems. The optimal actions are decided with maximized rewards for peer-to-peer resilient energy trading. The networked microgrid system is trained with the proposed learning mechanism for resilient energy trading. The proposed LSTM-SAC reinforcement learning is tested on a networked microgrid system comprised of IEEE 14 bus systems. |
| format | Article |
| id | doaj-art-dfa96e58ca9d41d1a71c3c6efb1c7a59 |
| institution | Kabale University |
| issn | 2578-1588 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | AIMS Press |
| record_format | Article |
| series | AIMS Electronics and Electrical Engineering |
| spelling | doaj-art-dfa96e58ca9d41d1a71c3c6efb1c7a592025-08-20T03:48:27ZengAIMS PressAIMS Electronics and Electrical Engineering2578-15882025-03-019216519110.3934/electreng.2025009LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid systemDesh Deepak Sharma0Ramesh C Bansal1Department of Electrical Engineering, MJP Rohilkhnad University, BareillyElectrical Engineering Department, University of Sharjah, Sharjah, United Arab EmiratesOn the whole, the present microgrid constitutes numerous actors in highly decentralized environments and liberalized electricity markets. The networked microgrid system must be capable of detecting electricity price changes and unknown variations in the presence of rare and extreme events. The networked microgrid system comprised of interconnected microgrids must be adaptive and resilient to undesirable environmental conditions such as the occurrence of different kinds of faults and interruptions in the main grid supply. The uncertainties and stochasticity in the load and distributed generation are considered. In this study, we propose resilient energy trading incorporating DC-OPF, which takes generator failures and line outages (topology change) into account. This paper proposes a design of Long Short-Term Memory (LSTM) - soft actor-critic (SAC) reinforcement learning for the development of a platform to obtain resilient peer-to-peer energy trading in networked microgrid systems during extreme events. A Markov Decision Process (MDP) is used to develop the reinforcement learning-based resilient energy trade process that includes the state transition probability and a grid resilience factor for networked microgrid systems. LSTM-SAC continuously refines policies in real-time, thus ensuring optimal trading strategies in rapidly changing energy markets. The LSTM networks have been used to estimate the optimal Q-values in soft actor-critic reinforcement learning. This learning mechanism takes care of the out-of-range estimates of Q-values while reducing the gradient problems. The optimal actions are decided with maximized rewards for peer-to-peer resilient energy trading. The networked microgrid system is trained with the proposed learning mechanism for resilient energy trading. The proposed LSTM-SAC reinforcement learning is tested on a networked microgrid system comprised of IEEE 14 bus systems.https://www.aimspress.com/article/doi/10.3934/electreng.2025009dc optimal power flowlstmnetworked microgridsreinforcement learningresilient energy tradingsac |
| spellingShingle | Desh Deepak Sharma Ramesh C Bansal LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system AIMS Electronics and Electrical Engineering dc optimal power flow lstm networked microgrids reinforcement learning resilient energy trading sac |
| title | LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system |
| title_full | LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system |
| title_fullStr | LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system |
| title_full_unstemmed | LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system |
| title_short | LSTM-SAC reinforcement learning based resilient energy trading for networked microgrid system |
| title_sort | lstm sac reinforcement learning based resilient energy trading for networked microgrid system |
| topic | dc optimal power flow lstm networked microgrids reinforcement learning resilient energy trading sac |
| url | https://www.aimspress.com/article/doi/10.3934/electreng.2025009 |
| work_keys_str_mv | AT deshdeepaksharma lstmsacreinforcementlearningbasedresilientenergytradingfornetworkedmicrogridsystem AT rameshcbansal lstmsacreinforcementlearningbasedresilientenergytradingfornetworkedmicrogridsystem |