Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning
With the liberalization of the retail market, customers can sell their demand response (DR) resources to the distribution company (Disco) through the DR aggregator (DRA). In this paper, an intelligent DR resource trading framework between Disco and DRA is proposed by exploiting the benefits of deep...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
China electric power research institute
2024-01-01
|
| Series: | CSEE Journal of Power and Energy Systems |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9535402/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841533390412578816 |
|---|---|
| author | Yufan Zhang Qian Ai Zhaoyu Li |
| author_facet | Yufan Zhang Qian Ai Zhaoyu Li |
| author_sort | Yufan Zhang |
| collection | DOAJ |
| description | With the liberalization of the retail market, customers can sell their demand response (DR) resources to the distribution company (Disco) through the DR aggregator (DRA). In this paper, an intelligent DR resource trading framework between Disco and DRA is proposed by exploiting the benefits of deep reinforcement learning (DRL). The hierarchical decision process of the two players is modeled as a Stackelberg game. In the game, Disco is the leader who determines the retail price while DRA is the follower who responds to it. To protect their privacy, a dueling deep Q-network (dueling DQN) is then constructed to model the bi-Ievel Stackelberg game, such that the lower-level problem doesn't need to reveal its detailed model to the upper-level. In the learning process, the uncertainties from the DRA's baseline load and wind power are considered. In order to boost the robustness against the estimation error, the baseline load is discretized into symbols before being used as the input states of the dueling DQN. And to mitigate the uncertainty of wind power, the scenario-based method is introduced when designing the reward. We demonstrate that the proposed dueling DQN-based method has good performance and is more robust against uncertainties. |
| format | Article |
| id | doaj-art-1d9b492098a146cc9491b96d5ddb018d |
| institution | Kabale University |
| issn | 2096-0042 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | China electric power research institute |
| record_format | Article |
| series | CSEE Journal of Power and Energy Systems |
| spelling | doaj-art-1d9b492098a146cc9491b96d5ddb018d2025-01-16T00:02:20ZengChina electric power research instituteCSEE Journal of Power and Energy Systems2096-00422024-01-011062621263010.17775/CSEEJPES.2020.055409535402Intelligent Demand Response Resource Trading Using Deep Reinforcement LearningYufan Zhang0Qian Ai1Zhaoyu Li2Ministry of Education, Shanghai Jiao Tong University,Key Laboratory of Control of Power Transmission and Conversion,Department of Electrical Engineering,Shanghai,China,200240Ministry of Education, Shanghai Jiao Tong University,Key Laboratory of Control of Power Transmission and Conversion,Department of Electrical Engineering,Shanghai,China,200240Ministry of Education, Shanghai Jiao Tong University,Key Laboratory of Control of Power Transmission and Conversion,Department of Electrical Engineering,Shanghai,China,200240With the liberalization of the retail market, customers can sell their demand response (DR) resources to the distribution company (Disco) through the DR aggregator (DRA). In this paper, an intelligent DR resource trading framework between Disco and DRA is proposed by exploiting the benefits of deep reinforcement learning (DRL). The hierarchical decision process of the two players is modeled as a Stackelberg game. In the game, Disco is the leader who determines the retail price while DRA is the follower who responds to it. To protect their privacy, a dueling deep Q-network (dueling DQN) is then constructed to model the bi-Ievel Stackelberg game, such that the lower-level problem doesn't need to reveal its detailed model to the upper-level. In the learning process, the uncertainties from the DRA's baseline load and wind power are considered. In order to boost the robustness against the estimation error, the baseline load is discretized into symbols before being used as the input states of the dueling DQN. And to mitigate the uncertainty of wind power, the scenario-based method is introduced when designing the reward. We demonstrate that the proposed dueling DQN-based method has good performance and is more robust against uncertainties.https://ieeexplore.ieee.org/document/9535402/Demand responseeconomic interactionreinforcement learningstackelberg gameuncertainty |
| spellingShingle | Yufan Zhang Qian Ai Zhaoyu Li Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning CSEE Journal of Power and Energy Systems Demand response economic interaction reinforcement learning stackelberg game uncertainty |
| title | Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning |
| title_full | Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning |
| title_fullStr | Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning |
| title_full_unstemmed | Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning |
| title_short | Intelligent Demand Response Resource Trading Using Deep Reinforcement Learning |
| title_sort | intelligent demand response resource trading using deep reinforcement learning |
| topic | Demand response economic interaction reinforcement learning stackelberg game uncertainty |
| url | https://ieeexplore.ieee.org/document/9535402/ |
| work_keys_str_mv | AT yufanzhang intelligentdemandresponseresourcetradingusingdeepreinforcementlearning AT qianai intelligentdemandresponseresourcetradingusingdeepreinforcementlearning AT zhaoyuli intelligentdemandresponseresourcetradingusingdeepreinforcementlearning |