Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System
This paper discusses the development and performance evaluation of a single-inverter-based microgrid control system using a Deep Reinforcement Learning (DRL) agent trained with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The controller is specifically designed to address unp...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11039624/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839638796615811072 |
|---|---|
| author | Noer Fadzri Perdana Dinata Makbul Anwari Muhammad Ramli Muhammad Irfan Jambak Prisma Megantoro Muhammad Abu Bakar Sidik |
| author_facet | Noer Fadzri Perdana Dinata Makbul Anwari Muhammad Ramli Muhammad Irfan Jambak Prisma Megantoro Muhammad Abu Bakar Sidik |
| author_sort | Noer Fadzri Perdana Dinata |
| collection | DOAJ |
| description | This paper discusses the development and performance evaluation of a single-inverter-based microgrid control system using a Deep Reinforcement Learning (DRL) agent trained with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The controller is specifically designed to address unplanned transitions during fluctuating load conditions and transitions between grid-tied and islanded modes. An analysis of Final Exploitation Rates (FER) ranging from 5% to 1% was conducted to assess transient responses and steady-state performance. A comparison with a traditional Proportional-Integral-Derivative (PID) controller reveals that while the DRL agent with 2%-FER demonstrates strong performance in managing transient events, the PID controller provides superior stability in steady-state operation. Key performance indicators, including Rise Time, Settling Time, Settling Min/Max, Peak Time, and Mean Absolute Deviation (MAD), were examined. The results show that the DRL agent with 2%-FER effectively limits frequency deviations to ±1% and voltage deviations to within ±2.5% during transitions, whereas agents with FER 3-5% displayed instability with deviations of up to 2% for frequency and 4% for voltage. The PID controller, however, maintained more consistent steady-state performance, with frequency tightly distributed around 49.95-50.05 Hz and voltage near 1 pu. These findings suggest that DRL-based control is better suited for handling unexpected switching events, where proactive adjustment is crucial, while PID control is more effective for steady-state conditions where a passive control approach accommodates system dynamics more naturally. |
| format | Article |
| id | doaj-art-147de1e49fd6493a865d088831cf9472 |
| institution | Matheson Library |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-147de1e49fd6493a865d088831cf94722025-07-04T23:00:40ZengIEEEIEEE Access2169-35362025-01-011311273111274810.1109/ACCESS.2025.358099211039624Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control SystemNoer Fadzri Perdana Dinata0https://orcid.org/0000-0002-9447-4930Makbul Anwari Muhammad Ramli1https://orcid.org/0000-0002-5281-0377Muhammad Irfan Jambak2https://orcid.org/0000-0002-4205-3907Prisma Megantoro3https://orcid.org/0000-0001-5898-4522Muhammad Abu Bakar Sidik4https://orcid.org/0000-0003-4532-8862Faculty of Engineering, Universitas Sriwijaya, Palembang, South Sumatra, IndonesiaResearch Centre for New and Renewable Energy Engineering, Universitas Airlangga, Surabaya, IndonesiaFaculty of Engineering, Universitas Sriwijaya, Palembang, South Sumatra, IndonesiaResearch Centre for New and Renewable Energy Engineering, Universitas Airlangga, Surabaya, IndonesiaFaculty of Engineering, Universitas Sriwijaya, Palembang, South Sumatra, IndonesiaThis paper discusses the development and performance evaluation of a single-inverter-based microgrid control system using a Deep Reinforcement Learning (DRL) agent trained with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The controller is specifically designed to address unplanned transitions during fluctuating load conditions and transitions between grid-tied and islanded modes. An analysis of Final Exploitation Rates (FER) ranging from 5% to 1% was conducted to assess transient responses and steady-state performance. A comparison with a traditional Proportional-Integral-Derivative (PID) controller reveals that while the DRL agent with 2%-FER demonstrates strong performance in managing transient events, the PID controller provides superior stability in steady-state operation. Key performance indicators, including Rise Time, Settling Time, Settling Min/Max, Peak Time, and Mean Absolute Deviation (MAD), were examined. The results show that the DRL agent with 2%-FER effectively limits frequency deviations to ±1% and voltage deviations to within ±2.5% during transitions, whereas agents with FER 3-5% displayed instability with deviations of up to 2% for frequency and 4% for voltage. The PID controller, however, maintained more consistent steady-state performance, with frequency tightly distributed around 49.95-50.05 Hz and voltage near 1 pu. These findings suggest that DRL-based control is better suited for handling unexpected switching events, where proactive adjustment is crucial, while PID control is more effective for steady-state conditions where a passive control approach accommodates system dynamics more naturally.https://ieeexplore.ieee.org/document/11039624/Deep reinforcement learning (DRL)frequency stabilitymicrogridtwin delayed deep deterministic policy gradient (TD3)voltage stability |
| spellingShingle | Noer Fadzri Perdana Dinata Makbul Anwari Muhammad Ramli Muhammad Irfan Jambak Prisma Megantoro Muhammad Abu Bakar Sidik Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System IEEE Access Deep reinforcement learning (DRL) frequency stability microgrid twin delayed deep deterministic policy gradient (TD3) voltage stability |
| title | Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System |
| title_full | Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System |
| title_fullStr | Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System |
| title_full_unstemmed | Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System |
| title_short | Simulation-Based Analysis of DRL Exploitation Rate for Voltage and Frequency Stability in a Single-Inverter Microgrid Control System |
| title_sort | simulation based analysis of drl exploitation rate for voltage and frequency stability in a single inverter microgrid control system |
| topic | Deep reinforcement learning (DRL) frequency stability microgrid twin delayed deep deterministic policy gradient (TD3) voltage stability |
| url | https://ieeexplore.ieee.org/document/11039624/ |
| work_keys_str_mv | AT noerfadzriperdanadinata simulationbasedanalysisofdrlexploitationrateforvoltageandfrequencystabilityinasingleinvertermicrogridcontrolsystem AT makbulanwarimuhammadramli simulationbasedanalysisofdrlexploitationrateforvoltageandfrequencystabilityinasingleinvertermicrogridcontrolsystem AT muhammadirfanjambak simulationbasedanalysisofdrlexploitationrateforvoltageandfrequencystabilityinasingleinvertermicrogridcontrolsystem AT prismamegantoro simulationbasedanalysisofdrlexploitationrateforvoltageandfrequencystabilityinasingleinvertermicrogridcontrolsystem AT muhammadabubakarsidik simulationbasedanalysisofdrlexploitationrateforvoltageandfrequencystabilityinasingleinvertermicrogridcontrolsystem |