A Vision-Based End-to-End Reinforcement Learning Framework for Drone Target Tracking
Drone target tracking, which involves instructing drone movement to follow a moving target, encounters several challenges: (1) traditional methods need accurate state estimation of both the drone and target; (2) conventional Proportional–Derivative (PD) controllers require tedious parameter tuning a...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-10-01
|
| Series: | Drones |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-446X/8/11/628 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Drone target tracking, which involves instructing drone movement to follow a moving target, encounters several challenges: (1) traditional methods need accurate state estimation of both the drone and target; (2) conventional Proportional–Derivative (PD) controllers require tedious parameter tuning and struggle with nonlinear properties; and (3) reinforcement learning methods, though promising, rely on the drone’s self-state estimation, adding complexity and computational load and reducing reliability. To address these challenges, this study proposes an innovative model-free end-to-end reinforcement learning framework, the VTD3 (Vision-Based Twin Delayed Deep Deterministic Policy Gradient), for drone target tracking tasks. This framework focuses on controlling the drone to follow a moving target while maintaining a specific distance. VTD3 is a pure vision-based tracking algorithm which integrates the YOLOv8 detector, the BoT-SORT tracking algorithm, and the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. It diminishes reliance on GPS and other sensors while simultaneously enhancing the tracking capability for complex target motion trajectories. In a simulated environment, we assess the tracking performance of VTD3 across four complex target motion trajectories (triangular, square, sawtooth, and square wave, including scenarios with occlusions). The experimental results indicate that our proposed VTD3 reinforcement learning algorithm substantially outperforms conventional PD controllers in drone target tracking applications. Across various target trajectories, the VTD3 algorithm demonstrates a significant reduction in average tracking errors along the X-axis and Y-axis of up to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>34.35</mn><mo>%</mo></mrow></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>45.36</mn><mo>%</mo></mrow></semantics></math></inline-formula>, respectively. Additionally, it achieves a notable improvement of up to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>66.10</mn><mo>%</mo></mrow></semantics></math></inline-formula> in altitude control precision. In terms of motion smoothness, the VTD3 algorithm markedly enhances performance metrics, with improvements of up to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>37.70</mn><mo>%</mo></mrow></semantics></math></inline-formula> in jitter and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>60.64</mn><mo>%</mo></mrow></semantics></math></inline-formula> in Jerk RMS. Empirical results verify the superiority and feasibility of our proposed VTD3 framework for drone target tracking. |
|---|---|
| ISSN: | 2504-446X |