MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data
Abstract Enhanced interaction between solar‐wind and Earth's magnetosphere can cause space weather and geomagnetic storms that have the potential to damage critical technologies, such as magnetic navigation, radio communications, and power grids. The severity of a geomagnetic storm is measured...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-10-01
|
| Series: | Space Weather |
| Online Access: | https://doi.org/10.1029/2023SW003514 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841536375284826112 |
|---|---|
| author | Manoj Nair Rob Redmon Li‐Yin Young Arnaud Chulliat Belinda Trotta Christine Chung Greg Lipstein Isaac Slavitt |
| author_facet | Manoj Nair Rob Redmon Li‐Yin Young Arnaud Chulliat Belinda Trotta Christine Chung Greg Lipstein Isaac Slavitt |
| author_sort | Manoj Nair |
| collection | DOAJ |
| description | Abstract Enhanced interaction between solar‐wind and Earth's magnetosphere can cause space weather and geomagnetic storms that have the potential to damage critical technologies, such as magnetic navigation, radio communications, and power grids. The severity of a geomagnetic storm is measured using the disturbance‐storm‐time (Dst) index. The Dst index is calculated by averaging the horizontal component of the magnetic field observed at four near‐equatorial observatories and is used to drive geomagnetic disturbance models. As a key specification of the magnetospheric dynamics, the Dst index is used to drive geomagnetic disturbance models such as the High Definition Geomagnetic Model—Real Time. Since 1975, forecasting models have been proposed to forecast Dst solely from solar wind observations at the Lagrangian‐1 position. However, while the recent Machine‐Learning (ML) models generally perform better than other approaches, many are unsuitable for operational use. Recent exponential growth in data‐science research and the democratization of ML tools have opened up the possibility of crowd‐sourcing specific problem‐solving tasks with clear constraints and evaluation metrics. To this end, National Oceanic and Atmospheric Administration (NOAA)'s National Centers for Environmental Information and the University of Colorado's Cooperative Institute for Research in Environmental Sciences conducted an open data‐science challenge called “MagNet: Model the Geomagnetic Field.” The challenge attracted 622 participants, resulting in 1,197 model submissions that used various ML approaches. The top models that met the evaluation criteria are operationally viable and retrainable and suitable for NOAA's operational needs. The paper summarizes the competition results and lessons learned. |
| format | Article |
| id | doaj-art-ef2a7545f01a4dc7954c3710ad0d30e2 |
| institution | Kabale University |
| issn | 1542-7390 |
| language | English |
| publishDate | 2023-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | Space Weather |
| spelling | doaj-art-ef2a7545f01a4dc7954c3710ad0d30e22025-01-14T16:31:16ZengWileySpace Weather1542-73902023-10-012110n/an/a10.1029/2023SW003514MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind DataManoj Nair0Rob Redmon1Li‐Yin Young2Arnaud Chulliat3Belinda Trotta4Christine Chung5Greg Lipstein6Isaac Slavitt7Cooperative Institute for Research in Environmental Sciences University of Colorado Boulder CO USANOAA's National Centers for Environmental Information Boulder CO USACooperative Institute for Research in Environmental Sciences University of Colorado Boulder CO USACooperative Institute for Research in Environmental Sciences University of Colorado Boulder CO USABureau of Meteorology Melbourne VIC AustraliaDrivenData Inc. Denver CO USADrivenData Inc. Denver CO USADrivenData Inc. Denver CO USAAbstract Enhanced interaction between solar‐wind and Earth's magnetosphere can cause space weather and geomagnetic storms that have the potential to damage critical technologies, such as magnetic navigation, radio communications, and power grids. The severity of a geomagnetic storm is measured using the disturbance‐storm‐time (Dst) index. The Dst index is calculated by averaging the horizontal component of the magnetic field observed at four near‐equatorial observatories and is used to drive geomagnetic disturbance models. As a key specification of the magnetospheric dynamics, the Dst index is used to drive geomagnetic disturbance models such as the High Definition Geomagnetic Model—Real Time. Since 1975, forecasting models have been proposed to forecast Dst solely from solar wind observations at the Lagrangian‐1 position. However, while the recent Machine‐Learning (ML) models generally perform better than other approaches, many are unsuitable for operational use. Recent exponential growth in data‐science research and the democratization of ML tools have opened up the possibility of crowd‐sourcing specific problem‐solving tasks with clear constraints and evaluation metrics. To this end, National Oceanic and Atmospheric Administration (NOAA)'s National Centers for Environmental Information and the University of Colorado's Cooperative Institute for Research in Environmental Sciences conducted an open data‐science challenge called “MagNet: Model the Geomagnetic Field.” The challenge attracted 622 participants, resulting in 1,197 model submissions that used various ML approaches. The top models that met the evaluation criteria are operationally viable and retrainable and suitable for NOAA's operational needs. The paper summarizes the competition results and lessons learned.https://doi.org/10.1029/2023SW003514 |
| spellingShingle | Manoj Nair Rob Redmon Li‐Yin Young Arnaud Chulliat Belinda Trotta Christine Chung Greg Lipstein Isaac Slavitt MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data Space Weather |
| title | MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data |
| title_full | MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data |
| title_fullStr | MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data |
| title_full_unstemmed | MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data |
| title_short | MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data |
| title_sort | magnet a data science competition to predict disturbance storm time index dst from solar wind data |
| url | https://doi.org/10.1029/2023SW003514 |
| work_keys_str_mv | AT manojnair magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT robredmon magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT liyinyoung magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT arnaudchulliat magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT belindatrotta magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT christinechung magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT greglipstein magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata AT isaacslavitt magnetadatasciencecompetitiontopredictdisturbancestormtimeindexdstfromsolarwinddata |