KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters
IntroductionTracking the hidden states of dynamic systems is a fundamental task in signal processing. Recursive Kalman Filters (KF) are widely regarded as an efficient solution for linear and Gaussian systems, offering low computational complexity. However, real-world applications often involve non-...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2024.1460255/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841556761784352768 |
|---|---|
| author | Siyuan Shen Jichen Chen Guanfeng Yu Zhengjun Zhai Pujie Han |
| author_facet | Siyuan Shen Jichen Chen Guanfeng Yu Zhengjun Zhai Pujie Han |
| author_sort | Siyuan Shen |
| collection | DOAJ |
| description | IntroductionTracking the hidden states of dynamic systems is a fundamental task in signal processing. Recursive Kalman Filters (KF) are widely regarded as an efficient solution for linear and Gaussian systems, offering low computational complexity. However, real-world applications often involve non-linear dynamics, making it challenging for traditional Kalman Filters to achieve accurate state estimation. Additionally, the accurate modeling of system dynamics and noise in practical scenarios is often difficult. To address these limitations, we propose the KalmanFormer, a hybrid model-driven and data-driven state estimator. By leveraging data, the KalmanFormer promotes the performance of state estimation under non-linear conditions and partial information scenarios.MethodsThe proposed KalmanFormer integrates classical Kalman Filter with a Transformer framework. Specifically, it utilizes the Transformer to learn the Kalman Gain directly from data without requiring prior knowledge of noise parameters. The learned Kalman Gain is then incorporated into the standard Kalman Filter workflow, enabling the system to better handle non-linearities and model mismatches. The hybrid approach combines the strengths of data-driven learning and model-driven methodologies to achieve robust state estimation.Results and discussionTo evaluate the effectiveness of KalmanFormer, we conducted numerical experiments in both synthetic and real-world dataset. The results demonstrate that KalmanFormer outperforms the classical Extended Kalman Filter (EKF) in the same settings. It achieves superior accuracy in tracking hidden states, demonstrating resilience to non-linearities and imprecise system models. |
| format | Article |
| id | doaj-art-f4131359f2ba4fa582f0eea4d547ae08 |
| institution | Kabale University |
| issn | 1662-5218 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj-art-f4131359f2ba4fa582f0eea4d547ae082025-01-07T06:40:50ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182025-01-011810.3389/fnbot.2024.14602551460255KalmanFormer: using transformer to model the Kalman Gain in Kalman FiltersSiyuan Shen0Jichen Chen1Guanfeng Yu2Zhengjun Zhai3Pujie Han4School of Computer Science, Northwestern Polytechnical University, Xi'an, ChinaFourth Technical Department, Xi'an Microelectronics Technology Institute, Xi'an, ChinaResearch Office 16, AVIC Xi'an Aeronautics Computing Technique Research Institute, Xi'an, ChinaSchool of Computer Science, Northwestern Polytechnical University, Xi'an, ChinaSoftware Engineering College, Zhengzhou University of Light Industry, Zhengzhou, ChinaIntroductionTracking the hidden states of dynamic systems is a fundamental task in signal processing. Recursive Kalman Filters (KF) are widely regarded as an efficient solution for linear and Gaussian systems, offering low computational complexity. However, real-world applications often involve non-linear dynamics, making it challenging for traditional Kalman Filters to achieve accurate state estimation. Additionally, the accurate modeling of system dynamics and noise in practical scenarios is often difficult. To address these limitations, we propose the KalmanFormer, a hybrid model-driven and data-driven state estimator. By leveraging data, the KalmanFormer promotes the performance of state estimation under non-linear conditions and partial information scenarios.MethodsThe proposed KalmanFormer integrates classical Kalman Filter with a Transformer framework. Specifically, it utilizes the Transformer to learn the Kalman Gain directly from data without requiring prior knowledge of noise parameters. The learned Kalman Gain is then incorporated into the standard Kalman Filter workflow, enabling the system to better handle non-linearities and model mismatches. The hybrid approach combines the strengths of data-driven learning and model-driven methodologies to achieve robust state estimation.Results and discussionTo evaluate the effectiveness of KalmanFormer, we conducted numerical experiments in both synthetic and real-world dataset. The results demonstrate that KalmanFormer outperforms the classical Extended Kalman Filter (EKF) in the same settings. It achieves superior accuracy in tracking hidden states, demonstrating resilience to non-linearities and imprecise system models.https://www.frontiersin.org/articles/10.3389/fnbot.2024.1460255/fullKalman Filterdeep learningtransformerKalman Gainsupervised paradigm |
| spellingShingle | Siyuan Shen Jichen Chen Guanfeng Yu Zhengjun Zhai Pujie Han KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters Frontiers in Neurorobotics Kalman Filter deep learning transformer Kalman Gain supervised paradigm |
| title | KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters |
| title_full | KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters |
| title_fullStr | KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters |
| title_full_unstemmed | KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters |
| title_short | KalmanFormer: using transformer to model the Kalman Gain in Kalman Filters |
| title_sort | kalmanformer using transformer to model the kalman gain in kalman filters |
| topic | Kalman Filter deep learning transformer Kalman Gain supervised paradigm |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2024.1460255/full |
| work_keys_str_mv | AT siyuanshen kalmanformerusingtransformertomodelthekalmangaininkalmanfilters AT jichenchen kalmanformerusingtransformertomodelthekalmangaininkalmanfilters AT guanfengyu kalmanformerusingtransformertomodelthekalmangaininkalmanfilters AT zhengjunzhai kalmanformerusingtransformertomodelthekalmangaininkalmanfilters AT pujiehan kalmanformerusingtransformertomodelthekalmangaininkalmanfilters |