Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation
Online Object Detection (OOD) requires learning new object categories from a stream of images, similar to an agent exploring new environments. In this context, the widely used architecture Faster R-CNN (Region Convolutional Neural Network) faces catastrophic forgetting: the acquisition of new knowle...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10817562/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841542591022104576 |
|---|---|
| author | Baptiste Wagner Denis Pellerin Sylvain Huet |
| author_facet | Baptiste Wagner Denis Pellerin Sylvain Huet |
| author_sort | Baptiste Wagner |
| collection | DOAJ |
| description | Online Object Detection (OOD) requires learning new object categories from a stream of images, similar to an agent exploring new environments. In this context, the widely used architecture Faster R-CNN (Region Convolutional Neural Network) faces catastrophic forgetting: the acquisition of new knowledge leads to the loss of previously learned information. In this paper, we investigate the learning and forgetting mechanisms of Faster R-CNN in OOD through three main contributions. First, we observe that the forgetting curves of the Faster R-CNN exhibit patterns similar to those described in human memory studies by Hermann Ebbinghaus: knowledge is lost exponentially over time and recall improves knowledge retention. Second, we present a new methodology to analyse the Faster R-CNN architecture and quantify forgetting across the Faster R-CNN components. We show that forgetting is mainly localised in the Softmax classification layer. Finally, we propose a new training strategy for OOD called Configurable Recall (CR). CR performs recalls on old data using images stored in a memory buffer with variable frequency and recall length to ensure efficient learning. CR also masks the logits of old objects in the softmax classification layer to mitigate forgetting. We evaluate our strategy against state-of-the-art methods on three OOD benchmarks. We analyse the effectiveness of different types of recall in mitigating forgetting and show that CR outperforms existing methods. |
| format | Article |
| id | doaj-art-e40df798de554b1e97e6846957df2573 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e40df798de554b1e97e6846957df25732025-01-14T00:02:09ZengIEEEIEEE Access2169-35362025-01-01136067607910.1109/ACCESS.2024.352363710817562Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and MitigationBaptiste Wagner0https://orcid.org/0009-0001-4863-2731Denis Pellerin1https://orcid.org/0000-0002-3792-1706Sylvain Huet2https://orcid.org/0000-0002-2989-3610CNRS, Grenoble INP, GIPSA-Laboratory, Université Grenoble Alpes, Grenoble, FranceCNRS, Grenoble INP, GIPSA-Laboratory, Université Grenoble Alpes, Grenoble, FranceCNRS, Grenoble INP, GIPSA-Laboratory, Université Grenoble Alpes, Grenoble, FranceOnline Object Detection (OOD) requires learning new object categories from a stream of images, similar to an agent exploring new environments. In this context, the widely used architecture Faster R-CNN (Region Convolutional Neural Network) faces catastrophic forgetting: the acquisition of new knowledge leads to the loss of previously learned information. In this paper, we investigate the learning and forgetting mechanisms of Faster R-CNN in OOD through three main contributions. First, we observe that the forgetting curves of the Faster R-CNN exhibit patterns similar to those described in human memory studies by Hermann Ebbinghaus: knowledge is lost exponentially over time and recall improves knowledge retention. Second, we present a new methodology to analyse the Faster R-CNN architecture and quantify forgetting across the Faster R-CNN components. We show that forgetting is mainly localised in the Softmax classification layer. Finally, we propose a new training strategy for OOD called Configurable Recall (CR). CR performs recalls on old data using images stored in a memory buffer with variable frequency and recall length to ensure efficient learning. CR also masks the logits of old objects in the softmax classification layer to mitigate forgetting. We evaluate our strategy against state-of-the-art methods on three OOD benchmarks. We analyse the effectiveness of different types of recall in mitigating forgetting and show that CR outperforms existing methods.https://ieeexplore.ieee.org/document/10817562/Catastrophic forgettingonline object detectionfaster R-CNNEbbinghaus forgetting curvenatural replayonline continual learning |
| spellingShingle | Baptiste Wagner Denis Pellerin Sylvain Huet Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation IEEE Access Catastrophic forgetting online object detection faster R-CNN Ebbinghaus forgetting curve natural replay online continual learning |
| title | Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation |
| title_full | Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation |
| title_fullStr | Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation |
| title_full_unstemmed | Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation |
| title_short | Studying Forgetting in Faster R-CNN for Online Object Detection: Analysis Scenarios, Localization in the Architecture, and Mitigation |
| title_sort | studying forgetting in faster r cnn for online object detection analysis scenarios localization in the architecture and mitigation |
| topic | Catastrophic forgetting online object detection faster R-CNN Ebbinghaus forgetting curve natural replay online continual learning |
| url | https://ieeexplore.ieee.org/document/10817562/ |
| work_keys_str_mv | AT baptistewagner studyingforgettinginfasterrcnnforonlineobjectdetectionanalysisscenarioslocalizationinthearchitectureandmitigation AT denispellerin studyingforgettinginfasterrcnnforonlineobjectdetectionanalysisscenarioslocalizationinthearchitectureandmitigation AT sylvainhuet studyingforgettinginfasterrcnnforonlineobjectdetectionanalysisscenarioslocalizationinthearchitectureandmitigation |