BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio
This paper presents a comprehensive study on real-time music rhythm analysis, covering joint beat and downbeat tracking for diverse kinds of music signals. We introduce BeatNet+, a two-stage approach to real-time rhythm analysis built on a previous state-of-the-art method named BeatNet. The main inn...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Ubiquity Press
2024-12-01
|
| Series: | Transactions of the International Society for Music Information Retrieval |
| Subjects: | |
| Online Access: | https://account.transactions.ismir.net/index.php/up-j-tismir/article/view/198 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841554853290049536 |
|---|---|
| author | Mojtaba Heydari Zhiyao Duan |
| author_facet | Mojtaba Heydari Zhiyao Duan |
| author_sort | Mojtaba Heydari |
| collection | DOAJ |
| description | This paper presents a comprehensive study on real-time music rhythm analysis, covering joint beat and downbeat tracking for diverse kinds of music signals. We introduce BeatNet+, a two-stage approach to real-time rhythm analysis built on a previous state-of-the-art method named BeatNet. The main innovation of the proposed method is the auxiliary training strategy that helps the neural network model to learn a representation invariant to the amount of percussive components in the music. Together with other architectural improvements, this strategy significantly improves the model performance for generic music. Another innovation is on the adaptation strategies that help develop real-time rhythm analysis models for challenging music scenarios, including isolated singing voices and non-percussive music. Two adaptation strategies are proposed and experimented with using different neural architectures and training schemes. Comprehensive experiments and comparisons with multiple baselines are conducted, and results show that BeatNet+ achieves superior beat tracking and downbeat tracking F1 scores for generic music, isolated singing voices, and non-percussive audio, with competitive latency and computational complexity. Finally, we release beat and downbeat annotations for two datasets that are designed for other tasks, and revised annotations of three existing datasets. We also release the code repository and pre-trained models on GitHub. |
| format | Article |
| id | doaj-art-393c7ce2039d4e8dad5843c4c498a6d7 |
| institution | Kabale University |
| issn | 2514-3298 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Ubiquity Press |
| record_format | Article |
| series | Transactions of the International Society for Music Information Retrieval |
| spelling | doaj-art-393c7ce2039d4e8dad5843c4c498a6d72025-01-08T08:41:56ZengUbiquity PressTransactions of the International Society for Music Information Retrieval2514-32982024-12-0171274–287274–28710.5334/tismir.198198BeatNet+: Real-Time Rhythm Analysis for Diverse Music AudioMojtaba Heydari0Zhiyao Duan1Department of Electrical and Computer Engineering, University of Rochester, Rochester, NYAssociate Professor of Electrical and Computer Engineering and of Computer Science, University of Rochester, Rochester, NYThis paper presents a comprehensive study on real-time music rhythm analysis, covering joint beat and downbeat tracking for diverse kinds of music signals. We introduce BeatNet+, a two-stage approach to real-time rhythm analysis built on a previous state-of-the-art method named BeatNet. The main innovation of the proposed method is the auxiliary training strategy that helps the neural network model to learn a representation invariant to the amount of percussive components in the music. Together with other architectural improvements, this strategy significantly improves the model performance for generic music. Another innovation is on the adaptation strategies that help develop real-time rhythm analysis models for challenging music scenarios, including isolated singing voices and non-percussive music. Two adaptation strategies are proposed and experimented with using different neural architectures and training schemes. Comprehensive experiments and comparisons with multiple baselines are conducted, and results show that BeatNet+ achieves superior beat tracking and downbeat tracking F1 scores for generic music, isolated singing voices, and non-percussive audio, with competitive latency and computational complexity. Finally, we release beat and downbeat annotations for two datasets that are designed for other tasks, and revised annotations of three existing datasets. We also release the code repository and pre-trained models on GitHub.https://account.transactions.ismir.net/index.php/up-j-tismir/article/view/198real-time beat trackingdownbeat trackingrhythm analysissinging voicesnon-percussive musicbeatnetbeatnet+ |
| spellingShingle | Mojtaba Heydari Zhiyao Duan BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio Transactions of the International Society for Music Information Retrieval real-time beat tracking downbeat tracking rhythm analysis singing voices non-percussive music beatnet beatnet+ |
| title | BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio |
| title_full | BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio |
| title_fullStr | BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio |
| title_full_unstemmed | BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio |
| title_short | BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio |
| title_sort | beatnet real time rhythm analysis for diverse music audio |
| topic | real-time beat tracking downbeat tracking rhythm analysis singing voices non-percussive music beatnet beatnet+ |
| url | https://account.transactions.ismir.net/index.php/up-j-tismir/article/view/198 |
| work_keys_str_mv | AT mojtabaheydari beatnetrealtimerhythmanalysisfordiversemusicaudio AT zhiyaoduan beatnetrealtimerhythmanalysisfordiversemusicaudio |