Probing the effects of broken symmetries in machine learning
Symmetry is one of the most central concepts in physics, and it is no surprise that it has also been widely adopted as an inductive bias for machine-learning models applied to the physical sciences. This is especially true for models targeting the properties of matter at the atomic scale. Both estab...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IOP Publishing
2024-01-01
|
| Series: | Machine Learning: Science and Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1088/2632-2153/ad86a0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849221427908050944 |
|---|---|
| author | Marcel F Langer Sergey N Pozdnyakov Michele Ceriotti |
| author_facet | Marcel F Langer Sergey N Pozdnyakov Michele Ceriotti |
| author_sort | Marcel F Langer |
| collection | DOAJ |
| description | Symmetry is one of the most central concepts in physics, and it is no surprise that it has also been widely adopted as an inductive bias for machine-learning models applied to the physical sciences. This is especially true for models targeting the properties of matter at the atomic scale. Both established and state-of-the-art approaches, with almost no exceptions, are built to be exactly equivariant to translations, permutations, and rotations of the atoms. Incorporating symmetries—rotations in particular—constrains the model design space and implies more complicated architectures that are often also computationally demanding. There are indications that unconstrained models can easily learn symmetries from data, and that doing so can even be beneficial for the accuracy of the model. We demonstrate that an unconstrained architecture can be trained to achieve a high degree of rotational invariance, testing the impacts of the small symmetry breaking in realistic scenarios involving simulations of gas-phase, liquid, and solid water. We focus specifically on physical observables that are likely to be affected—directly or indirectly—by non-invariant behavior under rotations, finding negligible consequences when the model is used in an interpolative, bulk, regime. Even for extrapolative gas-phase predictions, the model remains very stable, even though symmetry artifacts are noticeable. We also discuss strategies that can be used to systematically reduce the magnitude of symmetry breaking when it occurs, and assess their impact on the convergence of observables. |
| format | Article |
| id | doaj-art-12d73a0f10c94462b0931fbf07d08c23 |
| institution | Kabale University |
| issn | 2632-2153 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IOP Publishing |
| record_format | Article |
| series | Machine Learning: Science and Technology |
| spelling | doaj-art-12d73a0f10c94462b0931fbf07d08c232024-11-13T14:59:35ZengIOP PublishingMachine Learning: Science and Technology2632-21532024-01-015404LT0110.1088/2632-2153/ad86a0Probing the effects of broken symmetries in machine learningMarcel F Langer0https://orcid.org/0000-0002-1270-3016Sergey N Pozdnyakov1https://orcid.org/0000-0001-5980-5813Michele Ceriotti2https://orcid.org/0000-0003-2571-2832Laboratory of Computational Science and Modeling and National Centre for Computational Design and Discovery of Novel Materials MARVEL, Institute of Materials , École Polytechnique Fédérale de Lausanne, 1015 Lausanne, SwitzerlandLaboratory of Computational Science and Modeling and National Centre for Computational Design and Discovery of Novel Materials MARVEL, Institute of Materials , École Polytechnique Fédérale de Lausanne, 1015 Lausanne, SwitzerlandLaboratory of Computational Science and Modeling and National Centre for Computational Design and Discovery of Novel Materials MARVEL, Institute of Materials , École Polytechnique Fédérale de Lausanne, 1015 Lausanne, SwitzerlandSymmetry is one of the most central concepts in physics, and it is no surprise that it has also been widely adopted as an inductive bias for machine-learning models applied to the physical sciences. This is especially true for models targeting the properties of matter at the atomic scale. Both established and state-of-the-art approaches, with almost no exceptions, are built to be exactly equivariant to translations, permutations, and rotations of the atoms. Incorporating symmetries—rotations in particular—constrains the model design space and implies more complicated architectures that are often also computationally demanding. There are indications that unconstrained models can easily learn symmetries from data, and that doing so can even be beneficial for the accuracy of the model. We demonstrate that an unconstrained architecture can be trained to achieve a high degree of rotational invariance, testing the impacts of the small symmetry breaking in realistic scenarios involving simulations of gas-phase, liquid, and solid water. We focus specifically on physical observables that are likely to be affected—directly or indirectly—by non-invariant behavior under rotations, finding negligible consequences when the model is used in an interpolative, bulk, regime. Even for extrapolative gas-phase predictions, the model remains very stable, even though symmetry artifacts are noticeable. We also discuss strategies that can be used to systematically reduce the magnitude of symmetry breaking when it occurs, and assess their impact on the convergence of observables.https://doi.org/10.1088/2632-2153/ad86a0machine learningsymmetry-constrained modelsatomistic modelingmolecular simulations |
| spellingShingle | Marcel F Langer Sergey N Pozdnyakov Michele Ceriotti Probing the effects of broken symmetries in machine learning Machine Learning: Science and Technology machine learning symmetry-constrained models atomistic modeling molecular simulations |
| title | Probing the effects of broken symmetries in machine learning |
| title_full | Probing the effects of broken symmetries in machine learning |
| title_fullStr | Probing the effects of broken symmetries in machine learning |
| title_full_unstemmed | Probing the effects of broken symmetries in machine learning |
| title_short | Probing the effects of broken symmetries in machine learning |
| title_sort | probing the effects of broken symmetries in machine learning |
| topic | machine learning symmetry-constrained models atomistic modeling molecular simulations |
| url | https://doi.org/10.1088/2632-2153/ad86a0 |
| work_keys_str_mv | AT marcelflanger probingtheeffectsofbrokensymmetriesinmachinelearning AT sergeynpozdnyakov probingtheeffectsofbrokensymmetriesinmachinelearning AT micheleceriotti probingtheeffectsofbrokensymmetriesinmachinelearning |