DropKAN: Dropout Kolmogorov–Arnold Networks
We propose DropKAN (Dropout Kolmogorov—Arnold Networks), a regularization method that introduces dropout masks at the edge level within Kolmogorov—Arnold Networks (KANs) layers, randomly masking a subset of activation outputs in the computation graph. Forward pass analysis reve...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11121822/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849233784344412160 |
|---|---|
| author | Mohammed Ghaith Altarabichi |
| author_facet | Mohammed Ghaith Altarabichi |
| author_sort | Mohammed Ghaith Altarabichi |
| collection | DOAJ |
| description | We propose DropKAN (Dropout Kolmogorov—Arnold Networks), a regularization method that introduces dropout masks at the edge level within Kolmogorov—Arnold Networks (KANs) layers, randomly masking a subset of activation outputs in the computation graph. Forward pass analysis reveals that DropKAN, when combined with scaling, accurately preserves the expected output signal magnitude in line with theoretical expectations. In contrast, conventional neuron-level Dropout—with its scaling—distorts signal propagation due to the nonlinear nature of KANs activations. Through extensive experiments on several classical benchmark datasets, DropKAN consistently achieves superior test accuracy compared to both unregularized KANs and KANs employing standard Dropout. Sensitivity analysis across dropout rates reveals DropKAN’s robustness, effectively mitigating underfitting at high dropout levels by preserving partial neuron activity via edge-level masking. Additionally, DropKAN demonstrates enhanced sample efficiency under limited training data conditions, outperforming baselines on multiple large-scale datasets. In computer vision benchmarks (MNIST, Fashion MNIST, EMNIST, CIFAR-10), DropKAN further validates its regularization efficacy by consistently improving generalization over standard KAN and Dropout configurations. These results establish DropKAN as a principled and practical regularization technique for KANs architectures. Our implementation of DropKAN is available at (<uri>https://github.com/ghaith81/dropkan</uri>). |
| format | Article |
| id | doaj-art-6d723ea8d65d4b9184204586490e89a1 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-6d723ea8d65d4b9184204586490e89a12025-08-20T04:03:25ZengIEEEIEEE Access2169-35362025-01-011314160914161810.1109/ACCESS.2025.359755411121822DropKAN: Dropout Kolmogorov–Arnold NetworksMohammed Ghaith Altarabichi0https://orcid.org/0000-0002-6040-2269Center for Applied Intelligent Systems Research, Halmstad University, Halmstad, SwedenWe propose DropKAN (Dropout Kolmogorov—Arnold Networks), a regularization method that introduces dropout masks at the edge level within Kolmogorov—Arnold Networks (KANs) layers, randomly masking a subset of activation outputs in the computation graph. Forward pass analysis reveals that DropKAN, when combined with scaling, accurately preserves the expected output signal magnitude in line with theoretical expectations. In contrast, conventional neuron-level Dropout—with its scaling—distorts signal propagation due to the nonlinear nature of KANs activations. Through extensive experiments on several classical benchmark datasets, DropKAN consistently achieves superior test accuracy compared to both unregularized KANs and KANs employing standard Dropout. Sensitivity analysis across dropout rates reveals DropKAN’s robustness, effectively mitigating underfitting at high dropout levels by preserving partial neuron activity via edge-level masking. Additionally, DropKAN demonstrates enhanced sample efficiency under limited training data conditions, outperforming baselines on multiple large-scale datasets. In computer vision benchmarks (MNIST, Fashion MNIST, EMNIST, CIFAR-10), DropKAN further validates its regularization efficacy by consistently improving generalization over standard KAN and Dropout configurations. These results establish DropKAN as a principled and practical regularization technique for KANs architectures. Our implementation of DropKAN is available at (<uri>https://github.com/ghaith81/dropkan</uri>).https://ieeexplore.ieee.org/document/11121822/Dropout Kolmogorov-Arnold networksKolmogorov-Arnold networksdropoutregularization in neural networks |
| spellingShingle | Mohammed Ghaith Altarabichi DropKAN: Dropout Kolmogorov–Arnold Networks IEEE Access Dropout Kolmogorov-Arnold networks Kolmogorov-Arnold networks dropout regularization in neural networks |
| title | DropKAN: Dropout Kolmogorov–Arnold Networks |
| title_full | DropKAN: Dropout Kolmogorov–Arnold Networks |
| title_fullStr | DropKAN: Dropout Kolmogorov–Arnold Networks |
| title_full_unstemmed | DropKAN: Dropout Kolmogorov–Arnold Networks |
| title_short | DropKAN: Dropout Kolmogorov–Arnold Networks |
| title_sort | dropkan dropout kolmogorov x2013 arnold networks |
| topic | Dropout Kolmogorov-Arnold networks Kolmogorov-Arnold networks dropout regularization in neural networks |
| url | https://ieeexplore.ieee.org/document/11121822/ |
| work_keys_str_mv | AT mohammedghaithaltarabichi dropkandropoutkolmogorovx2013arnoldnetworks |