MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields
Hinged on the representation power of neural networks, neural radiance fields (NeRF) have recently emerged as one of the promising and widely applicable methods for 3D object and scene representation. However, NeRF faces challenges in practical applications, such as large-scale scenes and edge devic...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11029250/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839611155758186496 |
|---|---|
| author | Jaeyoung Chung Kanggeon Lee Sungyong Baik Kyoung Mu Lee |
| author_facet | Jaeyoung Chung Kanggeon Lee Sungyong Baik Kyoung Mu Lee |
| author_sort | Jaeyoung Chung |
| collection | DOAJ |
| description | Hinged on the representation power of neural networks, neural radiance fields (NeRF) have recently emerged as one of the promising and widely applicable methods for 3D object and scene representation. However, NeRF faces challenges in practical applications, such as large-scale scenes and edge devices with a limited amount of memory, where data needs to be processed sequentially. Under such incremental learning scenarios, neural networks are known to suffer catastrophic forgetting: easily forgetting previously seen data after training with new data. We observe that previous incremental learning algorithms are limited by either low performance or memory scalability issues. As such, we develop a Memory-Efficient Incremental Learning algorithm for NeRF (MEIL-NeRF). MEIL-NeRF takes inspiration from NeRF itself in that a neural network can serve as a memory that returns pixel RGB values, given rays as queries. Upon the motivation, our framework learns which rays to query NeRF to extract previous pixel values. The extracted pixel values are then used to train NeRF in a self-distillation manner to prevent catastrophic forgetting. As a result, MEIL-NeRF demonstrates constant memory consumption and competitive performance. |
| format | Article |
| id | doaj-art-ff35e61bc802496c9a4b9530f185eff6 |
| institution | Matheson Library |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-ff35e61bc802496c9a4b9530f185eff62025-07-28T23:01:00ZengIEEEIEEE Access2169-35362025-01-011313042013042910.1109/ACCESS.2025.357839311029250MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance FieldsJaeyoung Chung0https://orcid.org/0000-0002-5251-9926Kanggeon Lee1https://orcid.org/0009-0001-6530-4764Sungyong Baik2https://orcid.org/0000-0001-5702-4618Kyoung Mu Lee3https://orcid.org/0000-0001-7210-1036Department of Electrical and Computer Engineering, Automation and Systems Research Institute (ASRI), Seoul National University, Seoul, South KoreaDepartment of Electrical and Computer Engineering, Automation and Systems Research Institute (ASRI), Seoul National University, Seoul, South KoreaDepartment of Data Science, Hanyang University, Seoul, South KoreaDepartment of Electrical and Computer Engineering, Automation and Systems Research Institute (ASRI), Seoul National University, Seoul, South KoreaHinged on the representation power of neural networks, neural radiance fields (NeRF) have recently emerged as one of the promising and widely applicable methods for 3D object and scene representation. However, NeRF faces challenges in practical applications, such as large-scale scenes and edge devices with a limited amount of memory, where data needs to be processed sequentially. Under such incremental learning scenarios, neural networks are known to suffer catastrophic forgetting: easily forgetting previously seen data after training with new data. We observe that previous incremental learning algorithms are limited by either low performance or memory scalability issues. As such, we develop a Memory-Efficient Incremental Learning algorithm for NeRF (MEIL-NeRF). MEIL-NeRF takes inspiration from NeRF itself in that a neural network can serve as a memory that returns pixel RGB values, given rays as queries. Upon the motivation, our framework learns which rays to query NeRF to extract previous pixel values. The extracted pixel values are then used to train NeRF in a self-distillation manner to prevent catastrophic forgetting. As a result, MEIL-NeRF demonstrates constant memory consumption and competitive performance.https://ieeexplore.ieee.org/document/11029250/Neural radiance fieldsincremental learning3D representationmapping |
| spellingShingle | Jaeyoung Chung Kanggeon Lee Sungyong Baik Kyoung Mu Lee MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields IEEE Access Neural radiance fields incremental learning 3D representation mapping |
| title | MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields |
| title_full | MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields |
| title_fullStr | MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields |
| title_full_unstemmed | MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields |
| title_short | MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields |
| title_sort | meil nerf memory efficient incremental learning of neural radiance fields |
| topic | Neural radiance fields incremental learning 3D representation mapping |
| url | https://ieeexplore.ieee.org/document/11029250/ |
| work_keys_str_mv | AT jaeyoungchung meilnerfmemoryefficientincrementallearningofneuralradiancefields AT kanggeonlee meilnerfmemoryefficientincrementallearningofneuralradiancefields AT sungyongbaik meilnerfmemoryefficientincrementallearningofneuralradiancefields AT kyoungmulee meilnerfmemoryefficientincrementallearningofneuralradiancefields |