Evaluation of Confusion Behaviors in SEI Models
Radio Frequency Machine Learning (RFML) has in recent years become a popular method for performing a variety of classification tasks on received signals. Among these tasks is Specific Emitter Identification (SEI), which seeks to associate a received signal with the physical emitter that transmitted...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/13/4006 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839631502712766464 |
|---|---|
| author | Brennan Olds Ethan Maas Alan J. Michaels |
| author_facet | Brennan Olds Ethan Maas Alan J. Michaels |
| author_sort | Brennan Olds |
| collection | DOAJ |
| description | Radio Frequency Machine Learning (RFML) has in recent years become a popular method for performing a variety of classification tasks on received signals. Among these tasks is Specific Emitter Identification (SEI), which seeks to associate a received signal with the physical emitter that transmitted it. Many different model architectures, including individual classifiers and ensemble methods, have proven their capabilities for producing high accuracy classification results when performing SEI. Though the works studying different model architectures report on successes, there is a notable absence regarding the examination of systemic failures and negative traits associated with learned behaviors. This work studies those failure patterns for a 64-radio SEI classification problem by isolating common patterns in incorrect classification results across multiple model architectures and two distinct control variables: Signal-to-Noise Ratio (SNR) and the quantity of training data utilized. This work finds that many of the RFML-based models devolve to selecting from amongst a small subset of classes (≈10% of classes) as SNRs decrease and that observed errors are reasonably consistent across different SEI models and architectures. Moreover, our results validate the expectation that ensemble models are generally less brittle, particularly at a low SNR, yet they appear not to be the highest-performing option at a high SNR. |
| format | Article |
| id | doaj-art-17a5ca8a1f974ba992cf3fd73b655e2a |
| institution | Matheson Library |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-17a5ca8a1f974ba992cf3fd73b655e2a2025-07-11T14:43:08ZengMDPI AGSensors1424-82202025-06-012513400610.3390/s25134006Evaluation of Confusion Behaviors in SEI ModelsBrennan Olds0Ethan Maas1Alan J. Michaels2Virginia Tech National Security Institute, Blacksburg, VA 24060, USAVirginia Tech National Security Institute, Blacksburg, VA 24060, USAVirginia Tech National Security Institute, Blacksburg, VA 24060, USARadio Frequency Machine Learning (RFML) has in recent years become a popular method for performing a variety of classification tasks on received signals. Among these tasks is Specific Emitter Identification (SEI), which seeks to associate a received signal with the physical emitter that transmitted it. Many different model architectures, including individual classifiers and ensemble methods, have proven their capabilities for producing high accuracy classification results when performing SEI. Though the works studying different model architectures report on successes, there is a notable absence regarding the examination of systemic failures and negative traits associated with learned behaviors. This work studies those failure patterns for a 64-radio SEI classification problem by isolating common patterns in incorrect classification results across multiple model architectures and two distinct control variables: Signal-to-Noise Ratio (SNR) and the quantity of training data utilized. This work finds that many of the RFML-based models devolve to selecting from amongst a small subset of classes (≈10% of classes) as SNRs decrease and that observed errors are reasonably consistent across different SEI models and architectures. Moreover, our results validate the expectation that ensemble models are generally less brittle, particularly at a low SNR, yet they appear not to be the highest-performing option at a high SNR.https://www.mdpi.com/1424-8220/25/13/4006Radio Frequency Machine Learning (RFML)Specific Emitter Identification (SEI)RF Fingerprintingconfusion matrices |
| spellingShingle | Brennan Olds Ethan Maas Alan J. Michaels Evaluation of Confusion Behaviors in SEI Models Sensors Radio Frequency Machine Learning (RFML) Specific Emitter Identification (SEI) RF Fingerprinting confusion matrices |
| title | Evaluation of Confusion Behaviors in SEI Models |
| title_full | Evaluation of Confusion Behaviors in SEI Models |
| title_fullStr | Evaluation of Confusion Behaviors in SEI Models |
| title_full_unstemmed | Evaluation of Confusion Behaviors in SEI Models |
| title_short | Evaluation of Confusion Behaviors in SEI Models |
| title_sort | evaluation of confusion behaviors in sei models |
| topic | Radio Frequency Machine Learning (RFML) Specific Emitter Identification (SEI) RF Fingerprinting confusion matrices |
| url | https://www.mdpi.com/1424-8220/25/13/4006 |
| work_keys_str_mv | AT brennanolds evaluationofconfusionbehaviorsinseimodels AT ethanmaas evaluationofconfusionbehaviorsinseimodels AT alanjmichaels evaluationofconfusionbehaviorsinseimodels |