SAMatch: training-free object detection for HMI screens
Abstract Recognising objects within graphical user interface (GUI) images presents unexpected challenges, particularly in cases of diverse objects and limited labelled data. In this paper, we enumerate the unique characteristics of GUI images from human-machine interface (HMI) screens and investigat...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2025-06-01
|
| Series: | Discover Data |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s44248-025-00038-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Abstract Recognising objects within graphical user interface (GUI) images presents unexpected challenges, particularly in cases of diverse objects and limited labelled data. In this paper, we enumerate the unique characteristics of GUI images from human-machine interface (HMI) screens and investigate several techniques for detecting appropriate objects present in them. We propose SAMatch, a novel training-free matching-based approach utilising a frozen foundation model, SAM for region proposal and a CNN-based model for deep template MATCHing. Through experimental evaluation, this paper compares approaches toward efficient processing of HMI screens using Computer Vision. |
|---|---|
| ISSN: | 2731-6955 |