ConflLlama: Domain-specific adaptation of large language models for conflict event classification
We present ConflLlama, demonstrating how efficient fine-tuning of large language models can advance automated classification tasks in political science research. While classification of political events has traditionally relied on manual coding or rigid rule-based systems, modern language models off...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-07-01
|
| Series: | Research & Politics |
| Online Access: | https://doi.org/10.1177/20531680251356282 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839635933374185472 |
|---|---|
| author | Shreyas Meher Patrick T. Brandt |
| author_facet | Shreyas Meher Patrick T. Brandt |
| author_sort | Shreyas Meher |
| collection | DOAJ |
| description | We present ConflLlama, demonstrating how efficient fine-tuning of large language models can advance automated classification tasks in political science research. While classification of political events has traditionally relied on manual coding or rigid rule-based systems, modern language models offer the potential for more nuanced, context-aware analysis. However, deploying these models requires overcoming significant technical and resource barriers. We demonstrate how to adapt open-source language models to specialized political science tasks, using conflict event classification as our proof of concept. Through quantization and efficient fine-tuning techniques, we show state-of-the-art performance while minimizing computational requirements. Our approach achieves a macro-averaged AUC of 0.791 and a weighted F1-score of 0.753, representing a 37.6% improvement over the base model, with accuracy gains of up to 1463% in challenging classifications. We offer a roadmap for political scientists to adapt these methods to their own research domains, democratizing access to advanced NLP capabilities across the discipline. This work bridges the gap between cutting-edge AI developments and practical political science research needs, enabling broader adoption of these powerful analytical tools. |
| format | Article |
| id | doaj-art-d68233cf2b0c41f4a2b6d15c7ad43eec |
| institution | Matheson Library |
| issn | 2053-1680 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Research & Politics |
| spelling | doaj-art-d68233cf2b0c41f4a2b6d15c7ad43eec2025-07-08T15:03:40ZengSAGE PublishingResearch & Politics2053-16802025-07-011210.1177/20531680251356282ConflLlama: Domain-specific adaptation of large language models for conflict event classificationShreyas MeherPatrick T. BrandtWe present ConflLlama, demonstrating how efficient fine-tuning of large language models can advance automated classification tasks in political science research. While classification of political events has traditionally relied on manual coding or rigid rule-based systems, modern language models offer the potential for more nuanced, context-aware analysis. However, deploying these models requires overcoming significant technical and resource barriers. We demonstrate how to adapt open-source language models to specialized political science tasks, using conflict event classification as our proof of concept. Through quantization and efficient fine-tuning techniques, we show state-of-the-art performance while minimizing computational requirements. Our approach achieves a macro-averaged AUC of 0.791 and a weighted F1-score of 0.753, representing a 37.6% improvement over the base model, with accuracy gains of up to 1463% in challenging classifications. We offer a roadmap for political scientists to adapt these methods to their own research domains, democratizing access to advanced NLP capabilities across the discipline. This work bridges the gap between cutting-edge AI developments and practical political science research needs, enabling broader adoption of these powerful analytical tools.https://doi.org/10.1177/20531680251356282 |
| spellingShingle | Shreyas Meher Patrick T. Brandt ConflLlama: Domain-specific adaptation of large language models for conflict event classification Research & Politics |
| title | ConflLlama: Domain-specific adaptation of large language models for conflict event classification |
| title_full | ConflLlama: Domain-specific adaptation of large language models for conflict event classification |
| title_fullStr | ConflLlama: Domain-specific adaptation of large language models for conflict event classification |
| title_full_unstemmed | ConflLlama: Domain-specific adaptation of large language models for conflict event classification |
| title_short | ConflLlama: Domain-specific adaptation of large language models for conflict event classification |
| title_sort | conflllama domain specific adaptation of large language models for conflict event classification |
| url | https://doi.org/10.1177/20531680251356282 |
| work_keys_str_mv | AT shreyasmeher conflllamadomainspecificadaptationoflargelanguagemodelsforconflicteventclassification AT patricktbrandt conflllamadomainspecificadaptationoflargelanguagemodelsforconflicteventclassification |