Hybrid LLM and Sentence Transformer Evaluation for ITS Programming Exercises
| UDC.coleccion | Investigación | |
| UDC.conferenceTitle | ICETC 2025 | |
| UDC.departamento | Ciencias da Computación e Tecnoloxías da Información | |
| UDC.endPage | 225 | |
| UDC.grupoInv | Laboratorio de Investigación e Desenvolvemento en Intelixencia Artificial (LIDIA) | |
| UDC.grupoInv | Grupo Integrado de Enxeñaría (GII) | |
| UDC.institutoCentro | CITIC - Centro de Investigación de Tecnoloxías da Información e da Comunicación | |
| UDC.startPage | 221 | |
| dc.contributor.author | Ortega Morla, Javier | |
| dc.contributor.author | Ferreiro Villamor, Daniel | |
| dc.contributor.author | Paz-López, Alejandro | |
| dc.contributor.author | Pérez-Sánchez, Beatriz | |
| dc.contributor.author | Guerreiro-Santalla, Sara | |
| dc.date.accessioned | 2026-04-21T07:30:09Z | |
| dc.date.available | 2026-04-21T07:30:09Z | |
| dc.date.issued | 2025 | |
| dc.description | © 2025 IEEE. This version of the article has been accepted for publication, after peer review. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The Version of Record is available online at: https://doi.org/10.1109/ICETC66579.2025.11387358 Presented at: 2025 International Conference on Education Technology and Computers (ICETC), 18-21 September 2025, Barcelona, Spain | |
| dc.description.abstract | [Abstract]: This paper explores automatic evaluation and feedback generation for varied student output in programming exercises. To this end, a hybrid approach was implemented, combining Sentence Transformer Models for output classification with General Purpose Large Language Models for generating brief, informative feedback. The solution was tested on a dataset of 1,300 manually labeled student outputs from programming exams at the University of A Coruña. Finally, this system will be integrated into ProgTutor, an Intelligent Tutoring System, to replace the current evaluation method and enable its assessment in large-scale, real-world conditions. | |
| dc.description.sponsorship | The TED2021-131172B-I00 grant was funded by MCIN/AEI and the European Union NextGenerationEU/PRTR. Support was also received from the Galician Research Center (CITIC), funded by the Government of Galicia and the European Union (FEDER GALICIA 2014- 2020 program), through grant ED431G 2019/01, as well as from the Galician Government group (ED431C 2022/44 and ED431C 2021/39) with FEDER funds. | |
| dc.description.sponsorship | Xunta de Galicia; ED431G 2019/01 | |
| dc.description.sponsorship | Xunta de Galicia; ED431C 2022/44 | |
| dc.description.sponsorship | Xunta de Galicia; ED431C 2021/39 | |
| dc.identifier.citation | J. Ortega-Morla, D. Ferreiro, A. Paz-Lopez, B. Pérez-Sánchez and S. Guerreiro-Santalla, "Hybrid LLM and Sentence Transformer Evaluation for ITS Programming Exercises," 2025 International Conference on Education Technology and Computers (ICETC), Barcelona, Spain, 2025, pp. 221-225, doi: 10.1109/ICETC66579.2025.11387358. | |
| dc.identifier.doi | 10.1109/ICETC66579.2025.11387358 | |
| dc.identifier.isbn | 979-8-3315-9791-7 | |
| dc.identifier.uri | https://hdl.handle.net/2183/48046 | |
| dc.language.iso | eng | |
| dc.publisher | IEEE | |
| dc.relation.projectID | info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023/TED2021-131172B-I00/ES/TUTOR DE PROGRAMACION INTELIGENTE MEDIANTE ROBOTICA SIMULADA | |
| dc.relation.uri | https://doi.org/10.1109/ICETC66579.2025.11387358 | |
| dc.rights | Copyright © 2025, IEEE | |
| dc.rights.accessRights | open access | |
| dc.subject | Intelligent Tutoring Systems | |
| dc.subject | ITS | |
| dc.subject | Large Language Models | |
| dc.subject | LLM | |
| dc.subject | Automatic Evaluation | |
| dc.title | Hybrid LLM and Sentence Transformer Evaluation for ITS Programming Exercises | |
| dc.type | conference output | |
| dspace.entity.type | Publication | |
| relation.isAuthorOfPublication | 5a2c7afc-804a-4032-b8ed-604ec7355150 | |
| relation.isAuthorOfPublication | 1729347a-a5bc-4ab0-a914-6c7a1dce7eb9 | |
| relation.isAuthorOfPublication | e3b7030b-b748-473b-9914-a447f81bf8d2 | |
| relation.isAuthorOfPublication.latestForDiscovery | 5a2c7afc-804a-4032-b8ed-604ec7355150 |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- PerezSanchez_Beatriz_2025_Hybrid_LLM_and_Sentence_Transformer_Evaluation.pdf
- Size:
- 508.48 KB
- Format:
- Adobe Portable Document Format