Ortega Morla, JavierFerreiro Villamor, DanielPaz-López, AlejandroPérez-Sánchez, BeatrizGuerreiro-Santalla, Sara2026-04-212026-04-212025J. Ortega-Morla, D. Ferreiro, A. Paz-Lopez, B. Pérez-Sánchez and S. Guerreiro-Santalla, "Hybrid LLM and Sentence Transformer Evaluation for ITS Programming Exercises," 2025 International Conference on Education Technology and Computers (ICETC), Barcelona, Spain, 2025, pp. 221-225, doi: 10.1109/ICETC66579.2025.11387358.979-8-3315-9791-7https://hdl.handle.net/2183/48046© 2025 IEEE. This version of the article has been accepted for publication, after peer review. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The Version of Record is available online at: https://doi.org/10.1109/ICETC66579.2025.11387358 Presented at: 2025 International Conference on Education Technology and Computers (ICETC), 18-21 September 2025, Barcelona, Spain[Abstract]: This paper explores automatic evaluation and feedback generation for varied student output in programming exercises. To this end, a hybrid approach was implemented, combining Sentence Transformer Models for output classification with General Purpose Large Language Models for generating brief, informative feedback. The solution was tested on a dataset of 1,300 manually labeled student outputs from programming exams at the University of A Coruña. Finally, this system will be integrated into ProgTutor, an Intelligent Tutoring System, to replace the current evaluation method and enable its assessment in large-scale, real-world conditions.engCopyright © 2025, IEEEIntelligent Tutoring SystemsITSLarge Language ModelsLLMAutomatic Evaluationconference outputopen access10.1109/ICETC66579.2025.11387358