Towards a FAIR Dataset for Non-Functional Requirements

Abstract

[Abstract] In the last years, the application of supervised Machine Learning (ML) algorithms in Requirements Engineering (RE) has allowed increasing the performance (e.g. accuracy, precision) and scalability of automatic requirements classification. However, the lack of publicly labeled datasets is still one concern when conducting ML experiments. Few publicly labeled datasets for non-functional requirements classification are available, and even less in the Spanish language. Moreover, most of the available datasets present some limitations, such as imbalanced classes (e.g. PROMISE NFR). This study aims to generate a FAIR dataset of non-functional requirements in the Spanish language for facilitating reuse in ML classification experiments. 109 non-functional requirements were collected from final degree projects from the University of A Coruña. We conducted a pilot quasi-experiment for non-functional requirements labeling in the categories and subcategories of the ISO/IEC 25010 quality model. The labeling process was accomplished by 7 annotators. The inter-annotator agreement using a Fleiss' Kappa test obtained a substantial agreement in the category level (0.78) and a moderate agreement (0.48) when the classification is per subcategory.