Comparing LLM-generated and human-authored news text using formal syntactic theory

Zamaraeva, Olga; Flickinger, Dan; Bond, Francis; Gómez-Rodríguez, Carlos

Use this link to cite:

https://hdl.handle.net/2183/47473

Comparing LLM-generated and human-authored news text using formal syntactic theory

Files

Zamaraeva_Olga_2025_Comparing_LLM_human_text.pdf (599.44 KB)

Identifiers

URI: https://hdl.handle.net/2183/47473

DOI: 10.18653/v1/2025.acl-long.443

Publication date

2025

Authors

Zamaraeva, Olga

Flickinger, Dan

Bond, Francis

Gómez-Rodríguez, Carlos

Bibliographic citation

O. Zamaraeva, D. Flickinger, F. Bond, y C. Gómez-Rodríguez, «Comparing LLM-generated and human-authored news text using formal syntactic theory», en Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria: Association for Computational Linguistics, 2025, pp. 9041-9060. doi: 10.18653/v1/2025.acl-long.443.

Abstract

[Abstract]: This study provides the first comprehensive comparison of New York Times-style text generated by six large language models against real, human-authored NYT writing. The comparison is based on a formal syntactic theory. We use Head-driven Phrase Structure Grammar (HPSG) to analyze the grammatical structure of the texts. We then investigate and illustrate the differences in the distributions of HPSG grammar types, revealing systematic distinctions between human and LLM-generated writing. These findings contribute to a deeper understanding of the syntactic behavior of LLMs as well as humans, within the NYT genre.