Do LLMs Encode Frame Semantics? Evidence from Frame Identification

Jayanth Krishna Chundru, Rudrashis Poddar, Jie Cao, Tianyu Jiang

Published: 2025/9/23

Abstract

We investigate whether large language models encode latent knowledge of frame semantics, focusing on frame identification, a core challenge in frame semantic parsing that involves selecting the appropriate semantic frame for a target word in context. Using the FrameNet lexical resource, we evaluate models under prompt-based inference and observe that they can perform frame identification effectively even without explicit supervision. To assess the impact of task-specific training, we fine-tune the model on FrameNet data, which substantially improves in-domain accuracy while generalizing well to out-of-domain benchmarks. Further analysis shows that the models can generate semantically coherent frame definitions, highlighting the model's internalized understanding of frame semantics.

Do LLMs Encode Frame Semantics? Evidence from Frame Identification | SummarXiv | SummarXiv