Characterizing Knowledge Graph Tasks in LLM Benchmarks Using Cognitive Complexity Frameworks
Sara Todorovikj, Lars-Peter Meyer, Michael Martin
Published: 2025/9/17
Abstract
Large Language Models (LLMs) are increasingly used for tasks involving Knowledge Graphs (KGs), whose evaluation typically focuses on accuracy and output correctness. We propose a complementary task characterization approach using three complexity frameworks from cognitive psychology. Applying this to the LLM-KG-Bench framework, we highlight value distributions, identify underrepresented demands and motivate richer interpretation and diversity for benchmark evaluation tasks.