2024-08-07 16:35:17.AIbase.10.9k
Peking University/Tongfang Research Institute Releases Challenging LooGLE Benchmark Test for Long Text Understanding: Large Models Fail Completely!
Long context understanding is a key challenge in the field of natural language processing, especially when large language models (LLMs) handle text that exceeds their context window size. To address this issue, researchers developed the LooGLE benchmark test, aimed at assessing LLMs' capability to understand extremely long documents (averaging 19.3k words, comprising 776 articles across multiple domains). LooGLE includes 7 tasks that cover both short and long dependencies, evaluating model understanding across texts of varying lengths. The test data is sourced from open-access content after 2022.
