skLEP

skLEP (General Language Understanding Evaluation benchmark for Slovak) is the first comprehensive benchmark specifically designed for evaluating Slovak natural language understanding (NLU) models. The benchmark encompasses nine diverse tasks that span token-level, sentence-pair, and document-level challenges, thereby offering a thorough assessment of model capabilities. To create this benchmark, we curated new, original datasets tailored for Slovak and meticulously translated established English NLU resources with native speaker post-editing to ensure high quality evaluation. skLEP includes nine tasks across three categories: Token-Level Tasks: Part-of-Speech (POS) Tagging using Universal Dependencies Named Entity Recognition using Universal NER (UNER) Named Entity Recognition using WikiGoldSK (WGSK) Sentence-Pair Tasks: Recognizing Textual Entailment (RTE) Natural Language Inference (NLI) Semantic Textual Similarity (STS) Document-Level Tasks: Hate Speech Classification (HS) Sentiment Analysis (SA) Question Answering (QA) based on SK-QuAD