Datasets

Widely used datasets for training and evaluating AI models. 1,580 listed, most popular first.

prompts.chat — a.k.a. Awesome ChatGPT Prompts This is a Dataset Repository mirror of prompts.chat — a social platform for AI prompts. 📢 Notice This Huggi…
FineWeb — 🍷 FineWeb 15 trillion tokens of the finest data the 🌐 web has to offer What is it? The 🍷 FineWeb dataset consists of more than 18.5T tok…
hh-rlhf — Dataset Card for HH-RLHF Dataset Summary This repository provides access to two different kinds of data: Human preference data about helpfu…
Grade School Math 8K — Dataset Card for GSM8K Dataset Summary GSM8K (Grade School Math 8K) is a dataset of 8.5K high quality linguistically diverse grade school m…
wikipedia — Dataset Card for Wikimedia Wikipedia Dataset Summary Wikipedia dataset containing cleaned articles of all languages. The dataset is built f…
OpenAssistant Conversations — OpenAssistant Conversations Dataset (OASST1) Dataset Summary In an effort to democratize research on large-scale alignment, we release Open…
Alpaca — Dataset Card for Alpaca Dataset Summary Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-00…
databricks-dolly-15k — Summary databricks-dolly-15k is an open source dataset of instruction-following records generated by thousands of Databricks employees in s…
FineWeb-Edu — 📚 FineWeb-Edu 1.3 trillion tokens of the finest educational data the 🌐 web has to offer Paper: https://arxiv.org/abs/2406.17557 What is i…
IMDB — Dataset Card for "imdb" Dataset Summary Large Movie Review Dataset. This is a dataset for binary sentiment classification containing substa…
EasyNegative — Negative Embedding This is a Negative Embedding trained with Counterfeit. Please use it in the "\stable-diffusion-webui\embeddings" folder.…
SQuAD — Dataset Card for SQuAD Dataset Summary Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questi…
TinyStories — Dataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary. Described in the followin…
UltraChat 200k — Dataset Card for UltraChat 200k Dataset Description This is a heavily filtered version of the UltraChat dataset and was used to train Zephy…
GLUE (General Language Understanding Evaluation benchmark) — Dataset Card for GLUE Dataset Summary GLUE, the General Language Understanding Evaluation benchmark (https://gluebenchmark.com/) is a colle…
PhysicalAI-Autonomous-Vehicles — PHYSICAL AI AUTONOMOUS VEHICLES The PhysicalAI-Autonomous-Vehicles dataset provides one of the largest, geographically diverse collections…
hle — [!NOTE] IMPORTANT: Please help us protect the integrity of this benchmark by not publicly sharing, re-uploading, or distributing the datase…
ImageNet — Dataset Card for ImageNet Dataset Summary ILSVRC 2012, commonly known as 'ImageNet' is an image dataset organized according to the WordNet…
Falcon RefinedWeb — 📀 Falcon RefinedWeb Falcon RefinedWeb is a massive English web dataset built by TII and released under an ODC-By 1.0 license. See the 📓 p…
📄 FinePDFs — Liberating 3T of the finest tokens from PDFs What is this? As we run out of web pages to process, the natural question has always been: wha…
ShareGPT_Vicuna_unfiltered — Further cleaning done. Please look through the dataset and ensure that I didn't miss anything. Update: Confirmed working method for trainin…
OpenThoughts-114k — [!NOTE] We have released a paper for OpenThoughts! See our paper here. Open-Thoughts-114k Open synthetic reasoning dataset with 114k high-q…
OpenHermes 2.5 — Dataset Card for Dataset Name This is the dataset that made OpenHermes 2.5 and Nous Hermes 2 series of models. Support me on GitHub sponsor…
🥂 FineWeb 2 — 🥂 FineWeb2 A sparkling update with 1000s of languages What is it? This is the second iteration of the popular 🍷 FineWeb dataset, bringing…
Alpaca-Cleaned — Dataset Card for Alpaca-Cleaned Repository: https://github.com/gururise/AlpacaDataCleaned Dataset Description This is a cleaned version of…
Measuring Massive Multitask Language Understanding — Dataset Card for MMLU Dataset Summary Measuring Massive Multitask Language Understanding by Dan Hendrycks, Collin Burns, Steven Basart, And…
WikiText — Dataset Card for "wikitext" Dataset Summary The WikiText language modeling dataset is a collection of over 100 million tokens extracted fro…
OpenR1-Math-220k — OpenR1-Math-220k Dataset description OpenR1-Math-220k is a large-scale dataset for mathematical reasoning. It consists of 220k math problem…
COIG-CQIA — COIG-CQIA：Quality is All you need for Chinese Instruction Fine-tuning Dataset Details Dataset Description 欢迎来到COIG-CQIA，COIG-CQIA全称为Chinese…
MiMo-V2.6-RL-oss — Agentic RL Environments RL training environments for LLM agents. Domain Task Family Verifier Code Software engineering Executable tests Cyb…
cosmopedia — Cosmopedia v0.1 Image generated by DALL-E, the prompt was generated by Mixtral-8x7B-Instruct-v0.1 Note: Cosmopedia v0.2 is available at smo…
xlam-function-calling-60k — APIGen Function-Calling Datasets Paper | Website | Models This repo contains 60,000 data collected by APIGen, an automated data generation…
CulturaX — CulturaX Cleaned, Enormous, and Public: The Multilingual Fuel to Democratize Large Language Models for 167 Languages Dataset Summary We pre…
Wikipedia — Wikipedia dataset containing cleaned articles of all languages. The datasets are built from the Wikipedia dump (https://dumps.wikimedia.org…
C4 — C4 Dataset Summary A colossal, cleaned version of Common Crawl's web crawl corpus. Based on Common Crawl dataset: "https://commoncrawl.org"…
MNBVC — MNBVC: Massive Never-ending BT Vast Chinese corpus
NuminaMath CoT — Dataset Card for NuminaMath CoT Dataset Summary Approximately 860k math problems, where each solution is formatted in a Chain of Thought (C…
arXiv Complete Corpus — arXiv Complete Corpus A snapshot of arXiv's metadata, version history, submission files and rendered documents. It covers 3,148,796 papers…
No Robots — Dataset Card for No Robots 🙅‍♂️🤖 Look Ma, an instruction dataset that wasn't generated by GPTs! Dataset Summary No Robots is a high-quali…
GPQA — Dataset Card for GPQA GPQA is a multiple-choice, Q&A dataset of very hard questions written and validated by experts in biology, physics, a…
the_cauldron — Dataset Card for The Cauldron Dataset description The Cauldron is part of the Idefics2 release. It is a massive collection of 50 vision-lan…
gdpval — Dataset for GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. Paper | Blog | Site 220 real-world knowledge…
The-Stack — StarCoder Training Dataset Dataset description This is the dataset used for training StarCoder and StarCoderBase. It contains 783GB of code…
OpenWebText — Dataset Card for "openwebtext" Dataset Summary An open-source replication of the WebText dataset from OpenAI, that was used to train GPT-2.…
glaive-function-calling-v2 — Dataset by glaiveai.
cad-1000-hours — CAD-1K Open v2 - 1,018.1229 Hours 509 end-to-end, single-display Windows CAD task recordings across seven CAD software families. Each task…
FineVision — Fine Vision FineVision is a massive collection of datasets with 17.3M images, 24.3M samples, 88.9M turns, and 9.5B answer tokens, designed…
MMLU-Pro — MMLU-Pro Dataset MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously ben…
orca-math-word-problems-200k — Dataset Card This dataset contains ~200K grade school math word problems. All the answers in this dataset is generated using Azure GPT4-Tur…
Emilia — Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation This is the official repository 👑 for the…
smollm-corpus — SmolLM-Corpus This dataset is a curated collection of high-quality educational and synthetic data designed for training small language mode…
hermes-function-calling-v1 — Hermes Function-Calling V1 This dataset is the compilation of structured output and function calling data used in the Hermes 2 Pro series o…
MetaMathQA — View the project page: https://meta-math.github.io/ see our paper at https://arxiv.org/abs/2309.12284 Note All MetaMathQA data are augmente…
OpenMathReasoning — OpenMathReasoning OpenMathReasoning is a large-scale math reasoning dataset for training large language models (LLMs). This dataset contain…
The Cross-lingual TRansfer Evaluation of Multilingual Encoders for Speech (XTREME-S) benc… — FLEURS Fleurs is the speech version of the FLoRes machine translation benchmark. We use 2009 n-way parallel sentences from the FLoRes dev a…
objaverse — Objaverse Objaverse is a Massive Dataset with 800K+ Annotated 3D Objects. More documentation is coming soon. In the meantime, please see ou…
Emotion — Dataset Card for "emotion" Dataset Summary Emotion is a dataset of English Twitter messages with six basic emotions: anger, fear, joy, love…
WildChat-1M — Dataset Card for WildChat Dataset Description Paper: https://arxiv.org/abs/2405.01470 Interactive Search Tool: https://wildvisualizer.com (…
HelpSteer2 — HelpSteer2: Open-source dataset for training top-performing reward models HelpSteer2 is an open-source Helpfulness Dataset (CC-BY-4.0) that…
Ultra-FineWeb — Ultra-FineWeb 📜 Technical Report | 📦 UltraData Collection | 🌐 UltraData | 🤗 MiniCPM4 Series | 🤗 MiniCPM5 Series English | 中文 📚 Introd…
SmolTalk — SmolTalk Dataset description This is a synthetic dataset designed for supervised finetuning (SFT) of LLMs. It was used to build SmolLM2-Ins…
common_corpus — Common Corpus Full paper - ICLR 2026 oral Common Corpus is the largest open licensed text dataset, comprising 2.27 trillion tokens (2,267,3…
github-code — The GitHub Code dataest consists of 115M code files from GitHub in 32 programming languages with 60 extensions totalling in 1TB of text dat…
UltraData-SFT-2605 — UltraData-SFT-2605 📦 UltraData Collection | 🌐 UltraData | 🤗 MiniCPM5 Series English | 中文 📚 Introduction UltraData-SFT-2605 is the full…
Ai2Arc — Dataset Card for "ai2_arc" Dataset Summary A new dataset of 7,787 genuine grade-school level, multiple-choice science questions, assembled…
OpenAI HumanEval — Dataset Card for OpenAI HumanEval Dataset Summary The HumanEval dataset released by OpenAI includes 164 programming problems with a functio…
SWE-bench_Verified — Dataset Summary SWE-bench Verified is a subset of 500 samples from the SWE-bench test set, which have been human-validated for quality. SWE…
The Stack v3 — 🥞 The Stack v3 What is it? What is being released How to download and use it Dataset statistics Dataset structure Dataset creation Conside…
finemath — 📐 FineMath What is it? 📐 FineMath consists of 34B tokens (FineMath-3+) and 54B tokens (FineMath-3+ with InfiMM-WebMath-3+) of mathematica…
Aya Dataset — Dataset Summary The Aya Dataset is a multilingual instruction fine-tuning dataset curated by an open-science community via Aya Annotation P…
Amazon-Reviews-2023 — Amazon Review 2023 is an updated version of the Amazon Review 2018 dataset. This dataset mainly includes reviews (ratings, text) and item m…
open-web-math — Keiran Paster*, Marco Dos Santos*, Zhangir Azerbayev, Jimmy Ba GitHub | ArXiv | PDF OpenWebMath is a dataset containing the majority of the…
CNN / Daily Mail — Dataset Card for CNN Dailymail Dataset Dataset Summary The CNN / DailyMail Dataset is an English-language dataset containing just over 300k…
Egocentric-10K — Egocentric-10K is the largest egocentric dataset. It is the first dataset collected exclusively in real factories. Your browser does not su…
Bespoke-Stratos-17k — Bespoke-Stratos-17k We replicated and improved the Berkeley Sky-T1 data pipeline using SFT distillation data from DeepSeek-R1 to create Bes…
python_code_instructions_18k_alpaca — Dataset Card for python_code_instructions_18k_alpaca The dataset contains problem descriptions and code in python language. This dataset is…
UltraData-Math — UltraData-Math 🤗 Dataset | 💻 Source Code | 🇨🇳 中文 README UltraData-Math is a large-scale, high-quality mathematical pre-training dataset…
Hacker News - Complete Archive — Hacker News - Complete Archive Every Hacker News item since 2006, live-updated every 5 minutes What is it? This dataset contains the comple…
PubMedQA — Dataset Card for [Dataset Name] Dataset Summary The task of PubMedQA is to answer research questions with yes/no/maybe (e.g.: Do preoperati…
MATH-500 — Dataset Card for MATH-500 This dataset contains a subset of 500 problems from the MATH benchmark that OpenAI created in their Let's Verify…
CodeSearchNet — Dataset Card for CodeSearchNet corpus Dataset Summary CodeSearchNet corpus is a dataset of 2 milllion (comment, code) pairs from opensource…
Ultra-FineWeb-L3 — Ultra-FineWeb-L3 📜 Ultra-FineWeb Technical Report | 📦 UltraData Collection | 🌐 UltraData | 🤗 MiniCPM5 Series English | 中文 📚 Introducti…
mmmu — MMMU (A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI) 🌐 Homepage | 🏆 Leaderboard | 🤗 Dataset…
HotpotQA — Dataset Card for "hotpot_qa" Dataset Summary HotpotQA is a new dataset with 113k Wikipedia-based question-answer pairs with four key featur…
CT-RATE: Chest CT Volumes with Radiology Reports — The CT-RATE Team organizes the VLM3D Challenge VLM3D 2026 (2nd Edition) → Challenge Finals at MICCAI 2026 VLM3D 2025 (1st Edition) → Challe…
dclm-baseline-1.0 — DCLM-baseline DCLM-baseline is a 4T token / 3B document pretraining dataset that achieves strong performance on language model benchmarks.…
C-Eval — C-Eval is a comprehensive Chinese evaluation suite for foundation models. It consists of 13948 multi-choice questions spanning 52 diverse d…
ReActor — ReActor Assets The Fast and Simple Face Swap Extension ComfyUI-ReActor (ex. comfyui-reactor-node) sd-webui-reactor Models file source licen…
Docmatix — Dataset Card for Docmatix Dataset description Docmatix is part of the Idefics3 release (stay tuned). It is a massive dataset for Document V…
FineTranslations — 💬 FineTranslations The world's knowledge in 1+1T tokens of parallel text What is it? This dataset contains over 1 trillion tokens of paral…
olmOCR-bench — olmOCR-bench olmOCR-bench is a dataset of 1,403 PDF files, plus 7,010 unit test cases that capture properties of the output that a good OCR…
TruthfulQA — Dataset Card for truthful_qa Dataset Summary TruthfulQA is a benchmark to measure whether a language model is truthful in generating answer…
People's Speech — Dataset Card for People's Speech Dataset Summary The People's Speech Dataset is among the world's largest English speech recognition corpus…
OmniAction — RoboOmni: Proactive Robot Manipulation in Omni-modal Context 📖 arXiv Paper (Accepted to ICLR 2026 🎉) | 🌐 Website | 🤗 Model | 🤗 Dataset…
OpenVid-1M — Summary This is the dataset proposed in our paper [ICLR 2025] OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation.…
FineTome-100k — FineTome-100k The FineTome dataset is a subset of arcee-ai/The-Tome (without arcee-ai/qwen2-72b-magpie-en), re-filtered using HuggingFaceFW…
MNIST — Dataset Card for MNIST Dataset Summary The MNIST dataset consists of 70,000 28x28 black-and-white images of handwritten digits extracted fr…
PhysicalAI-Robotics-GR00T-X-Embodiment-Sim — PhysicalAI-Robotics-GR00T-X-Embodiment-Sim Github Repo: Isaac GR00T N1 We provide a set of datasets used for post-training of GR00T N1. Eac…
10Kh-RealOmin-OpenData — Boasting over 13,000 hours of cumulative data and 5 million+ clips, it ranks as the largest open-source embodied intelligence dataset in th…
various — malcolmrey's Various AI Model, Architecture & Research Repository Welcome to the central research and asset repository of malcolmrey. This…