Stackmark › Datasets

mmBERT-pretrain-p2-fineweb2-remaining

mmBERT Pre-training Data P2 [!CAUTION] 🚨 DEPRECATED: KNOWN BUG 🚨 This is an old version of the data with a critical bug: token IDs were stored as uint16 and overflow. mmBERT's vocabulary is larger than 65,535, so any token ID above that limit wraps around and is silently corrupted. ❌ Do NOT use this data for training. This card is kept for reference only (data mixture, sources, and documentation). ✅ Fixed data: orionweller/mmBERT-pretraining-data-chunk0 ℹ️… See the full description on the dataset page: https://huggingface.co/datasets/orionweller/mmBERT-pretrain-p2-fineweb2-remaining.

More datasets

  • prompts.chat — a.k.a. Awesome ChatGPT Prompts This is a Dataset Repository mirror of prompts.chat — a social platform for AI prompts. 📢 Notice This Huggi…
  • FineWeb — 🍷 FineWeb 15 trillion tokens of the finest data the 🌐 web has to offer What is it? The 🍷 FineWeb dataset consists of more than 18.5T tok…
  • hh-rlhf — Dataset Card for HH-RLHF Dataset Summary This repository provides access to two different kinds of data: Human preference data about helpfu…
  • Grade School Math 8K — Dataset Card for GSM8K Dataset Summary GSM8K (Grade School Math 8K) is a dataset of 8.5K high quality linguistically diverse grade school m…
  • wikipedia — Dataset Card for Wikimedia Wikipedia Dataset Summary Wikipedia dataset containing cleaned articles of all languages. The dataset is built f…
  • OpenAssistant Conversations — OpenAssistant Conversations Dataset (OASST1) Dataset Summary In an effort to democratize research on large-scale alignment, we release Open…
  • Alpaca — Dataset Card for Alpaca Dataset Summary Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-00…
  • databricks-dolly-15k — Summary databricks-dolly-15k is an open source dataset of instruction-following records generated by thousands of Databricks employees in s…

Privacy · Terms · llms.txt