OBELICS
Multimodal Dataset
OBELICS (Open Bimodal Examples from Large-scale Interleaved Captioned Sources) is a large-scale multimodal web document dataset released by Hugging Face. It contains 141 million web documents, interleaved with 353 million images and 115 billion text tokens. The data is extracted from CommonCrawl, preserving the natural arrangement of text and images, used for training multimodal large language models like IDEFICS.
Dataset Highlights
A large-scale dataset of images and text, providing a training foundation for the next generation of multimodal large language models.
Image-Text Interleaved Format
Preserves the natural arrangement of images and text in web documents, faithfully restoring the authentic reading experience of interleaved images and text, rather than simple image-text pairing.
Ultra Large Scale
Contains 141 million web documents, 353 million images, and 115 billion text tokens, making it one of the largest open-source multimodal interleaved datasets available.
Web-Native Structure
Data is extracted directly from real web pages, preserving the original document structure and contextual relationships, rather than artificially constructed image-text pairs, making it closer to natural scenarios.
Drives IDEFICS
As the core training data for Hugging Face's open-source multimodal large language model IDEFICS, it has been validated through large-scale model training in practical applications.
Quality Selected
Processed through multiple layers of filtering pipelines, including sensitive content filtering, document quality assessment, and duplicate document removal, ensuring the data is clean and usable.
Open and Reproducible
Utilizes the CC BY 4.0 license, deeply integrated with the Hugging Face Datasets library, allowing for direct streaming via API or bulk download.
Applicable Scenarios
From multimodal model training to document understanding, covering cutting-edge research and engineering implementation
Multimodal Large Model Training
Train multimodal large language models that can understand both images and text using a mixed format of text and images
Document Understanding
Learn the layout and structure of real web pages to enhance the model's understanding and information extraction capabilities for complex documents
Visual Question Answering
Build visual question answering models that can reason across images and text, handling complex questions that require multimodal understanding
Few-Shot Multimodal Learning
Utilize a mixed format of text and images for contextual learning, allowing the model to quickly grasp new tasks from a small number of examples
Data Preview
The following is a structural example of a single web document from the OBELICS dataset, showcasing a mixed arrangement of text and images
{
"document_url": "https://example.com/article",
"content": [
{ "type": "text", "value": "Exploring the latest advancements of deep learning in natural language processing..." },
{ "type": "image", "url": "https://example.com/img1.jpg", "alt": "Transformer architecture" },
{ "type": "text", "value": "As shown in the above image, the Transformer architecture consists of encoders and decoders..." },
{ "type": "image", "url": "https://example.com/img2.jpg", "alt": "Attention mechanism" },
{ "type": "text", "value": "The attention mechanism allows the model to focus on different parts of the input sequence..." }
]
}
3 Steps to Get Started Quickly
Quickly integrate OBELICS into your multimodal research workflow from browsing to training
Browse Datasets
Browse the OBELICS datasets on the Ace Data Cloud platform to understand details such as document structure, metadata, and licensing agreements.
Stream or Download
Stream read or batch download data through the Hugging Face Datasets library, flexibly choosing a loading method that suits your computing resources.
Integrate into Training Workflow
Load the interleaved text and image documents into your multimodal training pipeline and start training the next generation of large language models that can understand the relationship between text and images.
Start Exploring OBELICS Data
141 million web documents, 353 million images, interleaved multimodal training data. Whether you are researching multimodal large models or exploring document understanding, OBELICS is the ideal data foundation.