
# Arabic Text-Image Dataset from Wikipedia

## Overview

This repository contains a dataset created by scraping images and their captions from Wikipedia, focusing on content that represents the Arab world. The goal is to provide a diverse and representative collection of images with corresponding Arabic captions to support research and development in natural language processing (NLP), computer vision, and cross-modal applications.

## Dataset Structure

The dataset contains four columns:

1. **source:** Indicates the origin of the image-text pair.
2. **link:** Provides the link to the Wikipedia image.
3. **caption:** Arabic captions corresponding to each image.
4. **extension:** Specifies the file extension of each image.


## Recommended Data Preprocessing

It is recommended to incorporate various filtering techniques inspired by established practices:

1. **Text-Based Filtering:**
   - Discarded images with captions containing fewer than three tokens.
   - Maintained digits in captions to increase dataset complexity.
   - Preserved Latin words to minimize information loss for foreign entities or scientific terms.

2. **Diacritics Removal:**
   - Employed the Araby library in Python for diacritics removal in captions.

3. **Speech Tagger Utilization:**
   - Utilized the CAMeL-Lab/bert-base-arabic-camelbert-msa-pos-msa tagger to eliminate captions that are mostly proper nouns to generate higher-quality captions.
   - Aimed to prioritize learning more generic concepts over fine details.

5. **Image-Based Filtering:**
   - Removed images with fewer than one hundred pixels to supply our model with detail-rich images.
   - Included only JPEG images in the final dataset due to potential information loss in other formats.

6. **Handling Non-JPEG Images:**
   - Excluded images of other extensions as they usually contain maps, logos or non-rich information content.

7. **Color System Standardization:**
   - Converted images into RGB using Python OpenCV to ensure a standardized color space.

**Note:** The dataset lacks protection for individual personal images due to the absence of Arabic graph tools that substitute specific concepts for more generic ones. This presents an opportunity for future research and improvement in image privacy and protection.

## Usage

Researchers and developers are encouraged to use this preprocessed dataset for tasks such as image captioning, cross-modal learning, and other NLP and computer vision applications. Please adhere to ethical standards and ensure that the usage of this dataset aligns with Wikipedia's terms of service and licensing.



task_categories:image-to-text size_categories:100K<n<1M language:ar region:us
{"language": ["ar"], "size_categories": ["100K<n<1M"], "task_categories": ["image-to-text"], "pretty_name": "WAP"}
TAGS
#task_categories-image-to-text #size_categories-100K<n<1M #language-Arabic #region-us 


# Arabic Text-Image Dataset from Wikipedia

## Overview

This repository contains a dataset created by scraping images and their captions from Wikipedia, focusing on content that represents the Arab world. The goal is to provide a diverse and representative collection of images with corresponding Arabic captions to support research and development in natural language processing (NLP), computer vision, and cross-modal applications.

## Dataset Structure

The dataset contains four columns:

1. source: Indicates the origin of the image-text pair.
2. link: Provides the link to the Wikipedia image.
3. caption: Arabic captions corresponding to each image.
4. extension: Specifies the file extension of each image.


## Recommended Data Preprocessing

It is recommended to incorporate various filtering techniques inspired by established practices:

1. Text-Based Filtering:
   - Discarded images with captions containing fewer than three tokens.
   - Maintained digits in captions to increase dataset complexity.
   - Preserved Latin words to minimize information loss for foreign entities or scientific terms.

2. Diacritics Removal:
   - Employed the Araby library in Python for diacritics removal in captions.

3. Speech Tagger Utilization:
   - Utilized the CAMeL-Lab/bert-base-arabic-camelbert-msa-pos-msa tagger to eliminate captions that are mostly proper nouns to generate higher-quality captions.
   - Aimed to prioritize learning more generic concepts over fine details.

5. Image-Based Filtering:
   - Removed images with fewer than one hundred pixels to supply our model with detail-rich images.
   - Included only JPEG images in the final dataset due to potential information loss in other formats.

6. Handling Non-JPEG Images:
   - Excluded images of other extensions as they usually contain maps, logos or non-rich information content.

7. Color System Standardization:
   - Converted images into RGB using Python OpenCV to ensure a standardized color space.

Note: The dataset lacks protection for individual personal images due to the absence of Arabic graph tools that substitute specific concepts for more generic ones. This presents an opportunity for future research and improvement in image privacy and protection.

## Usage

Researchers and developers are encouraged to use this preprocessed dataset for tasks such as image captioning, cross-modal learning, and other NLP and computer vision applications. Please adhere to ethical standards and ensure that the usage of this dataset aligns with Wikipedia's terms of service and licensing.



# Arabic Text-Image Dataset from Wikipedia ## Overview

This repository contains a dataset created by scraping images and their captions from Wikipedia, focusing on content that represents the Arab world. The goal is to provide a diverse and representative collection of images with corresponding Arabic captions to support research and development in natural language processing (NLP), computer vision, and cross-modal applications. ## Dataset Structure

The dataset contains four columns:

1. source: Indicates the origin of the image-text pair.
2. link: Provides the link to the Wikipedia image.
3. caption: Arabic captions corresponding to each image.
4. extension: Specifies the file extension of each image. ## Recommended Data Preprocessing

It is recommended to incorporate various filtering techniques inspired by established practices:

1. Text-Based Filtering:
   - Discarded images with captions containing fewer than three tokens.
   - Maintained digits in captions to increase dataset complexity.
   - Preserved Latin words to minimize information loss for foreign entities or scientific terms.

2. Diacritics Removal:
   - Employed the Araby library in Python for diacritics removal in captions.

3. Speech Tagger Utilization:
   - Utilized the CAMeL-Lab/bert-base-arabic-camelbert-msa-pos-msa tagger to eliminate captions that are mostly proper nouns to generate higher-quality captions.
   - Aimed to prioritize learning more generic concepts over fine details.

5. Image-Based Filtering:
   - Removed images with fewer than one hundred pixels to supply our model with detail-rich images.
   - Included only JPEG images in the final dataset due to potential information loss in other formats.

6. Handling Non-JPEG Images:
   - Excluded images of other extensions as they usually contain maps, logos or non-rich information content.

7. Color System Standardization:
   - Converted images into RGB using Python OpenCV to ensure a standardized color space.

Note: The dataset lacks protection for individual personal images due to the absence of Arabic graph tools that substitute specific concepts for more generic ones. This presents an opportunity for future research and improvement in image privacy and protection. ## Usage

Researchers and developers are encouraged to use this preprocessed dataset for tasks such as image captioning, cross-modal learning, and other NLP and computer vision applications. Please adhere to ethical standards and ensure that the usage of this dataset aligns with Wikipedia's terms of service and licensing.
TAGS
#task_categories-image-to-text #size_categories-100K<n<1M #language-Arabic #region-us 
 # Arabic Text-Image Dataset from Wikipedia ## Overview

This repository contains a dataset created by scraping images and their captions from Wikipedia, focusing on content that represents the Arab world. The goal is to provide a diverse and representative collection of images with corresponding Arabic captions to support research and development in natural language processing (NLP), computer vision, and cross-modal applications. ## Dataset Structure

The dataset contains four columns:

1. source: Indicates the origin of the image-text pair.
2. link: Provides the link to the Wikipedia image.
3. caption: Arabic captions corresponding to each image.
4. extension: Specifies the file extension of each image. ## Recommended Data Preprocessing

It is recommended to incorporate various filtering techniques inspired by established practices:

1. Text-Based Filtering:
   - Discarded images with captions containing fewer than three tokens.
   - Maintained digits in captions to increase dataset complexity.
   - Preserved Latin words to minimize information loss for foreign entities or scientific terms.

2. Diacritics Removal:
   - Employed the Araby library in Python for diacritics removal in captions.

3. Speech Tagger Utilization:
   - Utilized the CAMeL-Lab/bert-base-arabic-camelbert-msa-pos-msa tagger to eliminate captions that are mostly proper nouns to generate higher-quality captions.
   - Aimed to prioritize learning more generic concepts over fine details.

5. Image-Based Filtering:
   - Removed images with fewer than one hundred pixels to supply our model with detail-rich images.
   - Included only JPEG images in the final dataset due to potential information loss in other formats.

6. Handling Non-JPEG Images:
   - Excluded images of other extensions as they usually contain maps, logos or non-rich information content.

7. Color System Standardization:
   - Converted images into RGB using Python OpenCV to ensure a standardized color space.

Note: The dataset lacks protection for individual personal images due to the absence of Arabic graph tools that substitute specific concepts for more generic ones. This presents an opportunity for future research and improvement in image privacy and protection. ## Usage

Researchers and developers are encouraged to use this preprocessed dataset for tasks such as image captioning, cross-modal learning, and other NLP and computer vision applications. Please adhere to ethical standards and ensure that the usage of this dataset aligns with Wikipedia's terms of service and licensing.