Twitter2015-Urdu

The Twitter2015-Urdu Dataset is a multimodal resource designed for Multimodal Named Entity Recognition (MNER) research. It consists of 8,257 annotated text-image pairs, divided into training, validation, and test sets.

Using the Dataset:

1. Download the Dataset Files:

   • The dataset is divided into structured files for training, validation, and testing.
   • Each file contains text, corresponding images, and named entity annotations.

2. Dataset Structure:

   • Each entry includes:
     • Text: An Urdu text snippet (UTF-8 encoded).
     • Image: Associated visual content (JPG/PNG format).
     • Annotations: Named entity labels categorized as:
       • PER (Person)
       • LOC (Location)
       • ORG (Organization)
       • MISC (Miscellaneous)

3. Preprocessing:

   • The text is tokenized using the Urduhack library. If further tokenization or preprocessing is needed, ensure Urdu language compatibility.

4. Annotations:

   • Annotations align with the linguistic and cultural nuances of Urdu. Refer to the annotation guidelines in the documentation for consistency.

5. Multimodal Research:

   • Combine the text and image data for multimodal experiments using your preferred deep learning or NLP frameworks.