

======================================

**!important**: Be careful when using `caption_attribute_prediction` (We don't recommend to use)!

======================================

# Dataset Card for LP-MusicCaps-MSD

## Dataset Description

- **Repository:** [LP-MusicCaps repository](https://github.com/seungheondoh/lp-music-caps)
- **Paper:** [ArXiv](https://arxiv.org/abs/2307.16372)

## Dataset Summary

**LP-MusicCaps** is a Large Language Model based Pseudo Music Caption dataset for `text-to-music` and `music-to-text` tasks. We construct the music-to-caption pairs with tag-to-caption generation (using three existing multi-label tag datasets and four task instructions). The data sources are MusicCaps, Magnatagtune, and Million Song Dataset ECALS subset.

- **LP-MusicCaps MSD (This Repo)**: 0.5M Audio with 2.2M Caption. We utilize 1054 unique tags in the [MSD-ECALS](https://github.com/SeungHeonDoh/msd-subsets) to perform tag-to-caption generation through LLM.
- [LP-MusicCaps MTT](https://huggingface.co/datasets/seungheondoh/LP-MusicCaps-MTT): 22k Audio with 88k Caption
- [LP-MusicCaps MC](https://huggingface.co/datasets/seungheondoh/LP-MusicCaps-MC): 6k Audio with 22k Caption.


## Data Instances

Each instance in LP-MusicCaps MSD (This Repo) represents multiple image-text pair information with meta-attributes:

```
{
  'track_id': 'TRIHXPZ128F1466744',
  'title': 'In The Sunshine',
  'artist_name': 'ARRESTED DEVELOPMENT',
  'release': 'Zingalamaduni',
  'year': 1994,
  'tag': ['laid back mellow',
    'hip hop',
    'rnb',
    'amiable good natured',
    'rap',
    'urban',
    'gentle',
    'political rap',
    'soul',
    'calm peaceful',
    'summery',
    'cheerful',
    'alternative rap'
  ],
  'caption_writing': 'An amiable and laid back alternative rap tune, this summery and cheerful song blends elements of soul and R&B with a gentle, mellow rap flow to create a calm and peaceful urban vibe that is both hip hop and political in its message.',
  'caption_summary': 'This summery, alternative rap song is a mellow and gentle blend of hip hop, RnB, and political rap with a cheerful and amiable good natured vibe.',
  'caption_paraphrase': 'This laid back mellow rap song infuses soulful and urban elements while showcasing a gentle and amiable good natured vibe, perfect for a summery day. With hints of cheerful R&B and hip hop, the alternative political rap lyrics bring balance to this peaceful and calming tune.',
  'caption_attribute_prediction': 'This mellow, soulful tune is a perfect blend of rap and RnB, with a gentle beat and smooth flow that will transport you to the laid-back urban vibes of a sunny summertime day. The amiable good-natured lyrics touch on political themes, while the alternative rap style adds a cheerful, upbeat twist to the message. Overall, this is a hip-hop gem thats sure to put you in a peaceful, calm state of mind.',
  'path': '3/0/303545.clip.mp3'
}
```

## Pseudo Caption Example:

Input Tags: 

*"video game theme, no singer, instrumental, analog sounding, small keyboard, beatboxing, playful, cheerful, groovy"*

Output Pseudo Captions

*"instrumental track has a joyful and playful vibe, perfect for a video game theme. With no singer, the analog-sounding music features a small keyboard and beatboxing, creating a groovy and cheerful atmosphere"*

[More Information for pseudo caption generation](https://github.com/seungheondoh/lp-music-caps/blob/main/lpmc/llm_captioning/generate.py)
 

## Data Fields

| Name                         | Type            | Description                                                          |
|------------------------------|-----------------|----------------------------------------------------------------------|
| track_id                     | string          | Unique identifier for the track                                      |
| title                        | string          | Title of the song                                                    |
| artist_name                  | string          | Name of the artist performing the song                               |
| release                      | string          | Release name or album name of the song                               |
| year                         | integer         | Year of the song's release                                           |
| tag                          | list of strings | List of tags associated with the song                                |
| caption_writing              | string          | Pseudo caption generated through a writing instruction               |
| caption_summary              | string          | Pseudo caption generated through a summary instruction               |
| caption_paraphrase           | string          | Pseudo caption generated through a paraphrase instruction            |
| caption_attribute_prediction | string          | Pseudo caption generated through an attribute_prediction instruction |
| path                         | string          | File path or location of the audio clip                              |

## Data Splits

- train: 444865
- valid: 34481
- test: 34631
    
## Considerations for Using the Data

The LP-MusicCaps dataset is recommended to be used for research purposes. Due to the wrong labeling issue, we recommend not using caption_attribute_prediction and pseudo_attribute unless it is specifically for large-scale pretraining. Additionally, the field "is_crawled" indicates the samples used in the reference paper mentioned below.

## Discussion of Biases

It will be described in a paper to be released soon.

## Other Known Limitations

It will be described in a paper to be released soon.
size_categories:100K<n<1M language:en art music text-to-music music-to-text arxiv:2307.16372 region:us
{"language": ["en"], "size_categories": ["100K<n<1M"], "pretty_name": "LP-MusicCaps-MSD", "tags": ["art", "music", "text-to-music", "music-to-text"]}
TAGS
#size_categories-100K<n<1M #language-English #art #music #text-to-music #music-to-text #arxiv-2307.16372 #region-us 

======================================


!important: Be careful when using 'caption\_attribute\_prediction' (We don't recommend to use)!


======================================


Dataset Card for LP-MusicCaps-MSD
=================================


Dataset Description
-------------------


* Repository: LP-MusicCaps repository
* Paper: ArXiv


Dataset Summary
---------------


LP-MusicCaps is a Large Language Model based Pseudo Music Caption dataset for 'text-to-music' and 'music-to-text' tasks. We construct the music-to-caption pairs with tag-to-caption generation (using three existing multi-label tag datasets and four task instructions). The data sources are MusicCaps, Magnatagtune, and Million Song Dataset ECALS subset.


* LP-MusicCaps MSD (This Repo): 0.5M Audio with 2.2M Caption. We utilize 1054 unique tags in the MSD-ECALS to perform tag-to-caption generation through LLM.
* LP-MusicCaps MTT: 22k Audio with 88k Caption
* LP-MusicCaps MC: 6k Audio with 22k Caption.


Data Instances
--------------


Each instance in LP-MusicCaps MSD (This Repo) represents multiple image-text pair information with meta-attributes:


Pseudo Caption Example:
-----------------------


Input Tags:


*"video game theme, no singer, instrumental, analog sounding, small keyboard, beatboxing, playful, cheerful, groovy"*


Output Pseudo Captions


*"instrumental track has a joyful and playful vibe, perfect for a video game theme. With no singer, the analog-sounding music features a small keyboard and beatboxing, creating a groovy and cheerful atmosphere"*


More Information for pseudo caption generation


Data Fields
-----------


Name: track\_id, Type: string, Description: Unique identifier for the track
Name: title, Type: string, Description: Title of the song
Name: artist\_name, Type: string, Description: Name of the artist performing the song
Name: release, Type: string, Description: Release name or album name of the song
Name: year, Type: integer, Description: Year of the song's release
Name: tag, Type: list of strings, Description: List of tags associated with the song
Name: caption\_writing, Type: string, Description: Pseudo caption generated through a writing instruction
Name: caption\_summary, Type: string, Description: Pseudo caption generated through a summary instruction
Name: caption\_paraphrase, Type: string, Description: Pseudo caption generated through a paraphrase instruction
Name: caption\_attribute\_prediction, Type: string, Description: Pseudo caption generated through an attribute\_prediction instruction
Name: path, Type: string, Description: File path or location of the audio clip


Data Splits
-----------


* train: 444865
* valid: 34481
* test: 34631


Considerations for Using the Data
---------------------------------


The LP-MusicCaps dataset is recommended to be used for research purposes. Due to the wrong labeling issue, we recommend not using caption\_attribute\_prediction and pseudo\_attribute unless it is specifically for large-scale pretraining. Additionally, the field "is\_crawled" indicates the samples used in the reference paper mentioned below.


Discussion of Biases
--------------------


It will be described in a paper to be released soon.


Other Known Limitations
-----------------------


It will be described in a paper to be released soon.



TAGS
#size_categories-100K<n<1M #language-English #art #music #text-to-music #music-to-text #arxiv-2307.16372 #region-us 

passage: TAGS
#size_categories-100K<n<1M #language-English #art #music #text-to-music #music-to-text #arxiv-2307.16372 #region-us 
