
A dataset for benchmarking keyphrase extraction and generation techniques from news. For more details about the dataset please refer the original paper - [https://aclanthology.org/W19-8617.pdf](https://aclanthology.org/W19-8617.pdf)

Original source of the data - [https://github.com/ygorg/KPTimes](https://github.com/ygorg/KPTimes)

## Dataset Summary


<br>
<p align="center">
  <img src="https://huggingface.co/datasets/midas/kptimes/resolve/main/kptimes-details.png" alt="KPTimes dataset summary"  width="90%"/>
  <br>
</p>
<br>

KPTimes is a large scale dataset comprising of 279,923 news articles from NY Times and 10K from JPTimes. It is one of the datasets which has annotations of keyphrases curated by the editors who can be considered as experts. The motivation of the authors behind producing this dataset was to have a large dataset for training neural models for keyphrase generation in a domain other than the scientific domain, and to understand the differences between keyphrases annotated by experts and non-experts. The authors show that the editors tend to assign generic keyphrases that are not present in the actual news article's text, with 55% of them being abstractive keyphrases. The keyphrases in the news domain as presented in this work were also on an average shorter (1.4 words) than those in the scientific datasets (2.4 words).

The dataset is randomly divided into train (92.8%), validation (3.6%) and test (3.6%) splits. In order to enable the models trained on this dataset to generalize well the authors did not want to have the entire data taken from a single source (NY Times), and therefore added 10K more articles from JPTimes dataset. The authors collected free to read article URLs from NY Times spanning from 2006 to 2017, and obtained their corresponding HTML pages from the Internet Archive. They cleaned the HTML tags and extracted the title, and the main content of the articles using heuristics. The gold keyphrases were obtained from the metadata fields - *news_keywords* and *keywords*. The documents in the dataset are full-length news articles, which also makes it a suitable dataset for developing models for identifying keyphrases from long documents.

<br>
<p align="center">
  <img src="https://huggingface.co/datasets/midas/kptimes/resolve/main/KPTimesExample.png" alt="KPTimes sample"  width="90%"/>
  <br>
</p>
<br>



## Dataset Structure

## Dataset Statistics 
Table 1: Statistics on the length of the abstractive keyphrases for Train, Test, and Validation splits of Inspec dataset.

|                    	|  Train  	|   Test  	| Validation 	|
|:------------------:	|:-------:	|:-------:	|:----------:	|
|     Single word    	|  15.6%  	| 29.59%  	|   15.52%   	|
|      Two words     	|  36.7%  	|  36.88% 	|   12.38%   	|
|     Three words    	|  29.5%  	|  20.86% 	|   29.29%   	|
|     Four words     	|  12.5%  	|  8.88%  	|     0%     	|
|     Five words     	|   3.4%  	|  2.33%  	|    3.50%   	|
|      Six words     	|   1.4%  	|  0.93%  	|    1.38%   	|
|     Seven words    	|   0.4%  	|  0.27%  	|    0.37%   	|
|     Eight words    	|  0.24%  	|  0.13%  	|    0.21%   	|
|     Nine words     	|  0.14%  	|  0.013% 	|    0.10%   	|
|      Ten words     	|  0.02%  	| 0.0007% 	|    0.03%   	|
|    Eleven words    	|  0.01%  	|  0.01%  	|   0.003%   	|
|    Twelve words    	|  0.008% 	|  0.011% 	|   0.007%   	|
|   Thirteen words   	|  0.01%  	|  0.02%  	|    0.02%   	|
|   Fourteen words   	|  0.001% 	|    0%   	|     0%     	|
|    Fifteen words   	|  0.001% 	|  0.004% 	|   0.003%   	|
|    Sixteen words   	| 0.0004% 	|    0%   	|     0%     	|
|   Seventeen words  	| 0.0005% 	|    0%   	|     0%     	|
|   Eighteen words   	| 0.0004% 	|    0%   	|     0%     	|
|   Nineteen words   	| 0.0001% 	|    0%   	|     0%     	|
|    Twenty words    	| 0.0001% 	|    0%   	|     0%     	|
| Twenty-three words 	| 0.0001% 	|    0%   	|     0%     	|


Table 2: Statistics on the length of the extractive keyphrases for Train, Test, and Validation splits of Inspec dataset.

|                	|  Train  	|  Test  	| Validation 	|
|:--------------:	|:-------:	|:------:	|:----------:	|
|   Single word  	|  54.2%  	|  60.0% 	|   54.38%   	|
|    Two words   	|  33.9%  	|  32.4% 	|   33.73%   	|
|   Three words  	|   8.8%  	|  5.5%  	|    8.70%   	|
|   Four words   	|   1.9%  	|  1.04% 	|    1.97%   	|
|   Five words   	|   0.5%  	|  0.25% 	|    0.53%   	|
|    Six words   	|   0.4%  	|  0.16% 	|    0.44%   	|
|   Seven words  	|  0.12%  	|  0.06% 	|    0.15%   	|
|   Eight words  	|  0.05%  	|  0.03% 	|    0.08%   	|
| Nine words     	|  0.009% 	|   0%   	|     0%     	|
|    Ten words   	| 0.0007% 	| 0.001% 	|     0%     	|
| Eleven words   	| 0.0002% 	|   0%   	|     0%     	|
| Twelve words   	| 0.0002% 	|   0%   	|     0%     	|
| Thirteen words 	| 0.0002% 	|   0%   	|     0%     	||

Table 3: General statistics of the Inspec dataset.

|                 Type of Analysis                 	|         Train         	|          Test         	|       Validation      	|
|:------------------------------------------------:	|:---------------------:	|:---------------------:	|:---------------------:	|
|                  Annotator Type                  	| Professional Indexers 	| Professional Indexers 	| Professional Indexers 	|
|                   Document Type                  	|     News Articles     	|     News Articles     	|     News articles     	|
|                 No. of Documents                 	|        259,923        	|         20,000        	|         10,000        	|
|           Avg. Document length (words)           	|         783.32        	|         643.2         	|         784.65        	|
|            Max Document length (words)           	|          7278         	|          5503         	|          5627         	|
|  Max no. of abstractive keyphrases in a document 	|           10          	|           10          	|           10          	|
|  Min no. of abstractive keyphrases in a document 	|           0           	|           0           	|           0           	|
| Avg. no. of abstractive keyphrases  per document 	|          2.87         	|          2.30         	|          2.89         	|
|  Max no. of extractive keyphrases in a document  	|           10          	|           10          	|           9           	|
|  Min no. of extractive keyphrases in a document  	|           0           	|           0           	|           0           	|
|  Avg. no. of extractive keyphrases  per document 	|          2.15         	|          2.72         	|          2.13         	|


### Data Fields

- **id**: unique identifier of the document.
- **document**: Whitespace separated list of words in the document.
- **doc_bio_tags**: BIO tags for each word in the document. B stands for the beginning of a keyphrase and I stands for inside the keyphrase. O stands for outside the keyphrase and represents the word that isn't a part of the keyphrase at all.
- **extractive_keyphrases**: List of all the present keyphrases.
- **abstractive_keyphrase**: List of all the absent keyphrases.
- **other metadata**: Additional information present in the original dataset.
  - **id** : unique identifier for the document
  - **date** : publishing date (YYYY/MM/DD)
  - **categories** : categories of the article (1 or 2 categories)
  - **title** : title of the document
  - **abstract** : content of the article
  - **keyword** : list of keywords

### Data Splits

|Split| #datapoints  |
|--|--|
| Train | 259923 |
| Test | 20000 |
| Validation | 10000 |


## Usage

### Full Dataset

```python
from datasets import load_dataset

# get entire dataset
dataset = load_dataset("midas/kptimes", "raw")

# sample from the train split
print("Sample from training data split")
train_sample = dataset["train"][0]
print("Fields in the sample: ", [key for key in train_sample.keys()])
print("Tokenized Document: ", train_sample["document"])
print("Document BIO Tags: ", train_sample["doc_bio_tags"])
print("Extractive/present Keyphrases: ", train_sample["extractive_keyphrases"])
print("Abstractive/absent Keyphrases: ", train_sample["abstractive_keyphrases"])
print("Other Metadata: ", train_sample["other_metadata"])
print("\n-----------\n")

# sample from the validation split
print("Sample from validation data split")
validation_sample = dataset["validation"][0]
print("Fields in the sample: ", [key for key in validation_sample.keys()])
print("Tokenized Document: ", validation_sample["document"])
print("Document BIO Tags: ", validation_sample["doc_bio_tags"])
print("Extractive/present Keyphrases: ", validation_sample["extractive_keyphrases"])
print("Abstractive/absent Keyphrases: ", validation_sample["abstractive_keyphrases"])
print("Other Metadata: ", validation_sample["other_metadata"])
print("\n-----------\n")

# sample from the test split
print("Sample from test data split")
test_sample = dataset["test"][0]
print("Fields in the sample: ", [key for key in test_sample.keys()])
print("Tokenized Document: ", test_sample["document"])
print("Document BIO Tags: ", test_sample["doc_bio_tags"])
print("Extractive/present Keyphrases: ", test_sample["extractive_keyphrases"])
print("Abstractive/absent Keyphrases: ", test_sample["abstractive_keyphrases"])
print("Other Metadata: ", test_sample["other_metadata"])
print("\n-----------\n")
```
**Output**

```bash
Sample from training data split
Fields in the sample:  ['id', 'document', 'doc_bio_tags', 'extractive_keyphrases', 'abstractive_keyphrases', 'other_metadata']
Tokenized Document:  ['For', 'Donald', 'Trump’s', 'Big', 'Speech,', 'an', 'Added', 'Pressure:', 'No', 'Echoes', 'CLEVELAND', '—', 'Until', 'Monday', 'night,', 'Donald', 'J.', 'Trump’s', 'biggest', 'concern', 'about', 'his', 'convention', 'speech', 'was', 'how', 'much', 'to', 'reveal', 'about', 'himself', 'and', 'his', 'family', 'in', 'an', 'address', 'that', 'is', 'often', 'the', 'most', 'personal', 'one', 'a', 'presidential', 'candidate', 'delivers.', 'But', 'the', 'political', 'firestorm', 'over', 'his', 'wife’s', 'speech', ',', 'which', 'borrowed', 'passages', 'from', 'Michelle', 'Obama’s', 'convention', 'remarks', 'in', '2008,', 'raised', 'the', 'stakes', 'exponentially.', 'Mr.', 'Trump’s', 'speech', 'on', 'Thursday', 'night', 'cannot', 'merely', 'be', 'his', 'best', 'ever.', 'It', 'also', 'has', 'to', 'be', 'bulletproof.', 'By', 'Tuesday', 'morning,', 'word', 'had', 'spread', 'throughout', 'his', 'campaign', 'that', 'any', 'language', 'in', 'Mr.', 'Trump’s', 'address', 'even', 'loosely', 'inspired', 'by', 'speeches,', 'essays,', 'books', 'or', 'Twitter', 'posts', 'had', 'to', 'be', 'either', 'rewritten', 'or', 'attributed.', 'Mr.', 'Trump’s', 'chief', 'speechwriter,', 'Stephen', 'Miller,', 'reassured', 'colleagues', 'that', 'the', 'acceptance', 'speech', 'was', 'wholly', 'original,', 'according', 'to', 'two', 'staff', 'members', 'who', 'spoke', 'with', 'him', 'and', 'described', 'those', 'conversations', 'on', 'the', 'condition', 'of', 'anonymity.', 'Mr.', 'Miller', 'also', 'told', 'campaign', 'aides', 'that', 'he', 'had', 'looked', 'closely', 'at', 'passages', 'that', 'Mr.', 'Trump', 'had', 'contributed', '—', 'handwritten', 'on', 'unlined', 'white', 'pages', '—', 'and', 'was', 'confident', 'they', 'contained', 'no', 'problems.', '(Mr.', 'Miller', 'declined', 'an', 'interview', 'request.)', 'Even', 'so,', 'one', 'of', 'the', 'staff', 'members', 'downloaded', 'plagiarism-detection', 'software', 'and', 'ran', 'a', 'draft', 'of', 'the', 'speech', 'through', 'the', 'program.', 'No', 'red', 'flags', 'came', 'up.', 'The', 'intense', 'scrutiny', 'of', 'Mr.', 'Trump’s', 'words', 'added', 'new', 'pressure', 'to', 'a', 'speechwriting', 'process', 'that', 'has', 'been', 'one', 'of', 'the', 'most', 'unpredictable', 'and', 'free-form', 'in', 'modern', 'presidential', 'campaigns.', 'A', 'month', 'ago,', 'Mr.', 'Trump', 'began', 'giving', 'dictation', 'on', 'themes', 'for', 'the', 'speech,', 'and', 'he', 'tossed', 'ideas', 'and', 'phrases', 'to', 'Mr.', 'Miller', 'or', 'other', 'advisers', 'on', 'a', 'daily', 'basis.', 'On', 'printed', 'copies', 'of', 'each', 'draft,', 'he', 'circled', 'passages', 'he', 'liked,', 'crossed', 'out', 'or', 'put', 'question', 'marks', 'beside', 'lines', 'that', 'he', 'did', 'not', 'favor', 'and', 'frequently', 'suggested', 'new', 'words', 'or', 'phrases.', 'Image', 'Stephen', 'Miller,', 'left,', 'Mr.', 'Trump’s', 'chief', 'speechwriter,', 'and', 'Paul', 'Manafort,', 'the', 'campaign', 'chairman,', 'before', 'an', 'event', 'for', 'the', 'candidate', 'at', 'the', 'Trump', 'SoHo', 'hotel', 'in', 'New', 'York', 'last', 'month.', 'Credit', 'Damon', 'Winter/The', 'New', 'York', 'Times', '“I’ve', 'been', 'amending', 'the', 'drafts', 'big-league,”', 'Mr.', 'Trump', 'said', 'in', 'an', 'interview', 'in', 'his', 'Manhattan', 'office', 'before', 'the', 'convention.', '“I', 'get', 'ideas', 'from', 'a', 'lot', 'of', 'different', 'places,', 'a', 'lot', 'of', 'smart', 'people,', 'but', 'mostly', 'I', 'like', 'language', 'that', 'sounds', 'like', 'me.”', 'Yet', 'in', 'the', 'aftermath', 'of', 'Melania', 'Trump’s', 'speech,', 'campaign', 'advisers', 'have', 'fretted', 'that', 'they', 'do', 'not', 'know', 'for', 'sure', 'where', 'Mr.', 'Trump', 'gets', 'his', 'ideas', 'and', 'language', '—', 'whether', 'they', 'are', 'his', 'own,', 'in', 'other', 'words,', 'or', 'are', 'picked', 'up', 'from', 'Twitter,', 'television,', 'or,', 'say,', 'a', 'best', 'seller', 'by', 'Bill', 'O’Reilly', 'of', 'Fox', 'News,', 'a', 'commentator', 'whom', 'Mr.', 'Trump', 'likes.', 'Borrowing', 'or', 'adapting', 'may', 'not', 'always', 'be', 'tantamount', 'to', 'plagiarism,', 'but', 'several', 'Trump', 'advisers,', 'who', 'also', 'insisted', 'on', 'anonymity,', 'said', 'that', 'after', 'the', 'furor', 'over', 'Ms.', 'Trump’s', 'remarks,', 'the', 'campaign', 'cannot', 'allow', 'a', 'similar', 'blowup.', 'Ed', 'Rollins,', 'a', 'Republican', 'strategist', 'who', 'is', 'advising', 'a', '“super', 'PAC”', 'supporting', 'Mr.', 'Trump,', 'said', 'that', 'the', 'candidate', 'could', 'not', 'afford', 'any', 'mistakes.', '“His', 'speech', 'is', 'the', 'whole', 'game,”', 'Mr.', 'Rollins', 'said.', '“Viewers', 'have', 'to', 'watch', 'it', 'and', 'say,', '‘There', 'is', 'the', 'next', 'president', 'of', 'the', 'United', 'States.’”', 'In', 'the', 'interview,', 'Mr.', 'Trump', 'said', 'his', 'speech', 'would', 'center', 'on', 'his', 'vision', 'of', 'a', 'strong', 'and', 'secure', 'America', 'that', '“once', 'existed', 'and', 'no', 'longer', 'does,', 'but', 'can', 'again', 'under', 'a', 'Trump', 'administration.”', 'Latest', 'Election', 'Polls', '2016', 'Get', 'the', 'latest', 'national', 'and', 'state', 'polls', 'on', 'the', 'presidential', 'election', 'between', 'Hillary', 'Clinton', 'and', 'Donald', 'J.', 'Trump.', 'His', 'greatest', 'challenge,', 'he', 'said,', 'was', '“putting', 'myself', 'in', 'the', 'speech”', '—', 'discussing', 'his', 'upbringing', 'and', 'early', 'experiences', 'and', 'relating', 'them', 'to', 'the', 'hopes', 'and', 'aspirations', 'of', 'other', 'Americans.', '“I', 'was', 'never', 'comfortable', 'getting', 'personal', 'about', 'my', 'family', 'because', 'I', 'thought', 'it', 'was', 'special', 'territory,”', 'Mr.', 'Trump', 'said,', 'glancing', 'at', 'a', 'picture', 'of', 'his', 'father', 'on', 'his', 'desk.', '“It', 'can', 'feel', 'exploitative', 'to', 'use', 'family', 'stories', 'to', 'win', 'votes.', 'And', 'I', 'had', 'a', 'very', 'happy', 'and', 'comfortable', 'life', 'growing', 'up.', 'I', 'had', 'a', 'great', 'relationship', 'with', 'my', 'father.', 'But', 'my', 'focus', 'needs', 'to', 'be', 'on', 'all', 'the', 'Americans', 'who', 'are', 'struggling.”', 'He', 'said', 'he', 'was', 'unsure', 'if', 'he', 'would', 'discuss', 'his', 'older', 'brother', 'Fred,', 'who', 'died', 'as', 'an', 'alcoholic', 'in', '1981', 'at', '43', '—', 'and', 'whom', 'he', 'has', 'described', 'as', 'an', 'example', 'of', 'how', 'destructive', 'choices', 'can', 'damage', 'lives', 'that', 'seem', 'golden.', '“Without', 'my', 'brother', 'Fred', 'I', 'might', 'not', 'be', 'here,”', 'Mr.', 'Trump', 'said.', '“He', 'was', 'really', 'smart,', 'great-looking.', 'I', 'don’t', 'drink', 'or', 'smoke', 'because', 'of', 'what', 'happened', 'to', 'him.', 'I', 'focused', 'on', 'building', 'my', 'business', 'and', 'making', 'good', 'choices.', 'I', 'may', 'talk', 'about', 'that,', 'but', 'I', 'don’t', 'know', 'if', 'I', 'should.”', 'Acceptance', 'speeches', 'seldom', 'seem', 'complete', 'without', 'anecdotes', 'about', 'personal', 'trials', 'and', 'triumphs:', 'Mitt', 'Romney,', 'trying', 'to', 'persuade', 'voters', 'to', 'see', 'him', 'as', 'more', 'than', 'a', 'rich', 'businessman,', 'devoted', 'about', 'a', 'fourth', 'of', 'his', '2012', 'address', 'to', 'his', 'parents’', 'unconditional', 'love,', 'his', 'Mormon', 'faith', 'and', 'reminiscences', 'about', 'watching', 'the', 'moon', 'landing.', 'In', '2008', ',', 'Barack', 'Obama', 'described', 'how', 'his', 'grandfather', 'benefited', 'from', 'the', 'G.I.', 'Bill', 'and', 'how', 'his', 'mother', 'and', 'grandmother', 'taught', 'him', 'the', 'value', 'of', 'hard', 'work.', 'And', 'Bill', 'Clinton’s', '1992', 'speech', 'vividly', 'recalled', 'the', 'life', 'lessons', 'he', 'learned', 'from', 'his', 'mother', 'about', 'fighting', 'and', 'working', 'hard,', 'from', 'his', 'grandfather', 'about', 'racial', 'equality', '—', 'and', 'from', 'his', 'wife,', 'Hillary,', 'who,', 'Mr.', 'Clinton', 'said,', 'taught', 'him', 'that', 'every', 'child', 'could', 'learn.', 'Mr.', 'Clinton', 'finished', 'his', 'speech', 'with', 'a', 'now-famous', 'line', 'tying', 'his', 'Arkansas', 'hometown', 'to', 'the', 'American', 'dream.', '“I', 'end', 'tonight', 'where', 'it', 'all', 'began', 'for', 'me,”', 'he', 'said.', '“I', 'still', 'believe', 'in', 'a', 'place', 'called', 'Hope.”', 'James', 'Carville,', 'a', 'senior', 'strategist', 'for', 'Mr.', 'Clinton’s', '1992', 'campaign,', 'said', 'that', 'if', 'Mr.', 'Trump', 'hoped', 'to', 'change', 'the', 'minds', 'of', 'those', 'who', 'see', 'him', 'as', 'divisive', 'or', 'bigoted,', 'he', 'would', 'need', 'to', 'open', 'himself', 'up', 'to', 'voters', 'in', 'meaningfully', 'personal', 'ways', 'in', 'his', 'speech.', '“If', 'he’s', 'really', 'different', 'than', 'the', 'way', 'he', 'seems', 'in', 'television', 'interviews', 'or', 'at', 'his', 'rallies,', 'Thursday’s', 'speech', 'will', 'be', 'his', 'single', 'greatest', 'opportunity', 'to', 'show', 'voters', 'who', 'he', 'really', 'is,”', 'Mr.', 'Carville', 'said.', 'Paul', 'Manafort,', 'the', 'Trump', 'campaign', 'chairman,', 'said', 'that', 'Thursday’s', 'speech', 'would', 'be', '“very', 'much', 'a', 'reflection', 'of', 'Mr.', 'Trump’s', 'own', 'words,', 'as', 'opposed', 'to', 'remarks', 'that', 'others', 'create', 'and', 'the', 'campaign', 'puts', 'in', 'his', 'mouth.”', '“He’s', 'not', 'an', 'editor', '—', 'he', 'is', 'actually', 'the', 'creator', 'of', 'the', 'speech,”', 'Mr.', 'Manafort', 'said.', '“Mr.', 'Trump', 'has', 'given', 'Steve', 'Miller', 'and', 'I', 'very', 'specific', 'directions', 'about', 'how', 'he', 'views', 'the', 'speech,', 'what', 'he', 'wants', 'to', 'communicate,', 'and', 'ways', 'to', 'tie', 'together', 'things', 'that', 'he', 'has', 'been', 'talking', 'about', 'in', 'the', 'campaign.', 'The', 'speech', 'will', 'end', 'up', 'being', 'tone-perfect', 'because', 'the', 'speech’s', 'words', 'will', 'be', 'his', 'words.”', 'Mr.', 'Trump', 'prefers', 'speaking', 'off', 'the', 'cuff', 'with', 'handwritten', 'notes,', 'a', 'style', 'that', 'has', 'proved', 'successful', 'at', 'his', 'rallies,', 'where', 'he', 'has', 'shown', 'a', 'talent', 'for', 'connecting', 'with', 'and', 'electrifying', 'crowds.', 'But', 'his', 'adjustment', 'to', 'formal', 'speeches', 'remains', 'a', 'work', 'in', 'progress:', 'He', 'does', 'not', 'always', 'sound', 'like', 'himself,', 'and', 'reading', 'from', 'a', 'text', 'can', 'detract', 'from', 'the', 'sense', 'of', 'authenticity', 'that', 'his', 'supporters', 'prize.', 'One', 'question', 'is', 'whether,', 'or', 'how', 'much,', 'he', 'will', 'ad-lib.', 'He', 'has', 'sometimes', 'seemed', 'unable', 'to', 'resist', 'deviating', 'from', 'prepared', 'remarks,', 'often', 'to', 'ill', 'effect', '—', 'ranting', 'about', 'a', 'mosquito', ',', 'or', 'joking', 'that', 'a', 'passing', 'airplane', 'was', 'from', 'Mexico', 'and', 'was', '“', 'getting', 'ready', 'to', 'attack', '.”', '“Ad-libbing', 'is', 'instinct,', 'all', 'instinct,”', 'Mr.', 'Trump', 'said.', '“I', 'thought', 'maybe', 'about', 'doing', 'a', 'freewheeling', 'speech', 'for', 'the', 'convention,', 'but', 'that', 'really', 'wouldn’t', 'work.', 'But', 'even', 'with', 'a', 'teleprompter,', 'the', 'speech', 'will', 'be', 'me', '—', 'my', 'ideas,', 'my', 'beliefs,', 'my', 'words.”']
Document BIO Tags:  ['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O']
Extractive/present Keyphrases:  ['speeches', 'plagiarism']
Abstractive/absent Keyphrases:  ['2016 presidential election', 'donald trump', 'republican national convention,rnc', 'melania trump']
Other Metadata:  {'id': 'ny0282969', 'categories': ['us', 'politics'], 'date': '2016/07/21', 'title': 'For Donald Trump’s Big Speech, an Added Pressure: No Echoes', 'abstract': 'CLEVELAND — Until Monday night, Donald J. Trump’s biggest concern about his convention speech was how much to reveal about himself and his family in an address that is often the most personal one a presidential candidate delivers. But the political firestorm over his wife’s speech , which borrowed passages from Michelle Obama’s convention remarks in 2008, raised the stakes exponentially. Mr. Trump’s speech on Thursday night cannot merely be his best ever. It also has to be bulletproof. By Tuesday morning, word had spread throughout his campaign that any language in Mr. Trump’s address even loosely inspired by speeches, essays, books or Twitter posts had to be either rewritten or attributed. Mr. Trump’s chief speechwriter, Stephen Miller, reassured colleagues that the acceptance speech was wholly original, according to two staff members who spoke with him and described those conversations on the condition of anonymity. Mr. Miller also told campaign aides that he had looked closely at passages that Mr. Trump had contributed — handwritten on unlined white pages — and was confident they contained no problems. (Mr. Miller declined an interview request.) Even so, one of the staff members downloaded plagiarism-detection software and ran a draft of the speech through the program. No red flags came up. The intense scrutiny of Mr. Trump’s words added new pressure to a speechwriting process that has been one of the most unpredictable and free-form in modern presidential campaigns. A month ago, Mr. Trump began giving dictation on themes for the speech, and he tossed ideas and phrases to Mr. Miller or other advisers on a daily basis. On printed copies of each draft, he circled passages he liked, crossed out or put question marks beside lines that he did not favor and frequently suggested new words or phrases. Image Stephen Miller, left, Mr. Trump’s chief speechwriter, and Paul Manafort, the campaign chairman, before an event for the candidate at the Trump SoHo hotel in New York last month. Credit Damon Winter/The New York Times “I’ve been amending the drafts big-league,” Mr. Trump said in an interview in his Manhattan office before the convention. “I get ideas from a lot of different places, a lot of smart people, but mostly I like language that sounds like me.” Yet in the aftermath of Melania Trump’s speech, campaign advisers have fretted that they do not know for sure where Mr. Trump gets his ideas and language — whether they are his own, in other words, or are picked up from Twitter, television, or, say, a best seller by Bill O’Reilly of Fox News, a commentator whom Mr. Trump likes. Borrowing or adapting may not always be tantamount to plagiarism, but several Trump advisers, who also insisted on anonymity, said that after the furor over Ms. Trump’s remarks, the campaign cannot allow a similar blowup. Ed Rollins, a Republican strategist who is advising a “super PAC” supporting Mr. Trump, said that the candidate could not afford any mistakes. “His speech is the whole game,” Mr. Rollins said. “Viewers have to watch it and say, ‘There is the next president of the United States.’” In the interview, Mr. Trump said his speech would center on his vision of a strong and secure America that “once existed and no longer does, but can again under a Trump administration.” Latest Election Polls 2016 Get the latest national and state polls on the presidential election between Hillary Clinton and Donald J. Trump. His greatest challenge, he said, was “putting myself in the speech” — discussing his upbringing and early experiences and relating them to the hopes and aspirations of other Americans. “I was never comfortable getting personal about my family because I thought it was special territory,” Mr. Trump said, glancing at a picture of his father on his desk. “It can feel exploitative to use family stories to win votes. And I had a very happy and comfortable life growing up. I had a great relationship with my father. But my focus needs to be on all the Americans who are struggling.” He said he was unsure if he would discuss his older brother Fred, who died as an alcoholic in 1981 at 43 — and whom he has described as an example of how destructive choices can damage lives that seem golden. “Without my brother Fred I might not be here,” Mr. Trump said. “He was really smart, great-looking. I don’t drink or smoke because of what happened to him. I focused on building my business and making good choices. I may talk about that, but I don’t know if I should.” Acceptance speeches seldom seem complete without anecdotes about personal trials and triumphs: Mitt Romney, trying to persuade voters to see him as more than a rich businessman, devoted about a fourth of his 2012 address to his parents’ unconditional love, his Mormon faith and reminiscences about watching the moon landing. In 2008 , Barack Obama described how his grandfather benefited from the G.I. Bill and how his mother and grandmother taught him the value of hard work. And Bill Clinton’s 1992 speech vividly recalled the life lessons he learned from his mother about fighting and working hard, from his grandfather about racial equality — and from his wife, Hillary, who, Mr. Clinton said, taught him that every child could learn. Mr. Clinton finished his speech with a now-famous line tying his Arkansas hometown to the American dream. “I end tonight where it all began for me,” he said. “I still believe in a place called Hope.” James Carville, a senior strategist for Mr. Clinton’s 1992 campaign, said that if Mr. Trump hoped to change the minds of those who see him as divisive or bigoted, he would need to open himself up to voters in meaningfully personal ways in his speech. “If he’s really different than the way he seems in television interviews or at his rallies, Thursday’s speech will be his single greatest opportunity to show voters who he really is,” Mr. Carville said. Paul Manafort, the Trump campaign chairman, said that Thursday’s speech would be “very much a reflection of Mr. Trump’s own words, as opposed to remarks that others create and the campaign puts in his mouth.” “He’s not an editor — he is actually the creator of the speech,” Mr. Manafort said. “Mr. Trump has given Steve Miller and I very specific directions about how he views the speech, what he wants to communicate, and ways to tie together things that he has been talking about in the campaign. The speech will end up being tone-perfect because the speech’s words will be his words.” Mr. Trump prefers speaking off the cuff with handwritten notes, a style that has proved successful at his rallies, where he has shown a talent for connecting with and electrifying crowds. But his adjustment to formal speeches remains a work in progress: He does not always sound like himself, and reading from a text can detract from the sense of authenticity that his supporters prize. One question is whether, or how much, he will ad-lib. He has sometimes seemed unable to resist deviating from prepared remarks, often to ill effect — ranting about a mosquito , or joking that a passing airplane was from Mexico and was “ getting ready to attack .” “Ad-libbing is instinct, all instinct,” Mr. Trump said. “I thought maybe about doing a freewheeling speech for the convention, but that really wouldn’t work. But even with a teleprompter, the speech will be me — my ideas, my beliefs, my words.”', 'keyword': '2016 Presidential Election;Donald Trump;Republican National Convention,RNC;Speeches;Plagiarism;Melania Trump'}


-----------

Sample from validation data split
Fields in the sample:  ['id', 'document', 'doc_bio_tags', 'extractive_keyphrases', 'abstractive_keyphrases', 'other_metadata']
Tokenized Document:  ['Jack', 'Sock', 'Picks', 'Up', 'Where', 'He', 'Left', 'Off', 'at', 'Last', 'Year’s', 'U.S.', 'Open', 'When', 'we', 'last', 'saw', 'Jack', 'Sock', 'at', 'the', 'United', 'States', 'Open', ',', 'a', 'year', 'ago', 'September,', 'he', 'was', 'holding', 'a', 'trophy', 'over', 'his', 'head', 'and', '—', 'not', 'yet', '19', 'and', 'a', 'newly', 'declared', 'professional', '—', 'being', 'hailed', 'a', 'Grand', 'Slam', 'champion.', 'Granted,', 'as', 'major', 'titles', 'go,', 'mixed', 'doubles', '(with', 'Melanie', 'Oudin)', 'was', 'akin', 'to', 'a', 'serving', 'of', 'cheese', 'and', 'crackers,', 'with', 'the', 'steak,', 'or', 'singles', 'title,', 'still', 'lodged', 'in', 'the', 'freezer.', 'But', 'as', 'Sock', 'had', 'the', 'previous', 'year', 'also', 'won', 'the', 'junior', 'boys', 'title', 'in', 'Flushing', 'Meadows', 'and,', 'with', 'legend', 'holding', 'that', 'he', 'had', 'never', 'lost', 'a', 'high', 'school', 'match,', 'it', 'was', 'natural', '—', 'at', 'least', 'hopeful', '—', 'to', 'think', 'he', 'might', 'have', 'a', 'healthy', 'share', 'of', 'winning', 'genes', 'to', 'go', 'with', 'his', 'booming', 'serve.', 'And', 'his', 'name,', 'for', 'goodness', 'sakes,', 'is', 'Jack', 'Sock;', 'of', 'Lincoln,', 'Neb.,', 'a', 'proud', 'Cornhusker.', 'Does', 'it', 'get', 'any', 'more', 'wholesome', 'and', 'hearty', 'for', 'a', 'country', 'in', 'a', 'continuous', 'search', 'for', 'its', 'next', 'men’s', 'star', 'in', 'this', 'athletically', 'enhanced', 'smash-mouth', 'era?', 'So', 'after', 'Sock', 'introduced', 'himself', 'to', 'Florian', 'Mayer,', 'a', 'German', 'seeded', '22nd,', 'with', 'a', 'sizzling', 'ace', 'down', 'the', 'T', 'and', 'held', 'serve', 'to', 'begin', 'a', 'first-round', 'match', 'Monday', 'on', 'the', 'grandstand', 'court,', 'fans', 'responded', 'with', 'a', 'chant', 'of', '“Let’s', 'Go', 'Sock!”', 'Forgetting', 'for', 'the', 'moment', 'that', 'New', 'York', 'is', 'a', 'Yankees', 'town,', 'it', 'was', 'better', 'than', 'one', 'alternative', '—', 'Sock', 'it', 'to', 'him', '—', 'and', 'completely', 'understandable', 'as', 'Sock', 'was', 'in', 'the', 'process', 'of', 'feeding', 'America’s', 'slam', 'its', 'first', 'helping', 'of', 'nationalistic', 'fervor', 'by', 'overpowering', 'Mayer,', 'who', 'retired', 'while', 'trailing,', '6-3,', '6-2,', '3-2.', 'One', 'or', 'two', 'more', 'performances', 'like', 'this', 'and', 'we', 'can', 'expect', 'a', 'slew', 'of', 'word', 'play', 'headlines,', 'beginning', 'with', 'Sock', 'and', 'Awe.', 'It', 'doesn’t', 'take', 'much', 'to', 'fire', 'up', 'the', 'Next', 'Great', 'American', 'news', 'media', 'machine,', 'not', 'that', 'Sock', 'is', 'lacking', 'in', 'confidence', 'or', 'ambition.', '“I', 'feel', 'like', 'my', 'game', 'is', 'right', 'on', 'the', 'verge', 'of', 'going', 'to', 'the', 'next', 'level,”', 'he', 'said', 'after', 'winning', 'his', 'fourth', 'tour', 'match', 'of', '2012', 'against', 'six', 'losses.', 'To', 'explain', 'what', 'he', 'meant', 'of', 'taking', 'his', 'game', 'to', 'the', '“next', 'level,”', 'put', 'it', 'this', 'way:', 'from', 'his', 'current', 'ranking,', '243,', 'there', 'are', 'many', 'stops', 'to', 'make', 'on', 'the', 'ride', 'to', 'the', 'dizzying', 'heights', 'where', 'Roger', 'Federer', 'and', 'elite', 'company', 'reside', '—', 'beginning', 'with', 'leaping', 'into', 'position', 'near', 'another', 'young', 'and', 'hopeful', 'Yank,', 'Ryan', 'Harrison,', 'currently', 'No.', '61.', 'On', 'the', 'scale', 'of', 'youthful', 'and', 'potential', 'men’s', 'tour', 'heirs,', 'the', '21-year-old', 'Milos', 'Raonic', 'of', 'Canada', 'is', 'the', 'closest', 'to', 'a', 'major', 'breakthrough,', 'though', 'it', 'is', 'also', 'difficult', 'to', 'define', 'what', 'even', 'that', 'means', 'when', 'three', 'players', '—', 'Federer,', 'Rafael', 'Nadal', 'and', 'Novak', 'Djokovic', '—', 'have', 'won', '29', 'of', 'the', 'last', '30', 'slam', 'titles', 'and', 'show', 'little', 'inclination', 'of', 'easing', 'their', 'chokehold.', 'Compared', 'with', 'what', 'the', 'more', 'promising', 'newbies', 'face', 'these', 'days,', 'the', 'emergent', 'superstars', 'of', 'yore', 'practically', 'took', 'their', 'Grand', 'Slam', 'treats', 'by', 'merely', 'growing', 'tall', 'enough', 'to', 'reach', 'into', 'the', 'cookie', 'jar.', 'Boris', 'Becker', 'won', 'Wimbledon', 'as', 'a', '17-year-old', 'mop-haired', 'redhead.', 'John', 'McEnroe', 'and', 'Pete', 'Sampras', 'broke', 'through', 'in', 'New', 'York', 'at', '20', 'and', '19.', 'Into', 'the', '21st', 'century,', 'Nadal', 'began', 'his', 'domination', 'of', 'the', 'French', 'Open', 'at', '19,', 'Djokovic', 'won', 'the', 'Australian', 'Open', 'at', '21', 'and', 'Federer', 'sank', 'to', 'his', 'knees', 'at', 'Wimbledon', 'weeks', 'before', 'turning', '22.', 'These', 'days,', 'it', 'is', 'unfathomable', 'to', 'think', 'of', 'a', 'skinny', 'and', 'moon-balling', 'Michael', 'Chang', 'winning', 'the', 'French', 'Open', 'at', '17,', 'as', 'he', 'did', 'in', '1989,', 'or', 'a', 'teenager', 'winning', 'any', 'of', 'the', 'slams.', '“I', 'don’t', 'think', 'that’s', 'going', 'to', 'be', 'the', 'case', 'any', 'time', 'soon', 'because', 'this', 'game', 'is', 'so', 'physical', 'now', 'and', 'people', 'need', 'to', 'grow', 'into', 'their', 'body,”', 'said', 'John', 'Isner,', 'who', 'at', '27', 'has', 'reason', 'to', 'believe', 'that', 'his', 'best', 'results,', 'whatever', 'they', 'may', 'be,', 'are', 'still', 'ahead', 'of', 'him.', 'At', '31,', 'Federer,', 'who', 'absurdly', 'has', 'not', 'missed', 'a', 'Grand', 'Slam', 'tournament', 'in', '13', 'years,', 'may', 'be', 'the', 'best-conditioned', 'of', 'all.', 'Andy', 'Murray,', 'at', '25,', 'is', 'thought', 'to', 'be', 'on', 'the', 'verge', 'of', 'his', 'prime.', 'It', 'is', 'mind-boggling', 'to', 'think', 'that', 'Bjorn', 'Borg,', 'McEnroe,', 'Becker', 'and', 'others', 'were', 'playing', 'on', 'fumes,', 'their', 'best', 'matches', 'behind', 'them,', 'by', 'their', 'mid-20s.', 'A', 'no-kidding', 'adult’s', 'tour', 'that', 'provides', 'longevity', 'and', 'personal', 'context', 'is', 'so', 'much', 'richer', 'than', 'the', 'alternative.', 'But', 'given', 'such', 'dramatic', 'career', 'clock', 'changes,', 'patience', 'may', 'be', 'a', 'most', 'valuable', 'virtue', 'for', 'players', 'like', 'Raonic,', 'Harrison', 'and', 'Bernard', 'Tomic', 'of', 'Australia.', '“Those', 'guys,', 'it', 'might', 'take', 'them', 'a', 'little', 'while', 'to', 'see', 'their', 'very,', 'very', 'best', 'results,', 'but', 'they’re', 'certainly', 'not', 'doing', 'so', 'bad', 'right', 'now,”', 'said', 'Isner,', 'who', 'didn’t', 'hesitate', 'to', 'include', 'Sock,', 'calling', 'him', '“a', 'very', 'good', 'player.”', 'Sock', 'is', 'a', 'strapping', '’Husker,', '6', 'feet', '1', 'inch,', '180', 'pounds,', 'but', 'he', 'was', 'set', 'back', 'physically', 'in', 'March', 'by', 'surgery', 'to', 'repair', 'a', 'torn', 'abdominal', 'muscle.', 'In', 'a', 'brilliant', 'stroke,', 'he', 'has', 'been', 'working', 'in', 'Las', 'Vegas', 'with', 'the', 'trainer', 'Gil', 'Reyes,', 'who', 'whipped', 'the', 'once-profligate', 'Andre', 'Agassi', 'into', 'shape.', 'He', 'has', 'hired', 'the', 'former', 'Swedish', 'player,', 'Joakim', 'Nystrom,', 'to', 'help', 'him', 'play', 'a', 'more', 'patient', 'game.', 'On', 'today’s', 'altered', 'career', 'time', 'clock,', 'there', 'is', 'no', 'choice', 'but', 'to', 'wait', 'one’s', 'turn', 'and', 'see', 'what', 'happens.', 'In', 'a', 'microcosm', 'of', 'that', 'strategy,', 'Sock', 'fell', 'behind,', '0-40,', 'while', 'serving', 'at', '4-2', 'in', 'the', 'first', 'set,', 'rallied', 'to', 'deuce,', 'kept', 'his', 'cool', 'as', 'Mayer', 'challenged', 'two', 'line', 'calls', 'and', 'won', 'both,', 'and', 'wound', 'up', 'winning', 'the', 'long', 'game', 'with', 'the', 'help', 'of', 'his', 'own', 'challenge', 'of', 'an', 'out', 'call.', 'He', 'was', 'never', 'threatened', 'after', 'that,', 'cranking', 'his', 'first', 'serve', 'as', 'high', 'as', '134', 'miles', 'per', 'hour,', 'winning', '17', 'of', '25', 'second-service', 'points', 'and', 'shrugging', 'off', 'the', 'question', 'of', 'when', 'the', 'Next', 'Great', 'American', 'will', 'arrive', 'as', 'easily', 'as', 'he', 'did', 'Mayer.', '“Until', 'the', 'results', 'are', 'there,', 'until', 'the', 'rankings', 'and', 'everything', 'is', 'there,', 'not', 'a', 'different', 'answer', 'to', 'give,”', 'he', 'said.', 'Give', 'him', 'time,', 'in', 'other', 'words.', 'By', 'today’s', 'standards,', 'he’s', 'got', 'a', 'few', 'years', 'before', 'we', 'have', 'to', 'stop', 'asking.']
Document BIO Tags:  ['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O']
Extractive/present Keyphrases:  []
Abstractive/absent Keyphrases:  ['tennis', 'united states open (tennis)', 'sock jack']
Other Metadata:  {'id': 'ny0125215', 'categories': ['sports', 'tennis'], 'date': '2012/08/28', 'title': 'Jack Sock Picks Up Where He Left Off at Last Year’s U.S. Open', 'abstract': 'When we last saw Jack Sock at the United States Open , a year ago September, he was holding a trophy over his head and — not yet 19 and a newly declared professional — being hailed a Grand Slam champion. Granted, as major titles go, mixed doubles (with Melanie Oudin) was akin to a serving of cheese and crackers, with the steak, or singles title, still lodged in the freezer. But as Sock had the previous year also won the junior boys title in Flushing Meadows and, with legend holding that he had never lost a high school match, it was natural — at least hopeful — to think he might have a healthy share of winning genes to go with his booming serve. And his name, for goodness sakes, is Jack Sock; of Lincoln, Neb., a proud Cornhusker. Does it get any more wholesome and hearty for a country in a continuous search for its next men’s star in this athletically enhanced smash-mouth era? So after Sock introduced himself to Florian Mayer, a German seeded 22nd, with a sizzling ace down the T and held serve to begin a first-round match Monday on the grandstand court, fans responded with a chant of “Let’s Go Sock!” Forgetting for the moment that New York is a Yankees town, it was better than one alternative — Sock it to him — and completely understandable as Sock was in the process of feeding America’s slam its first helping of nationalistic fervor by overpowering Mayer, who retired while trailing, 6-3, 6-2, 3-2. One or two more performances like this and we can expect a slew of word play headlines, beginning with Sock and Awe. It doesn’t take much to fire up the Next Great American news media machine, not that Sock is lacking in confidence or ambition. “I feel like my game is right on the verge of going to the next level,” he said after winning his fourth tour match of 2012 against six losses. To explain what he meant of taking his game to the “next level,” put it this way: from his current ranking, 243, there are many stops to make on the ride to the dizzying heights where Roger Federer and elite company reside — beginning with leaping into position near another young and hopeful Yank, Ryan Harrison, currently No. 61. On the scale of youthful and potential men’s tour heirs, the 21-year-old Milos Raonic of Canada is the closest to a major breakthrough, though it is also difficult to define what even that means when three players — Federer, Rafael Nadal and Novak Djokovic — have won 29 of the last 30 slam titles and show little inclination of easing their chokehold. Compared with what the more promising newbies face these days, the emergent superstars of yore practically took their Grand Slam treats by merely growing tall enough to reach into the cookie jar. Boris Becker won Wimbledon as a 17-year-old mop-haired redhead. John McEnroe and Pete Sampras broke through in New York at 20 and 19. Into the 21st century, Nadal began his domination of the French Open at 19, Djokovic won the Australian Open at 21 and Federer sank to his knees at Wimbledon weeks before turning 22. These days, it is unfathomable to think of a skinny and moon-balling Michael Chang winning the French Open at 17, as he did in 1989, or a teenager winning any of the slams. “I don’t think that’s going to be the case any time soon because this game is so physical now and people need to grow into their body,” said John Isner, who at 27 has reason to believe that his best results, whatever they may be, are still ahead of him. At 31, Federer, who absurdly has not missed a Grand Slam tournament in 13 years, may be the best-conditioned of all. Andy Murray, at 25, is thought to be on the verge of his prime. It is mind-boggling to think that Bjorn Borg, McEnroe, Becker and others were playing on fumes, their best matches behind them, by their mid-20s. A no-kidding adult’s tour that provides longevity and personal context is so much richer than the alternative. But given such dramatic career clock changes, patience may be a most valuable virtue for players like Raonic, Harrison and Bernard Tomic of Australia. “Those guys, it might take them a little while to see their very, very best results, but they’re certainly not doing so bad right now,” said Isner, who didn’t hesitate to include Sock, calling him “a very good player.” Sock is a strapping ’Husker, 6 feet 1 inch, 180 pounds, but he was set back physically in March by surgery to repair a torn abdominal muscle. In a brilliant stroke, he has been working in Las Vegas with the trainer Gil Reyes, who whipped the once-profligate Andre Agassi into shape. He has hired the former Swedish player, Joakim Nystrom, to help him play a more patient game. On today’s altered career time clock, there is no choice but to wait one’s turn and see what happens. In a microcosm of that strategy, Sock fell behind, 0-40, while serving at 4-2 in the first set, rallied to deuce, kept his cool as Mayer challenged two line calls and won both, and wound up winning the long game with the help of his own challenge of an out call. He was never threatened after that, cranking his first serve as high as 134 miles per hour, winning 17 of 25 second-service points and shrugging off the question of when the Next Great American will arrive as easily as he did Mayer. “Until the results are there, until the rankings and everything is there, not a different answer to give,” he said. Give him time, in other words. By today’s standards, he’s got a few years before we have to stop asking.', 'keyword': 'Tennis;United States Open (Tennis);Sock Jack'}


-----------

Sample from test data split
Fields in the sample:  ['id', 'document', 'doc_bio_tags', 'extractive_keyphrases', 'abstractive_keyphrases', 'other_metadata']
Tokenized Document:  ['World', 'records', 'no', 'joke', 'to', 'frustrated', 'Pakistanis', 'ISLAMABAD', '-', 'One', 'young', 'contender', 'created', 'the', 'world’s', 'largest', 'sequin', 'mosaic', 'using', '325,000', 'of', 'the', 'sparkly', 'discs.', 'Two', 'other', 'youths', 'achieved', '123', 'consecutive', 'badminton', 'passes', 'in', 'one', 'minute.', 'And', '1,450', 'participants', 'broke', 'the', 'record', 'for', 'the', 'most', 'people', 'arm', 'wrestling.', 'Such', 'are', 'the', 'skills', 'that', 'Guinness', 'World', 'Records', 'are', 'made', 'of', 'in', 'Pakistan,', 'where', 'thousands', 'of', 'young', 'people', 'are', 'groomed', 'to', 'establish', 'their', 'unique', 'feats', 'for', 'posterity.', 'Last', 'week,', 'the', 'contestants', 'came', 'together', 'for', 'the', 'annual', 'Punjab', 'Youth', 'Festival', 'to', 'show', 'their', 'stuff', '—', 'many', 'in', 'athletics,', 'but', 'others', 'in', 'downright', 'quirky', 'displays,', 'including', 'one', 'young', 'boy', 'who', 'achieved', 'fame', 'by', 'kicking', '50', 'coconuts', 'from', 'on', 'top', 'of', 'the', 'heads', 'of', 'a', 'row', 'of', 'people.', 'It', 'seems', 'Pakistan', 'has', 'become', 'a', 'world', 'record-creating', 'machine,', 'with', 'the', 'coordinated', 'effort', 'reaping', 'an', 'impressive', '23', 'world', 'records,', 'event', 'organizers', 'boasted.', 'The', 'push', 'for', 'inclusion', 'of', 'Pakistanis', 'in', 'the', 'venerable', 'Guinness', 'World', 'Records', 'entries', '(which', 'began', 'in', 'book', 'form', 'in', '1955)', 'stems', 'in', 'part', 'from', 'festival', 'organizers’', 'desire', 'to', 'boost', 'the', 'image', 'of', 'a', 'country', 'often', 'associated', 'with', 'militancy,', 'religious', 'strife', 'and', 'economic', 'decline.', 'There', 'is', 'a', 'patriotic', 'element,', 'as', 'well:', 'Last', 'October,', 'for', 'instance,', '42,813', 'Pakistanis', 'got', 'together', 'in', 'a', 'Lahore', 'hockey', 'stadium', 'to', 'belt', 'out', 'the', 'national', 'anthem', 'and', 'create', 'yet', 'another', 'world', 'record', 'for', 'the', 'most', 'people', 'singing', 'their', 'country’s', 'anthem.', 'Days', 'later,', 'another', '24,200', 'people', 'held', 'green', 'and', 'white', 'boxes', '—', 'the', 'colors', 'of', 'the', 'national', 'flag', 'of', 'Pakistan', '—', 'to', 'set', 'the', 'world', 'record', 'for', 'creating', 'the', 'largest', 'human', 'flag.', 'Although', 'some', 'of', 'the', 'records', 'might', 'seem', 'amusing', 'to', 'others', '—', 'coconut', 'kicking', 'champ', 'Mohammad', 'Rashid', 'of', 'Karachi', 'last', 'week', 'claimed', 'his', 'fourth', 'world', 'record', 'by', 'breaking', '34', 'pine', 'boards', 'in', '32', 'seconds', 'with', 'his', 'head', '—', 'the', 'competitions', 'were', 'no', 'laughing', 'matter', 'to', 'participants.', 'Usman', 'Anwar,', 'director', 'of', 'the', 'Punjab', 'Youth', 'Festival,', 'explained', 'that', 'the', 'kids', 'have', 'been', 'training', 'for', 'eight', 'months.', '“We', 'started', 'at', 'the', 'neighborhood', 'and', 'village', 'level', 'so', 'that', 'children', 'could', 'come', 'out', 'and', 'participate,”', 'said', 'Anwar.', '“Our', 'main', 'objective', 'was', 'to', 'inculcate', 'interest', 'for', 'sports', 'in', 'the', 'public.”', 'Young', 'people', 'from', 'over', '55,000', 'neighborhood', 'and', 'village', 'councils', 'vied', 'for', 'a', 'chance', 'to', 'compete', 'in', 'the', 'games.', '“We', 'were', 'able', 'to', 'select', 'the', 'best', 'of', 'the', 'best', 'to', 'train', 'for', 'the', 'world', 'records,”', 'said', 'Anwar.', 'Because', 'of', 'terrorism,', 'political', 'upheaval', 'and', 'widespread', 'unemployment,', 'many', 'young', 'people', 'appear', 'to', 'have', 'little', 'hope', 'for', 'the', 'future,', 'says', 'Hafeez', 'Rehman,', 'a', 'professor', 'in', 'the', 'anthropology', 'department', 'at', 'Quaid-i-Azam', 'University', 'in', 'the', 'capital,', 'Islamabad.', 'Sports', 'competitions,', 'Rehman', 'said,', 'create', 'an', 'opportunity', 'for', 'youth', 'to', 'excel', 'personally', 'and', 'also', 'to', 'improve', 'Pakistan’s', 'image.', '“We', 'have', 'energetic', 'youth.', 'Pakistan', 'has', 'more', 'than', '55', 'million', 'young', 'people.', 'It', 'becomes', 'an', 'asset', 'for', 'the', 'country,”', 'he', 'added.', 'The', 'festival', 'itself', 'has', 'become', 'part', 'of', 'the', 'record-setting', 'mania.', 'It', 'was', 'recognized', 'for', 'having', 'more', 'participants', '—', '3.3', 'million,', 'most', 'of', 'whom', 'registered', 'online,', 'according', 'to', 'Anwar', '—', 'constituting', 'a', 'world', 'record', 'for', 'sporting', 'events.']
Document BIO Tags:  ['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O']
Extractive/present Keyphrases:  ['pakistan', 'guinness']
Abstractive/absent Keyphrases:  ['india']
Other Metadata:  {'id': 'jp0000001', 'categories': ['asia-pacific', 'offbeat-asia-pacific'], 'date': '2013/03/17', 'title': 'World records no joke to frustrated Pakistanis ', 'abstract': 'ISLAMABAD - One young contender created the world’s largest sequin mosaic using 325,000 of the sparkly discs. Two other youths achieved 123 consecutive badminton passes in one minute. And 1,450 participants broke the record for the most people arm wrestling. Such are the skills that Guinness World Records are made of in Pakistan, where thousands of young people are groomed to establish their unique feats for posterity. Last week, the contestants came together for the annual Punjab Youth Festival to show their stuff — many in athletics, but others in downright quirky displays, including one young boy who achieved fame by kicking 50 coconuts from on top of the heads of a row of people. It seems Pakistan has become a world record-creating machine, with the coordinated effort reaping an impressive 23 world records, event organizers boasted. The push for inclusion of Pakistanis in the venerable Guinness World Records entries (which began in book form in 1955) stems in part from festival organizers’ desire to boost the image of a country often associated with militancy, religious strife and economic decline. There is a patriotic element, as well: Last October, for instance, 42,813 Pakistanis got together in a Lahore hockey stadium to belt out the national anthem and create yet another world record for the most people singing their country’s anthem. Days later, another 24,200 people held green and white boxes — the colors of the national flag of Pakistan — to set the world record for creating the largest human flag. Although some of the records might seem amusing to others — coconut kicking champ Mohammad Rashid of Karachi last week claimed his fourth world record by breaking 34 pine boards in 32 seconds with his head — the competitions were no laughing matter to participants. Usman Anwar, director of the Punjab Youth Festival, explained that the kids have been training for eight months. “We started at the neighborhood and village level so that children could come out and participate,” said Anwar. “Our main objective was to inculcate interest for sports in the public.” Young people from over 55,000 neighborhood and village councils vied for a chance to compete in the games. “We were able to select the best of the best to train for the world records,” said Anwar. Because of terrorism, political upheaval and widespread unemployment, many young people appear to have little hope for the future, says Hafeez Rehman, a professor in the anthropology department at Quaid-i-Azam University in the capital, Islamabad. Sports competitions, Rehman said, create an opportunity for youth to excel personally and also to improve Pakistan’s image. “We have energetic youth. Pakistan has more than 55 million young people. It becomes an asset for the country,” he added. The festival itself has become part of the record-setting mania. It was recognized for having more participants — 3.3 million, most of whom registered online, according to Anwar — constituting a world record for sporting events.', 'keyword': 'india;pakistan;guinness'}


-----------
```

### Keyphrase Extraction
```python
from datasets import load_dataset

# get the dataset only for keyphrase extraction
dataset = load_dataset("midas/kptimes", "extraction")

print("Samples for Keyphrase Extraction")

# sample from the train split
print("Sample from training data split")
train_sample = dataset["train"][0]
print("Fields in the sample: ", [key for key in train_sample.keys()])
print("Tokenized Document: ", train_sample["document"])
print("Document BIO Tags: ", train_sample["doc_bio_tags"])
print("\n-----------\n")

# sample from the validation split
print("Sample from validation data split")
validation_sample = dataset["validation"][0]
print("Fields in the sample: ", [key for key in validation_sample.keys()])
print("Tokenized Document: ", validation_sample["document"])
print("Document BIO Tags: ", validation_sample["doc_bio_tags"])
print("\n-----------\n")

# sample from the test split
print("Sample from test data split")
test_sample = dataset["test"][0]
print("Fields in the sample: ", [key for key in test_sample.keys()])
print("Tokenized Document: ", test_sample["document"])
print("Document BIO Tags: ", test_sample["doc_bio_tags"])
print("\n-----------\n")
```

### Keyphrase Generation
```python
# get the dataset only for keyphrase generation
dataset = load_dataset("midas/kptimes", "generation")

print("Samples for Keyphrase Generation")

# sample from the train split
print("Sample from training data split")
train_sample = dataset["train"][0]
print("Fields in the sample: ", [key for key in train_sample.keys()])
print("Tokenized Document: ", train_sample["document"])
print("Extractive/present Keyphrases: ", train_sample["extractive_keyphrases"])
print("Abstractive/absent Keyphrases: ", train_sample["abstractive_keyphrases"])
print("\n-----------\n")

# sample from the validation split
print("Sample from validation data split")
validation_sample = dataset["validation"][0]
print("Fields in the sample: ", [key for key in validation_sample.keys()])
print("Tokenized Document: ", validation_sample["document"])
print("Extractive/present Keyphrases: ", validation_sample["extractive_keyphrases"])
print("Abstractive/absent Keyphrases: ", validation_sample["abstractive_keyphrases"])
print("\n-----------\n")

# sample from the test split
print("Sample from test data split")
test_sample = dataset["test"][0]
print("Fields in the sample: ", [key for key in test_sample.keys()])
print("Tokenized Document: ", test_sample["document"])
print("Extractive/present Keyphrases: ", test_sample["extractive_keyphrases"])
print("Abstractive/absent Keyphrases: ", test_sample["abstractive_keyphrases"])
print("\n-----------\n")
```
## Citation Information
```
@inproceedings{gallina2019kptimes,
  title={KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents},
  author={Gallina, Ygor and Boudin, Florian and Daille, B{\'e}atrice},
  booktitle={Proceedings of the 12th International Conference on Natural Language Generation},
  pages={130--135},
  year={2019}
}
```

## Contributions
Thanks to [@debanjanbhucs](https://github.com/debanjanbhucs), [@dibyaaaaax](https://github.com/dibyaaaaax), [@UmaGunturi](https://github.com/UmaGunturi) and [@ad6398](https://github.com/ad6398) for adding this dataset

region:us
{}
TAGS
#region-us 

A dataset for benchmarking keyphrase extraction and generation techniques from news. For more details about the dataset please refer the original paper - URL


Original source of the data - URL


Dataset Summary
---------------


  


![](URL alt=)
  




  

KPTimes is a large scale dataset comprising of 279,923 news articles from NY Times and 10K from JPTimes. It is one of the datasets which has annotations of keyphrases curated by the editors who can be considered as experts. The motivation of the authors behind producing this dataset was to have a large dataset for training neural models for keyphrase generation in a domain other than the scientific domain, and to understand the differences between keyphrases annotated by experts and non-experts. The authors show that the editors tend to assign generic keyphrases that are not present in the actual news article's text, with 55% of them being abstractive keyphrases. The keyphrases in the news domain as presented in this work were also on an average shorter (1.4 words) than those in the scientific datasets (2.4 words).


The dataset is randomly divided into train (92.8%), validation (3.6%) and test (3.6%) splits. In order to enable the models trained on this dataset to generalize well the authors did not want to have the entire data taken from a single source (NY Times), and therefore added 10K more articles from JPTimes dataset. The authors collected free to read article URLs from NY Times spanning from 2006 to 2017, and obtained their corresponding HTML pages from the Internet Archive. They cleaned the HTML tags and extracted the title, and the main content of the articles using heuristics. The gold keyphrases were obtained from the metadata fields - *news\_keywords* and *keywords*. The documents in the dataset are full-length news articles, which also makes it a suitable dataset for developing models for identifying keyphrases from long documents.


  


![](URL alt=)
  




  

Dataset Structure
-----------------


Dataset Statistics
------------------


Table 1: Statistics on the length of the abstractive keyphrases for Train, Test, and Validation splits of Inspec dataset.



Table 2: Statistics on the length of the extractive keyphrases for Train, Test, and Validation splits of Inspec dataset.



Table 3: General statistics of the Inspec dataset.



### Data Fields


* id: unique identifier of the document.
* document: Whitespace separated list of words in the document.
* doc\_bio\_tags: BIO tags for each word in the document. B stands for the beginning of a keyphrase and I stands for inside the keyphrase. O stands for outside the keyphrase and represents the word that isn't a part of the keyphrase at all.
* extractive\_keyphrases: List of all the present keyphrases.
* abstractive\_keyphrase: List of all the absent keyphrases.
* other metadata: Additional information present in the original dataset.
	+ id : unique identifier for the document
	+ date : publishing date (YYYY/MM/DD)
	+ categories : categories of the article (1 or 2 categories)
	+ title : title of the document
	+ abstract : content of the article
	+ keyword : list of keywords


### Data Splits



Usage
-----


### Full Dataset


Output


### Keyphrase Extraction


### Keyphrase Generation


Contributions
-------------


Thanks to @debanjanbhucs, @dibyaaaaax, @UmaGunturi and @ad6398 for adding this dataset



### Data Fields


* id: unique identifier of the document.
* document: Whitespace separated list of words in the document.
* doc\_bio\_tags: BIO tags for each word in the document. B stands for the beginning of a keyphrase and I stands for inside the keyphrase. O stands for outside the keyphrase and represents the word that isn't a part of the keyphrase at all.
* extractive\_keyphrases: List of all the present keyphrases.
* abstractive\_keyphrase: List of all the absent keyphrases.
* other metadata: Additional information present in the original dataset.
	+ id : unique identifier for the document
	+ date : publishing date (YYYY/MM/DD)
	+ categories : categories of the article (1 or 2 categories)
	+ title : title of the document
	+ abstract : content of the article
	+ keyword : list of keywords ### Data Splits



Usage
----- ### Full Dataset


Output ### Keyphrase Extraction ### Keyphrase Generation


Contributions
-------------


Thanks to @debanjanbhucs, @dibyaaaaax, @UmaGunturi and @ad6398 for adding this dataset
TAGS
#region-us 
 ### Data Fields


* id: unique identifier of the document.
* document: Whitespace separated list of words in the document.
* doc\_bio\_tags: BIO tags for each word in the document. B stands for the beginning of a keyphrase and I stands for inside the keyphrase. O stands for outside the keyphrase and represents the word that isn't a part of the keyphrase at all.
* extractive\_keyphrases: List of all the present keyphrases.
* abstractive\_keyphrase: List of all the absent keyphrases.
* other metadata: Additional information present in the original dataset.
	+ id : unique identifier for the document
	+ date : publishing date (YYYY/MM/DD)
	+ categories : categories of the article (1 or 2 categories)
	+ title : title of the document
	+ abstract : content of the article
	+ keyword : list of keywords ### Data Splits



Usage
----- ### Full Dataset


Output ### Keyphrase Extraction ### Keyphrase Generation


Contributions
-------------


Thanks to @debanjanbhucs, @dibyaaaaax, @UmaGunturi and @ad6398 for adding this dataset
passage: TAGS
#region-us 
### Data Fields


* id: unique identifier of the document.
* document: Whitespace separated list of words in the document.
* doc\_bio\_tags: BIO tags for each word in the document. B stands for the beginning of a keyphrase and I stands for inside the keyphrase. O stands for outside the keyphrase and represents the word that isn't a part of the keyphrase at all.
* extractive\_keyphrases: List of all the present keyphrases.
* abstractive\_keyphrase: List of all the absent keyphrases.
* other metadata: Additional information present in the original dataset.
	+ id : unique identifier for the document
	+ date : publishing date (YYYY/MM/DD)
	+ categories : categories of the article (1 or 2 categories)
	+ title : title of the document
	+ abstract : content of the article
	+ keyword : list of keywords### Data Splits



Usage
-----### Full Dataset


Output### Keyphrase Extraction### Keyphrase Generation


Contributions
-------------


Thanks to @debanjanbhucs, @dibyaaaaax, @UmaGunturi and @ad6398 for adding this dataset