
# Dataset Card for SF Nexus Extracted Features: Chapters and Chunks

## Dataset Description

- **Homepage: https://sfnexus.io/** 
- **Repository: https://github.com/SF-Nexus/extracted-features-notebooks** 
- **Point of Contact: Alex Wermer-Colan**

### Dataset Summary

The SF Nexus Extracted Features Chapters and Chunks dataset contains text and metadata from 403 mid-twentieth century science fiction books, originally digitized from Temple University Libraries' Paskow Science Fiction Collection.
After digitization, the books were cleaned using Abbyy FineReader.
Because this is a collection of copyrighted fiction, the books have been disaggregated. 
To improve performance of topic modeling and other nlp tasks, each book has also been split into chapters and then into chunks of approx. 1000 words. 
Each row of this dataset contains one "chunk" of text as well as metadata about that text's title, author and publication.

### About the SF Nexus Corpus
The Paskow Science Fiction collection contains primarily materials from post-WWII, especially mass-market works of the New Wave era (often dated to 1964-1980).
The digitized texts have also been ingested into HathiTrust's repository for preservation and data curation; they are now viewable on HathiTrust's [Temple page](https://babel.hathitrust.org/cgi/ls?field1=ocr;q1=%2A;a=srchls;facet=htsource%3A%22Temple%20University%22;pn=4) for non-consumptive research.
For more information on the project to digitize and curate a corpus of "New Wave" science fiction, see Alex Wermer-Colan's post on the Temple University Scholars Studio blog, ["Building a New Wave Science Fiction Corpus."](https://sites.temple.edu/tudsc/2017/12/20/building-new-wave-science-fiction-corpus/).

### Languages

English

## Dataset Structure

This dataset contains disaggregated "chunks" of text from mid-twentieth century science fiction books and associated metadata. For example: 

```
{'Unnamed': 7299,
 'Title': 'MILLENNIUM,
 'Author': 'VARLEY',
 'Pub Year': '1983',
 'Chapter': 'None',
 'Chunk': '105',
 'Text': '. . . . . . / 1958 1976 249 A Ambler, And As Baker Ben Berkley Bogart Bova Bova, By Casablanca DEMON Eric Eugene, Eugene, Goes HOTLINE, Herman Humphrey Hupfeld It John John MILLENNIUM MacQuitty, Millennium, Mister Night OF OPHIUCHI One Oregon Oregon...” Organisation, PERSISTENCE Rank Remember Roy Sam THE THE TITAN, The The Time Titanic, VISION, Varley Varley WIZARD, William a about acknowledgement: also an and and and asked author be began bestselling by by by by by by completed continued course, directed do excellent film final had in in in is is is is, name nothing novel novel novel, octopus of of of of of of pet play produced published rain-shrouded s screenplay sinking song soon that the the the the the the the this time title title to to to to travel trilogy was was with with with with written written ’ “ ”'
 'Clean Text': 'a ambler and as baker ben berkley bogart bova bova by casablanca demon eric eugene eugene goes hotline herman humphrey hupfeld it john john millennium macquitty millennium mister night of ophiuchi one oregon oregon organisation persistence rank remember roy sam the the titan the the time titanic vision varley varley wizard william a about acknowledgement also an and and and asked author be began bestselling by by by by by by completed continued course directed do excellent film final had in in in is is is is name nothing novel novel novel octopus of of of of of of pet play produced published rain shrouded s screenplay sinking song soon that the the the the the the the this time title title to to to to travel trilogy was was with with with with written written ''
 'Chunk Word Count': '948',
}
```

### Data Fields

- **Unnamed: int** A unique id for the text
- **Title: str** The title of the book from which the text has been extracted
- **Author: str** The author of the book from which the text has been extracted
- **Pub Year: str** The date on which the book was published (first printing)
- **Chapter: int** The chapter in the book from which the text has been extracted
- **Chunk: int** Number indicating which "chunk" of text has been extracted (chunks are numbered per book; each book was split by chapter and then into n chunks of approx. 1000 words)
- **Text: str** The chunk of text extracted from the book
- **Clean Text: str** The chunk of text extracted from the book with lowercasing performed and punctuation, numbers and extra spaces removed
- **Chunk Word Count: int** The number of words the chunk of text contains

To Be Added: 
- **summary: str** A brief summary of the book, if extracted from library records
- **pub_date: int** The date on which the book was published (first printing)
- **pub_city: int** The city in which the book was published (first printing)
- **lcgft_category: str** Information from the Library of Congress Genre/Form Terms for Library and Archival Materials, if known

### Loading the Dataset

Use the following code to load the dataset in a Python environment (note: does not work with repo set to private)
```
from datasets import load_dataset

# If the dataset is gated/private, make sure you have run huggingface-cli login
dataset = load_dataset("SF-Corpus/EF_Chapters_and_Chunks")
```
Or just clone the dataset repo

```
git lfs install
git clone https://huggingface.co/datasets/SF-Corpus/EF_Chapters_and_Chunks
# if you want to clone without large files – just their pointers
# prepend your git clone with the following env var:
GIT_LFS_SKIP_SMUDGE=1
```

## Dataset Creation

### Curation Rationale

For an overview of our approach to data curation of literary texts, see Alex Wermer-Colan’s and James Kopaczewski’s article, “The New Wave of Digital Collections: Speculating on the Future of Library Curation”(2022)

### Source Data

The Loretta C. Duckworth Scholars Studio has partnered with Temple University Libraries’ Special Collections Research Center (SCRC) and Digital Library Initiatives (DLI) to build a digitized corpus of copyrighted science fiction literature. Besides its voluminous Urban Archives, the SCRC also houses a significant collection of science-fiction literature. The Paskow Science Fiction Collection was originally established in 1972, when Temple acquired 5,000 science fiction paperbacks from a Temple alumnus, the late David C. Paskow. Subsequent donations, including troves of fanzines and the papers of such sci-fi writers as John Varley and Stanley G. Weinbaum, expanded the collection over the last few decades, both in size and in the range of genres. SCRC staff and undergraduate student workers recently performed the usual comparison of gift titles against cataloged books, removing science fiction items that were exact duplicates of existing holdings. A refocusing of the SCRC’s collection development policy for science fiction de-emphasized fantasy and horror titles, so some titles in those genres were removed as well.

## Considerations for Using the Data

This data card only exhibits extracted features for copyrighted fiction; no copyrighted work is being made available for consumption. These digitized files are made accessible for purposes of education and research. Temple University Libraries have given attribution to rights holders when possible. If you hold the rights to materials in our digitized collections that are unattributed, please let us know so that we may maintain accurate information about these materials.

If you are a rights holder and are concerned that you have found material on this website for which you have not granted permission (or is not covered by a copyright exception under US copyright laws), you may request the removal of the material from our site by writing to digitalscholarship@temple.edu.

For more information on non-consumptive research, check out HathiTrust Research Center’s Non-Consumptive Use Research Policy.


## Additional Information

### Dataset Curators
For a full list of conributors to the SF Nexus project, visit [https://sfnexus.io/people/](https://sfnexus.io/people/). 
language:en region:us
{"language": ["en"], "pretty_name": "sf-nexus-ef-chapters-and-chunks"}
TAGS
#language-English #region-us 


# Dataset Card for SF Nexus Extracted Features: Chapters and Chunks

## Dataset Description

- Homepage: URL 
- Repository: URL 
- Point of Contact: Alex Wermer-Colan

### Dataset Summary

The SF Nexus Extracted Features Chapters and Chunks dataset contains text and metadata from 403 mid-twentieth century science fiction books, originally digitized from Temple University Libraries' Paskow Science Fiction Collection.
After digitization, the books were cleaned using Abbyy FineReader.
Because this is a collection of copyrighted fiction, the books have been disaggregated. 
To improve performance of topic modeling and other nlp tasks, each book has also been split into chapters and then into chunks of approx. 1000 words. 
Each row of this dataset contains one "chunk" of text as well as metadata about that text's title, author and publication.

### About the SF Nexus Corpus
The Paskow Science Fiction collection contains primarily materials from post-WWII, especially mass-market works of the New Wave era (often dated to 1964-1980).
The digitized texts have also been ingested into HathiTrust's repository for preservation and data curation; they are now viewable on HathiTrust's Temple page for non-consumptive research.
For more information on the project to digitize and curate a corpus of "New Wave" science fiction, see Alex Wermer-Colan's post on the Temple University Scholars Studio blog, "Building a New Wave Science Fiction Corpus.".

### Languages

English

## Dataset Structure

This dataset contains disaggregated "chunks" of text from mid-twentieth century science fiction books and associated metadata. For example: 



### Data Fields

- Unnamed: int A unique id for the text
- Title: str The title of the book from which the text has been extracted
- Author: str The author of the book from which the text has been extracted
- Pub Year: str The date on which the book was published (first printing)
- Chapter: int The chapter in the book from which the text has been extracted
- Chunk: int Number indicating which "chunk" of text has been extracted (chunks are numbered per book; each book was split by chapter and then into n chunks of approx. 1000 words)
- Text: str The chunk of text extracted from the book
- Clean Text: str The chunk of text extracted from the book with lowercasing performed and punctuation, numbers and extra spaces removed
- Chunk Word Count: int The number of words the chunk of text contains

To Be Added: 
- summary: str A brief summary of the book, if extracted from library records
- pub_date: int The date on which the book was published (first printing)
- pub_city: int The city in which the book was published (first printing)
- lcgft_category: str Information from the Library of Congress Genre/Form Terms for Library and Archival Materials, if known

### Loading the Dataset

Use the following code to load the dataset in a Python environment (note: does not work with repo set to private)

Or just clone the dataset repo



## Dataset Creation

### Curation Rationale

For an overview of our approach to data curation of literary texts, see Alex Wermer-Colan’s and James Kopaczewski’s article, “The New Wave of Digital Collections: Speculating on the Future of Library Curation”(2022)

### Source Data

The Loretta C. Duckworth Scholars Studio has partnered with Temple University Libraries’ Special Collections Research Center (SCRC) and Digital Library Initiatives (DLI) to build a digitized corpus of copyrighted science fiction literature. Besides its voluminous Urban Archives, the SCRC also houses a significant collection of science-fiction literature. The Paskow Science Fiction Collection was originally established in 1972, when Temple acquired 5,000 science fiction paperbacks from a Temple alumnus, the late David C. Paskow. Subsequent donations, including troves of fanzines and the papers of such sci-fi writers as John Varley and Stanley G. Weinbaum, expanded the collection over the last few decades, both in size and in the range of genres. SCRC staff and undergraduate student workers recently performed the usual comparison of gift titles against cataloged books, removing science fiction items that were exact duplicates of existing holdings. A refocusing of the SCRC’s collection development policy for science fiction de-emphasized fantasy and horror titles, so some titles in those genres were removed as well.

## Considerations for Using the Data

This data card only exhibits extracted features for copyrighted fiction; no copyrighted work is being made available for consumption. These digitized files are made accessible for purposes of education and research. Temple University Libraries have given attribution to rights holders when possible. If you hold the rights to materials in our digitized collections that are unattributed, please let us know so that we may maintain accurate information about these materials.

If you are a rights holder and are concerned that you have found material on this website for which you have not granted permission (or is not covered by a copyright exception under US copyright laws), you may request the removal of the material from our site by writing to digitalscholarship@URL.

For more information on non-consumptive research, check out HathiTrust Research Center’s Non-Consumptive Use Research Policy.


## Additional Information

### Dataset Curators
For a full list of conributors to the SF Nexus project, visit URL 
# Dataset Card for SF Nexus Extracted Features: Chapters and Chunks ## Dataset Description

- Homepage: URL 
- Repository: URL 
- Point of Contact: Alex Wermer-Colan ### Dataset Summary

The SF Nexus Extracted Features Chapters and Chunks dataset contains text and metadata from 403 mid-twentieth century science fiction books, originally digitized from Temple University Libraries' Paskow Science Fiction Collection.
After digitization, the books were cleaned using Abbyy FineReader.
Because this is a collection of copyrighted fiction, the books have been disaggregated. 
To improve performance of topic modeling and other nlp tasks, each book has also been split into chapters and then into chunks of approx. 1000 words. 
Each row of this dataset contains one "chunk" of text as well as metadata about that text's title, author and publication. ### About the SF Nexus Corpus
The Paskow Science Fiction collection contains primarily materials from post-WWII, especially mass-market works of the New Wave era (often dated to 1964-1980).
The digitized texts have also been ingested into HathiTrust's repository for preservation and data curation; they are now viewable on HathiTrust's Temple page for non-consumptive research.
For more information on the project to digitize and curate a corpus of "New Wave" science fiction, see Alex Wermer-Colan's post on the Temple University Scholars Studio blog, "Building a New Wave Science Fiction Corpus.". ### Languages

English ## Dataset Structure

This dataset contains disaggregated "chunks" of text from mid-twentieth century science fiction books and associated metadata. For example: ### Data Fields

- Unnamed: int A unique id for the text
- Title: str The title of the book from which the text has been extracted
- Author: str The author of the book from which the text has been extracted
- Pub Year: str The date on which the book was published (first printing)
- Chapter: int The chapter in the book from which the text has been extracted
- Chunk: int Number indicating which "chunk" of text has been extracted (chunks are numbered per book; each book was split by chapter and then into n chunks of approx. 1000 words)
- Text: str The chunk of text extracted from the book
- Clean Text: str The chunk of text extracted from the book with lowercasing performed and punctuation, numbers and extra spaces removed
- Chunk Word Count: int The number of words the chunk of text contains

To Be Added: 
- summary: str A brief summary of the book, if extracted from library records
- pub_date: int The date on which the book was published (first printing)
- pub_city: int The city in which the book was published (first printing)
- lcgft_category: str Information from the Library of Congress Genre/Form Terms for Library and Archival Materials, if known ### Loading the Dataset

Use the following code to load the dataset in a Python environment (note: does not work with repo set to private)

Or just clone the dataset repo ## Dataset Creation ### Curation Rationale

For an overview of our approach to data curation of literary texts, see Alex Wermer-Colan’s and James Kopaczewski’s article, “The New Wave of Digital Collections: Speculating on the Future of Library Curation”(2022) ### Source Data

The Loretta C. Duckworth Scholars Studio has partnered with Temple University Libraries’ Special Collections Research Center (SCRC) and Digital Library Initiatives (DLI) to build a digitized corpus of copyrighted science fiction literature. Besides its voluminous Urban Archives, the SCRC also houses a significant collection of science-fiction literature. The Paskow Science Fiction Collection was originally established in 1972, when Temple acquired 5,000 science fiction paperbacks from a Temple alumnus, the late David C. Paskow. Subsequent donations, including troves of fanzines and the papers of such sci-fi writers as John Varley and Stanley G. Weinbaum, expanded the collection over the last few decades, both in size and in the range of genres. SCRC staff and undergraduate student workers recently performed the usual comparison of gift titles against cataloged books, removing science fiction items that were exact duplicates of existing holdings. A refocusing of the SCRC’s collection development policy for science fiction de-emphasized fantasy and horror titles, so some titles in those genres were removed as well. ## Considerations for Using the Data

This data card only exhibits extracted features for copyrighted fiction; no copyrighted work is being made available for consumption. These digitized files are made accessible for purposes of education and research. Temple University Libraries have given attribution to rights holders when possible. If you hold the rights to materials in our digitized collections that are unattributed, please let us know so that we may maintain accurate information about these materials.

If you are a rights holder and are concerned that you have found material on this website for which you have not granted permission (or is not covered by a copyright exception under US copyright laws), you may request the removal of the material from our site by writing to digitalscholarship@URL.

For more information on non-consumptive research, check out HathiTrust Research Center’s Non-Consumptive Use Research Policy. ## Additional Information ### Dataset Curators
For a full list of conributors to the SF Nexus project, visit URL
TAGS
#language-English #region-us 
 # Dataset Card for SF Nexus Extracted Features: Chapters and Chunks ## Dataset Description

- Homepage: URL 
- Repository: URL 
- Point of Contact: Alex Wermer-Colan ### Dataset Summary

The SF Nexus Extracted Features Chapters and Chunks dataset contains text and metadata from 403 mid-twentieth century science fiction books, originally digitized from Temple University Libraries' Paskow Science Fiction Collection.
After digitization, the books were cleaned using Abbyy FineReader.
Because this is a collection of copyrighted fiction, the books have been disaggregated. 
To improve performance of topic modeling and other nlp tasks, each book has also been split into chapters and then into chunks of approx. 1000 words. 
Each row of this dataset contains one "chunk" of text as well as metadata about that text's title, author and publication. ### About the SF Nexus Corpus
The Paskow Science Fiction collection contains primarily materials from post-WWII, especially mass-market works of the New Wave era (often dated to 1964-1980).
The digitized texts have also been ingested into HathiTrust's repository for preservation and data curation; they are now viewable on HathiTrust's Temple page for non-consumptive research.
For more information on the project to digitize and curate a corpus of "New Wave" science fiction, see Alex Wermer-Colan's post on the Temple University Scholars Studio blog, "Building a New Wave Science Fiction Corpus.". ### Languages

English ## Dataset Structure

This dataset contains disaggregated "chunks" of text from mid-twentieth century science fiction books and associated metadata. For example: ### Data Fields

- Unnamed: int A unique id for the text
- Title: str The title of the book from which the text has been extracted
- Author: str The author of the book from which the text has been extracted
- Pub Year: str The date on which the book was published (first printing)
- Chapter: int The chapter in the book from which the text has been extracted
- Chunk: int Number indicating which "chunk" of text has been extracted (chunks are numbered per book; each book was split by chapter and then into n chunks of approx. 1000 words)
- Text: str The chunk of text extracted from the book
- Clean Text: str The chunk of text extracted from the book with lowercasing performed and punctuation, numbers and extra spaces removed
- Chunk Word Count: int The number of words the chunk of text contains

To Be Added: 
- summary: str A brief summary of the book, if extracted from library records
- pub_date: int The date on which the book was published (first printing)
- pub_city: int The city in which the book was published (first printing)
- lcgft_category: str Information from the Library of Congress Genre/Form Terms for Library and Archival Materials, if known ### Loading the Dataset

Use the following code to load the dataset in a Python environment (note: does not work with repo set to private)

Or just clone the dataset repo ## Dataset Creation ### Curation Rationale

For an overview of our approach to data curation of literary texts, see Alex Wermer-Colan’s and James Kopaczewski’s article, “The New Wave of Digital Collections: Speculating on the Future of Library Curation”(2022) ### Source Data

The Loretta C. Duckworth Scholars Studio has partnered with Temple University Libraries’ Special Collections Research Center (SCRC) and Digital Library Initiatives (DLI) to build a digitized corpus of copyrighted science fiction literature. Besides its voluminous Urban Archives, the SCRC also houses a significant collection of science-fiction literature. The Paskow Science Fiction Collection was originally established in 1972, when Temple acquired 5,000 science fiction paperbacks from a Temple alumnus, the late David C. Paskow. Subsequent donations, including troves of fanzines and the papers of such sci-fi writers as John Varley and Stanley G. Weinbaum, expanded the collection over the last few decades, both in size and in the range of genres. SCRC staff and undergraduate student workers recently performed the usual comparison of gift titles against cataloged books, removing science fiction items that were exact duplicates of existing holdings. A refocusing of the SCRC’s collection development policy for science fiction de-emphasized fantasy and horror titles, so some titles in those genres were removed as well. ## Considerations for Using the Data

This data card only exhibits extracted features for copyrighted fiction; no copyrighted work is being made available for consumption. These digitized files are made accessible for purposes of education and research. Temple University Libraries have given attribution to rights holders when possible. If you hold the rights to materials in our digitized collections that are unattributed, please let us know so that we may maintain accurate information about these materials.

If you are a rights holder and are concerned that you have found material on this website for which you have not granted permission (or is not covered by a copyright exception under US copyright laws), you may request the removal of the material from our site by writing to digitalscholarship@URL.

For more information on non-consumptive research, check out HathiTrust Research Center’s Non-Consumptive Use Research Policy. ## Additional Information ### Dataset Curators
For a full list of conributors to the SF Nexus project, visit URL
passage: TAGS
#language-English #region-us 
# Dataset Card for SF Nexus Extracted Features: Chapters and Chunks## Dataset Description

- Homepage: URL 
- Repository: URL 
- Point of Contact: Alex Wermer-Colan### Dataset Summary

The SF Nexus Extracted Features Chapters and Chunks dataset contains text and metadata from 403 mid-twentieth century science fiction books, originally digitized from Temple University Libraries' Paskow Science Fiction Collection.
After digitization, the books were cleaned using Abbyy FineReader.
Because this is a collection of copyrighted fiction, the books have been disaggregated. 
To improve performance of topic modeling and other nlp tasks, each book has also been split into chapters and then into chunks of approx. 1000 words. 
Each row of this dataset contains one "chunk" of text as well as metadata about that text's title, author and publication.### About the SF Nexus Corpus
The Paskow Science Fiction collection contains primarily materials from post-WWII, especially mass-market works of the New Wave era (often dated to 1964-1980).
The digitized texts have also been ingested into HathiTrust's repository for preservation and data curation; they are now viewable on HathiTrust's Temple page for non-consumptive research.
For more information on the project to digitize and curate a corpus of "New Wave" science fiction, see Alex Wermer-Colan's post on the Temple University Scholars Studio blog, "Building a New Wave Science Fiction Corpus.".### Languages

English## Dataset Structure

This dataset contains disaggregated "chunks" of text from mid-twentieth century science fiction books and associated metadata. For example: passage: ### Data Fields

- Unnamed: int A unique id for the text
- Title: str The title of the book from which the text has been extracted
- Author: str The author of the book from which the text has been extracted
- Pub Year: str The date on which the book was published (first printing)
- Chapter: int The chapter in the book from which the text has been extracted
- Chunk: int Number indicating which "chunk" of text has been extracted (chunks are numbered per book; each book was split by chapter and then into n chunks of approx. 1000 words)
- Text: str The chunk of text extracted from the book
- Clean Text: str The chunk of text extracted from the book with lowercasing performed and punctuation, numbers and extra spaces removed
- Chunk Word Count: int The number of words the chunk of text contains

To Be Added: 
- summary: str A brief summary of the book, if extracted from library records
- pub_date: int The date on which the book was published (first printing)
- pub_city: int The city in which the book was published (first printing)
- lcgft_category: str Information from the Library of Congress Genre/Form Terms for Library and Archival Materials, if known### Loading the Dataset

Use the following code to load the dataset in a Python environment (note: does not work with repo set to private)

Or just clone the dataset repo## Dataset Creation### Curation Rationale

For an overview of our approach to data curation of literary texts, see Alex Wermer-Colan’s and James Kopaczewski’s article, “The New Wave of Digital Collections: Speculating on the Future of Library Curation”(2022)### Source Data

The Loretta C. Duckworth Scholars Studio has partnered with Temple University Libraries’ Special Collections Research Center (SCRC) and Digital Library Initiatives (DLI) to build a digitized corpus of copyrighted science fiction literature. Besides its voluminous Urban Archives, the SCRC also houses a significant collection of science-fiction literature. The Paskow Science Fiction Collection was originally established in 1972, when Temple acquired 5,000 science fiction paperbacks from a Temple alumnus, the late David C. Paskow. Subsequent donations, including troves of fanzines and the papers of such sci-fi writers as John Varley and Stanley G. Weinbaum, expanded the collection over the last few decades, both in size and in the range of genres. SCRC staff and undergraduate student workers recently performed the usual comparison of gift titles against cataloged books, removing science fiction items that were exact duplicates of existing holdings. A refocusing of the SCRC’s collection development policy for science fiction de-emphasized fantasy and horror titles, so some titles in those genres were removed as well.