
# Dataset Card for HebrewSentiment

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://thisworld.online/
- **Repository:** https://github.com/thisworld1/thisworld.online
- **Paper:** 
- **Leaderboard:**
- **Point of Contact:** 

### Dataset Summary

HebrewThisWorld is a data set consists of 2028 issues of the newspaper 'This World' edited by Uri Avnery and were published between 1950 and 1989. Released under the AGPLv3 license.

Data Annotation: 

### Supported Tasks and Leaderboards

Language modeling

### Languages

Hebrew

## Dataset Structure

csv file with "," delimeter

### Data Instances

Sample:

```json
{
  "issue_num": 637,
  "page_count": 16,
  "date": "1950-01-01",
  "date_he": "1 בינואר 1950",
  "year": "1950",
  "href": "https://thisworld.online/1950/637",
  "pdf": "https://olam.eu-central-1.linodeobjects.com/pdfs/B-I0637-D010150.pdf",
  "coverpage": "https://olam.eu-central-1.linodeobjects.com/pages/637/t-1.png",
  "backpage": "https://olam.eu-central-1.linodeobjects.com/pages/637/t-16.png",
  "content": "\nלפיד\nהנוער ־ בירושלים צילומים :\n\nב. רותנברג\n\nוזהו הלפיד\n...",
  "url": "https://thisworld.online/api/1950/637"
}
```

### Data Fields

- `issue_num`: ID/Number of the issue 
- `page_count`: Page count of the current issue
- `date`: Published date
- `date_he`: Published date in Hebrew
- `year`: Year of the issue
- `href`: URL to the issue to scan/print etc.
- `pdf`:  URL to the issue to scan in pdf
- `coverpage`:  URL to coverpage
- `backpage`: URL to backpage
- `content`: text content of the issue
- `url`: URL 


### Data Splits

|        | train | 
|--------|------:|
| corpus |  2028 |

## Dataset Creation


### Curation Rationale

[More Information Needed]

### Source Data

[thisworld.online](https://thisworld.online/)

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations


#### Annotation process

[More Information Needed]

#### Who are the annotators?

Researchers

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

GNU AGPLv3+

This is free software, and you are welcome to redistribute it under certain conditions.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.

### Citation Information

https://thisworld.online/

### Contributions

Thanks to [@lhoestq](https://github.com/lhoestq), [@imvladikon](https://github.com/imvladikon) for adding this dataset.
task_categories:text-generation task_categories:fill-mask task_ids:language-modeling task_ids:masked-language-modeling annotations_creators:expert-generated language_creators:found multilinguality:monolingual size_categories:1K<n<10K source_datasets:original language:he license:agpl-3.0 region:us
{"annotations_creators": ["expert-generated"], "language_creators": ["found"], "language": ["he"], "license": ["agpl-3.0"], "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "source_datasets": ["original"], "task_categories": ["text-generation", "fill-mask"], "task_ids": ["language-modeling", "masked-language-modeling"], "pretty_name": "HebrewSentiment", "dataset_info": {"features": [{"name": "issue_num", "dtype": "int64"}, {"name": "page_count", "dtype": "int64"}, {"name": "date", "dtype": "string"}, {"name": "date_he", "dtype": "string"}, {"name": "year", "dtype": "string"}, {"name": "href", "dtype": "string"}, {"name": "pdf", "dtype": "string"}, {"name": "coverpage", "dtype": "string"}, {"name": "backpage", "dtype": "string"}, {"name": "content", "dtype": "string"}, {"name": "url", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 678389435, "num_examples": 2028}], "download_size": 678322912, "dataset_size": 678389435}}
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-Hebrew #license-agpl-3.0 #region-us 

Dataset Card for HebrewSentiment
================================


Table of Contents
-----------------


* Dataset Description
	+ Dataset Summary
	+ Supported Tasks and Leaderboards
	+ Languages
* Dataset Structure
	+ Data Instances
	+ Data Fields
	+ Data Splits
* Dataset Creation
	+ Curation Rationale
	+ Source Data
	+ Annotations
	+ Personal and Sensitive Information
* Considerations for Using the Data
	+ Social Impact of Dataset
	+ Discussion of Biases
	+ Other Known Limitations
* Additional Information
	+ Dataset Curators
	+ Licensing Information
	+ Citation Information
	+ Contributions


Dataset Description
-------------------


* Homepage: URL
* Repository: URL
* Paper:
* Leaderboard:
* Point of Contact:


### Dataset Summary


HebrewThisWorld is a data set consists of 2028 issues of the newspaper 'This World' edited by Uri Avnery and were published between 1950 and 1989. Released under the AGPLv3 license.


Data Annotation:


### Supported Tasks and Leaderboards


Language modeling


### Languages


Hebrew


Dataset Structure
-----------------


csv file with "," delimeter


### Data Instances


Sample:


### Data Fields


* 'issue\_num': ID/Number of the issue
* 'page\_count': Page count of the current issue
* 'date': Published date
* 'date\_he': Published date in Hebrew
* 'year': Year of the issue
* 'href': URL to the issue to scan/print etc.
* 'pdf': URL to the issue to scan in pdf
* 'coverpage': URL to coverpage
* 'backpage': URL to backpage
* 'content': text content of the issue
* 'url': URL


### Data Splits



Dataset Creation
----------------


### Curation Rationale


### Source Data


URL


#### Initial Data Collection and Normalization


#### Who are the source language producers?


### Annotations


#### Annotation process


#### Who are the annotators?


Researchers


### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------


### Social Impact of Dataset


### Discussion of Biases


### Other Known Limitations


Additional Information
----------------------


### Dataset Curators


### Licensing Information


GNU AGPLv3+


This is free software, and you are welcome to redistribute it under certain conditions.


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.


This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.


You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <URL


URL


### Contributions


Thanks to @lhoestq, @imvladikon for adding this dataset.



### Dataset Summary


HebrewThisWorld is a data set consists of 2028 issues of the newspaper 'This World' edited by Uri Avnery and were published between 1950 and 1989. Released under the AGPLv3 license.


Data Annotation: ### Supported Tasks and Leaderboards


Language modeling ### Languages


Hebrew


Dataset Structure
-----------------


csv file with "," delimeter ### Data Instances


Sample: ### Data Fields


* 'issue\_num': ID/Number of the issue
* 'page\_count': Page count of the current issue
* 'date': Published date
* 'date\_he': Published date in Hebrew
* 'year': Year of the issue
* 'href': URL to the issue to scan/print etc.
* 'pdf': URL to the issue to scan in pdf
* 'coverpage': URL to coverpage
* 'backpage': URL to backpage
* 'content': text content of the issue
* 'url': URL ### Data Splits



Dataset Creation
---------------- ### Curation Rationale ### Source Data


URL #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators?


Researchers ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information


GNU AGPLv3+


This is free software, and you are welcome to redistribute it under certain conditions.


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.


This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.


You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <URL


URL ### Contributions


Thanks to @lhoestq, @imvladikon for adding this dataset.
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-Hebrew #license-agpl-3.0 #region-us 
 ### Dataset Summary


HebrewThisWorld is a data set consists of 2028 issues of the newspaper 'This World' edited by Uri Avnery and were published between 1950 and 1989. Released under the AGPLv3 license.


Data Annotation: ### Supported Tasks and Leaderboards


Language modeling ### Languages


Hebrew


Dataset Structure
-----------------


csv file with "," delimeter ### Data Instances


Sample: ### Data Fields


* 'issue\_num': ID/Number of the issue
* 'page\_count': Page count of the current issue
* 'date': Published date
* 'date\_he': Published date in Hebrew
* 'year': Year of the issue
* 'href': URL to the issue to scan/print etc.
* 'pdf': URL to the issue to scan in pdf
* 'coverpage': URL to coverpage
* 'backpage': URL to backpage
* 'content': text content of the issue
* 'url': URL ### Data Splits



Dataset Creation
---------------- ### Curation Rationale ### Source Data


URL #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators?


Researchers ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information


GNU AGPLv3+


This is free software, and you are welcome to redistribute it under certain conditions.


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.


This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.


You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <URL


URL ### Contributions


Thanks to @lhoestq, @imvladikon for adding this dataset.