# Dataset Card for "ts-training"

This is a subset of the TypeScript portion of [The Stack (dedup)](https://huggingface.co/datasets/bigcode/the-stack-dedup), uploaded to the Hugging Face Hub for convenience.

Files with dates _after_ the December 31, 2021 cutoff are excluded from this dataset, since we are using those files for evaluation. Therefore, the remaining files (in this dataset) are available for training.

A file is considered to be after the cutoff if all of `max_{stars|forks|issues}_repo_{stars|forks|issues}_event_min_datetime` (i.e., the first timestamp for a `{stars|forks|issues}` event) are after the cutoff. Otherwise (or if all timestamps are missing), the file is included in this dataset.

## Versions

The default version (`main`) is current `v1.1`.

|Version|Description|
|-|-|
|`v1.1`     | Original version of the training dataset, based on v1.1 of the Stack. Applies the training cutoff (December 31, 2021). Used to train OpenTau. |
|`v1.1full` | Training dataset based on v1.1 of the Stack. Does not apply the training cutoff (December 31, 2021), but applies a filter to remove files that do not parse as valid TypeScript. |
|`v1.1p1`   | Revision of v1.1. Applies a filter to remove files that do not parse as valid TypeScript. |

region:us
{"dataset_info": {"features": [{"name": "hexsha", "dtype": "string"}, {"name": "size", "dtype": "int64"}, {"name": "ext", "dtype": "string"}, {"name": "lang", "dtype": "string"}, {"name": "max_stars_repo_path", "dtype": "string"}, {"name": "max_stars_repo_name", "dtype": "string"}, {"name": "max_stars_repo_head_hexsha", "dtype": "string"}, {"name": "max_stars_repo_licenses", "sequence": "string"}, {"name": "max_stars_count", "dtype": "float64"}, {"name": "max_stars_repo_stars_event_min_datetime", "dtype": "string"}, {"name": "max_stars_repo_stars_event_max_datetime", "dtype": "string"}, {"name": "max_issues_repo_path", "dtype": "string"}, {"name": "max_issues_repo_name", "dtype": "string"}, {"name": "max_issues_repo_head_hexsha", "dtype": "string"}, {"name": "max_issues_repo_licenses", "sequence": "string"}, {"name": "max_issues_count", "dtype": "float64"}, {"name": "max_issues_repo_issues_event_min_datetime", "dtype": "string"}, {"name": "max_issues_repo_issues_event_max_datetime", "dtype": "string"}, {"name": "max_forks_repo_path", "dtype": "string"}, {"name": "max_forks_repo_name", "dtype": "string"}, {"name": "max_forks_repo_head_hexsha", "dtype": "string"}, {"name": "max_forks_repo_licenses", "sequence": "string"}, {"name": "max_forks_count", "dtype": "float64"}, {"name": "max_forks_repo_forks_event_min_datetime", "dtype": "string"}, {"name": "max_forks_repo_forks_event_max_datetime", "dtype": "string"}, {"name": "content", "dtype": "string"}, {"name": "avg_line_length", "dtype": "float64"}, {"name": "max_line_length", "dtype": "int64"}, {"name": "alphanum_fraction", "dtype": "float64"}], "splits": [{"name": "train", "num_bytes": 42270977435, "num_examples": 12133148}], "download_size": 17360072228, "dataset_size": 42270977435}, "extra_gated_prompt": "## Terms of Use for The Stack\nThe Stack dataset is a collection of source code in over 300 programming languages. We ask that you read and acknowledge the following points before using the dataset:\n1. The Stack is a collection of source code from repositories with various licenses. Any use of all or part of the code gathered in The Stack must abide by the terms of the original licenses, including attribution clauses when relevant. We facilitate this by providing provenance information for each data point.\n2. The Stack is regularly updated to enact validated data removal requests. By clicking on \"Access repository\", you agree to update your own version of The Stack to the most recent usable version specified by the maintainers in [the following thread](https://huggingface.co/datasets/bigcode/the-stack/discussions/7). If you have questions about dataset versions and allowed uses, please also ask them in the dataset\u2019s [community discussions](https://huggingface.co/datasets/bigcode/the-stack/discussions/new). We will also notify users via email when the latest usable version changes.\n3. To host, share, or otherwise provide access to The Stack dataset, you must include [these Terms of Use](https://huggingface.co/datasets/bigcode/the-stack#terms-of-use-for-the-stack) and require users to agree to it.\n\nBy clicking on \"Access repository\" below, you accept that your contact information (email address and username) can be shared with the dataset maintainers as well.", "extra_gated_fields": {"Email": "text", "I have read the License and agree with its terms": "checkbox"}}
TAGS
#region-us 

Dataset Card for "ts-training"
==============================


This is a subset of the TypeScript portion of The Stack (dedup), uploaded to the Hugging Face Hub for convenience.


Files with dates *after* the December 31, 2021 cutoff are excluded from this dataset, since we are using those files for evaluation. Therefore, the remaining files (in this dataset) are available for training.


A file is considered to be after the cutoff if all of 'max\_{stars|forks|issues}*repo*{stars|forks|issues}\_event\_min\_datetime' (i.e., the first timestamp for a '{stars|forks|issues}' event) are after the cutoff. Otherwise (or if all timestamps are missing), the file is included in this dataset.


Versions
--------


The default version ('main') is current 'v1.1'.




TAGS
#region-us 

passage: TAGS
#region-us 
