
# typescript-instruct
A dataset of TypeScript snippets, processed from the typescript subset of [the-stack-smol](https://huggingface.co/datasets/bigcode/the-stack-smol).


# Processing
- Each source file is parsed with the TypeScript AST and queried for 'semantic chunks' of the following types. 
```
ClassDeclaration - 2401                                                                                                                                                         
ArrowFunction - 16443                                                                                                                                                           
MethodDeclaration - 12096                                                                                                                                                       
FunctionDeclaration - 3226                                                                                                                                                      
TypeAliasDeclaration - 1489                                                                                                                                                     
InterfaceDeclaration - 5240                                                                                                                                                     
EnumDeclaration - 214 
```
- Leading comments are added to the front of `content`
- Removed all chunks over max sequence length (2048)
- Deduplicated / cleaned up
- Generated instructions w/ `gpt-3.5-turbo`
  - Ran into of OpenAI API for the month, will finish other half next month



# Dataset Structure
```python
from datasets import load_dataset
load_dataset("bleugreen/typescript-instruct")
DatasetDict({
    train: Dataset({
      features: ['type', 'content', 'repo', 'path', 'language', 'instruction'],
      num_rows: 41109
  })
})
```
task_categories:text-classification task_categories:text2text-generation task_categories:summarization size_categories:10K<n<100K language:en typescript instruct code region:us
{"language": ["en"], "size_categories": ["10K<n<100K"], "task_categories": ["text-classification", "text2text-generation", "summarization"], "tags": ["typescript", "instruct", "code"]}
TAGS
#task_categories-text-classification #task_categories-text2text-generation #task_categories-summarization #size_categories-10K<n<100K #language-English #typescript #instruct #code #region-us 


# typescript-instruct
A dataset of TypeScript snippets, processed from the typescript subset of the-stack-smol.


# Processing
- Each source file is parsed with the TypeScript AST and queried for 'semantic chunks' of the following types. 

- Leading comments are added to the front of 'content'
- Removed all chunks over max sequence length (2048)
- Deduplicated / cleaned up
- Generated instructions w/ 'gpt-3.5-turbo'
  - Ran into of OpenAI API for the month, will finish other half next month



# Dataset Structure

# typescript-instruct
A dataset of TypeScript snippets, processed from the typescript subset of the-stack-smol. # Processing
- Each source file is parsed with the TypeScript AST and queried for 'semantic chunks' of the following types. 

- Leading comments are added to the front of 'content'
- Removed all chunks over max sequence length (2048)
- Deduplicated / cleaned up
- Generated instructions w/ 'gpt-3.5-turbo'
  - Ran into of OpenAI API for the month, will finish other half next month # Dataset Structure
TAGS
#task_categories-text-classification #task_categories-text2text-generation #task_categories-summarization #size_categories-10K<n<100K #language-English #typescript #instruct #code #region-us 
 # typescript-instruct
A dataset of TypeScript snippets, processed from the typescript subset of the-stack-smol. # Processing
- Each source file is parsed with the TypeScript AST and queried for 'semantic chunks' of the following types. 

- Leading comments are added to the front of 'content'
- Removed all chunks over max sequence length (2048)
- Deduplicated / cleaned up
- Generated instructions w/ 'gpt-3.5-turbo'
  - Ran into of OpenAI API for the month, will finish other half next month # Dataset Structure
passage: TAGS
#task_categories-text-classification #task_categories-text2text-generation #task_categories-summarization #size_categories-10K<n<100K #language-English #typescript #instruct #code #region-us 
# typescript-instruct
A dataset of TypeScript snippets, processed from the typescript subset of the-stack-smol.# Processing
- Each source file is parsed with the TypeScript AST and queried for 'semantic chunks' of the following types. 

- Leading comments are added to the front of 'content'
- Removed all chunks over max sequence length (2048)
- Deduplicated / cleaned up
- Generated instructions w/ 'gpt-3.5-turbo'
  - Ran into of OpenAI API for the month, will finish other half next month# Dataset Structure