# Dataset Card for "github-code-haskell-function"

Rows: 3.26M
Download Size: 1.17GB

This dataset is extracted from [github-code-haskell-file](https://huggingface.co/datasets/blastwind/github-code-haskell-file).

Each row has 3 flavors of the same function: 
`uncommented_code`: Includes the function and its closest signature. 
`function_only_code`: Includes the function only.
`full_code`: Includes the function and its closest [signature](https://wiki.haskell.org/Type_signature) and comment. 
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the function 
is neither a signature nor comment, `full_code` is just the function. If the previous neighbor is one though, include 
them appropriately, then search the previous neighbor for the other node with the same logic.

Further, each row also contains attribute values for my personal analysis project. The attributes are calculated from the code in column `uncommented_code`. 
7% (225k) of the rows have cyclomatic complexity and LOC valued at `-1` because [`homplexity`](https://github.com/BlastWind/homplexity) failed in parsing the row's `uncommented_code`.

task_categories:text-generation size_categories:1M<n<10M code haskell region:us
{"size_categories": ["1M<n<10M"], "task_categories": ["text-generation"], "dataset_info": {"features": [{"name": "repo_name", "dtype": "string"}, {"name": "path", "dtype": "string"}, {"name": "license", "dtype": "string"}, {"name": "full_code", "dtype": "string"}, {"name": "full_size", "dtype": "int64"}, {"name": "uncommented_code", "dtype": "string"}, {"name": "uncommented_size", "dtype": "int64"}, {"name": "function_only_code", "dtype": "string"}, {"name": "function_only_size", "dtype": "int64"}, {"name": "is_commented", "dtype": "bool"}, {"name": "is_signatured", "dtype": "bool"}, {"name": "n_ast_errors", "dtype": "int64"}, {"name": "ast_max_depth", "dtype": "int64"}, {"name": "n_whitespaces", "dtype": "int64"}, {"name": "n_ast_nodes", "dtype": "int64"}, {"name": "n_ast_terminals", "dtype": "int64"}, {"name": "n_ast_nonterminals", "dtype": "int64"}, {"name": "loc", "dtype": "int64"}, {"name": "cycloplexity", "dtype": "int64"}], "splits": [{"name": "train", "num_bytes": 2166157579, "num_examples": 2284385}, {"name": "valid", "num_bytes": 307778276, "num_examples": 326341}, {"name": "test", "num_bytes": 620756348, "num_examples": 652682}], "download_size": 1597070903, "dataset_size": 3094692203}, "tags": ["code", "haskell"]}
TAGS
#task_categories-text-generation #size_categories-1M<n<10M #code #haskell #region-us 

# Dataset Card for "github-code-haskell-function"

Rows: 3.26M
Download Size: 1.17GB

This dataset is extracted from github-code-haskell-file.

Each row has 3 flavors of the same function: 
'uncommented_code': Includes the function and its closest signature. 
'function_only_code': Includes the function only.
'full_code': Includes the function and its closest signature and comment. 
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the function 
is neither a signature nor comment, 'full_code' is just the function. If the previous neighbor is one though, include 
them appropriately, then search the previous neighbor for the other node with the same logic.

Further, each row also contains attribute values for my personal analysis project. The attributes are calculated from the code in column 'uncommented_code'. 
7% (225k) of the rows have cyclomatic complexity and LOC valued at '-1' because 'homplexity' failed in parsing the row's 'uncommented_code'.

# Dataset Card for "github-code-haskell-function"

Rows: 3.26M
Download Size: 1.17GB

This dataset is extracted from github-code-haskell-file.

Each row has 3 flavors of the same function: 
'uncommented_code': Includes the function and its closest signature. 
'function_only_code': Includes the function only.
'full_code': Includes the function and its closest signature and comment. 
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the function 
is neither a signature nor comment, 'full_code' is just the function. If the previous neighbor is one though, include 
them appropriately, then search the previous neighbor for the other node with the same logic.

Further, each row also contains attribute values for my personal analysis project. The attributes are calculated from the code in column 'uncommented_code'. 
7% (225k) of the rows have cyclomatic complexity and LOC valued at '-1' because 'homplexity' failed in parsing the row's 'uncommented_code'.
TAGS
#task_categories-text-generation #size_categories-1M<n<10M #code #haskell #region-us 
 # Dataset Card for "github-code-haskell-function"

Rows: 3.26M
Download Size: 1.17GB

This dataset is extracted from github-code-haskell-file.

Each row has 3 flavors of the same function: 
'uncommented_code': Includes the function and its closest signature. 
'function_only_code': Includes the function only.
'full_code': Includes the function and its closest signature and comment. 
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the function 
is neither a signature nor comment, 'full_code' is just the function. If the previous neighbor is one though, include 
them appropriately, then search the previous neighbor for the other node with the same logic.

Further, each row also contains attribute values for my personal analysis project. The attributes are calculated from the code in column 'uncommented_code'. 
7% (225k) of the rows have cyclomatic complexity and LOC valued at '-1' because 'homplexity' failed in parsing the row's 'uncommented_code'.