# Dataset Card for intro_prog

## Dataset Description

### Dataset Summary

IntroProg is a collection of students' submissions to assignments in various introductory programming courses offered at different universities.
Currently, the dataset contains submissions collected from Dublin City University, and the University of Singapore.

#### Dublin

The Dublin programming dataset is a dataset composed of students' submissions to introductory programming assignments at the University of Dublin. 
Students submitted these programs for multiple programming courses over the duration of three academic years.

#### Singapore 

The Singapore dataset contains 2442 correct and 1783 buggy program attempts by 361 undergraduate students 
crediting an introduction to Python programming course at NUS (National University of Singapore).


### Supported Tasks and Leaderboards

#### "Metadata": Program synthesis

Similarly to the [Most Basic Python Programs](https://huggingface.co/datasets/mbpp) (mbpp), the data split can be used to evaluate
code generations models.

#### "Data"

The data configuration contains all the submissions as well as an indicator of whether these passed the required test. 
 
#### "repair": Program refinement/repair

The "repair" configuration of each dataset is a subset of the "data" configuration
augmented with educators' annotations on the corrections to the buggy programs. 
This configuration can be used for the task of program refinement. In [Computing Education Research](https://faculty.washington.edu/ajko/cer/) (CER), 
methods for automatically repairing student programs are used to provide students with feedback and help them debug their code.

#### "bug": Bug classification

[Coming soon]

### Languages

The assignments were written in Python. 

## Dataset Structure

One configuration is defined by one source dataset *dublin* or *singapore* and one subconfiguration ("metadata", "data", or "repair"):

* "dublin_metadata"
* "dublin_data"
* "dublin_repair"
* "singapore_metadata"
* "singapore_data"
* "singapore_repair"


### Data Instances

[More Information Needed]

### Data Fields

[More Information Needed]

Some of the fields are configuration specific

* submission_id: a unique number identifying the submission
* user: a unique string identifying the (anonymized) student who submitted the solution
* date: the timestamp at which the grading server received the submission
* func_code: the cleaned code submitted
* func_name: the name of the function that had to be implemented
* assingment_id: the unique (string) identifier of the assignment that had to be completed
* academic_year: the starting year of the academic year (e.g. 2015 for the academic year 2015-2016)
* module: the course/module
* test: a human eval-style string which can be used to execute the submitted solution on the provided test cases
* Description: a description of what the function is supposed to achieve
* correct: whether the solution passed all tests or not


### Data Splits

#### Dublin

The Dublin dataset is split into a training and validation set. The training set contains the submissions to the assignments
written during the academic years 2015-2016, and 2016-2017, while the test set contains programs written during the academic year 2017-2018. 

#### Singapore

The Singapore dataset only contains a training split, which can be used as a test split for evaluating how your feedback
methods perform on an unseen dataset (if, for instance, you train your methods on the Dublin Dataset). 

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information


#### Dublin 

#### Singapore

The data was released under a [GNU Lesser General Public License v3.0](https://github.com/githubhuyang/refactory/blob/master/LICENSE) license


### Citation Information

```
@inproceedings{azcona2019user2code2vec,
  title={user2code2vec: Embeddings for Profiling Students Based on Distributional Representations of Source Code},
  author={Azcona, David and Arora, Piyush and Hsiao, I-Han and Smeaton, Alan},
  booktitle={Proceedings of the 9th International Learning Analytics & Knowledge Conference (LAK’19)},
  year={2019},
  organization={ACM}
}
@inproceedings{DBLP:conf/edm/CleuziouF21,
  author    = {Guillaume Cleuziou and
               Fr{\'{e}}d{\'{e}}ric Flouvat},
  editor    = {Sharon I{-}Han Hsiao and
               Shaghayegh (Sherry) Sahebi and
               Fran{\c{c}}ois Bouchet and
               Jill{-}J{\^{e}}nn Vie},
  title     = {Learning student program embeddings using abstract execution traces},
  booktitle = {Proceedings of the 14th International Conference on Educational Data
               Mining, {EDM} 2021, virtual, June 29 - July 2, 2021},
  publisher = {International Educational Data Mining Society},
  year      = {2021},
  timestamp = {Wed, 09 Mar 2022 16:47:22 +0100},
  biburl    = {https://dblp.org/rec/conf/edm/CleuziouF21.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```

### Contributions

[More Information Needed]
region:us
{"dataset_info": [{"config_name": "dublin_metadata", "features": [{"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "reference_solution", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18983, "num_examples": 36}, {"name": "test", "num_bytes": 17403, "num_examples": 35}], "download_size": 41873, "dataset_size": 36386}, {"config_name": "singapore_metadata", "features": [{"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "reference_solution", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5577, "num_examples": 5}], "download_size": 6139, "dataset_size": 5577}, {"config_name": "dublin_data", "features": [{"name": "submission_id", "dtype": "int32"}, {"name": "func_code", "dtype": "string"}, {"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}, {"name": "correct", "dtype": "bool"}, {"name": "user", "dtype": "string"}, {"name": "academic_year", "dtype": "int32"}], "splits": [{"name": "train", "num_bytes": 4412068, "num_examples": 7486}, {"name": "test", "num_bytes": 7737585, "num_examples": 14259}], "download_size": 15756562, "dataset_size": 12149653}, {"config_name": "singapore_data", "features": [{"name": "submission_id", "dtype": "int32"}, {"name": "func_code", "dtype": "string"}, {"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}, {"name": "correct", "dtype": "bool"}], "splits": [{"name": "train", "num_bytes": 5098928, "num_examples": 4394}], "download_size": 5705043, "dataset_size": 5098928}, {"config_name": "dublin_repair", "features": [{"name": "submission_id", "dtype": "int32"}, {"name": "func_code", "dtype": "string"}, {"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}, {"name": "annotation", "dtype": "string"}, {"name": "user", "dtype": "string"}, {"name": "academic_year", "dtype": "int32"}], "splits": [{"name": "train", "num_bytes": 229683, "num_examples": 307}, {"name": "test", "num_bytes": 1451820, "num_examples": 1698}], "download_size": 1929518, "dataset_size": 1681503}, {"config_name": "singapore_repair", "features": [{"name": "submission_id", "dtype": "int32"}, {"name": "func_code", "dtype": "string"}, {"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}, {"name": "annotation", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18979, "num_examples": 18}], "download_size": 21737, "dataset_size": 18979}, {"config_name": "newcaledonia_metadata", "features": [{"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "reference_solution", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9053, "num_examples": 9}], "download_size": 9760, "dataset_size": 9053}, {"config_name": "newcaledonia_data", "features": [{"name": "submission_id", "dtype": "int32"}, {"name": "func_code", "dtype": "string"}, {"name": "assignment_id", "dtype": "string"}, {"name": "func_name", "dtype": "string"}, {"name": "description", "dtype": "string"}, {"name": "test", "dtype": "string"}, {"name": "correct", "dtype": "bool"}], "splits": [{"name": "train", "num_bytes": 932024, "num_examples": 1201}], "download_size": 1198518, "dataset_size": 932024}]}
TAGS
#region-us 

# Dataset Card for intro_prog

## Dataset Description

### Dataset Summary

IntroProg is a collection of students' submissions to assignments in various introductory programming courses offered at different universities.
Currently, the dataset contains submissions collected from Dublin City University, and the University of Singapore.

#### Dublin

The Dublin programming dataset is a dataset composed of students' submissions to introductory programming assignments at the University of Dublin. 
Students submitted these programs for multiple programming courses over the duration of three academic years.

#### Singapore 

The Singapore dataset contains 2442 correct and 1783 buggy program attempts by 361 undergraduate students 
crediting an introduction to Python programming course at NUS (National University of Singapore).


### Supported Tasks and Leaderboards

#### "Metadata": Program synthesis

Similarly to the Most Basic Python Programs (mbpp), the data split can be used to evaluate
code generations models.

#### "Data"

The data configuration contains all the submissions as well as an indicator of whether these passed the required test. 
 
#### "repair": Program refinement/repair

The "repair" configuration of each dataset is a subset of the "data" configuration
augmented with educators' annotations on the corrections to the buggy programs. 
This configuration can be used for the task of program refinement. In Computing Education Research (CER), 
methods for automatically repairing student programs are used to provide students with feedback and help them debug their code.

#### "bug": Bug classification

[Coming soon]

### Languages

The assignments were written in Python. 

## Dataset Structure

One configuration is defined by one source dataset *dublin* or *singapore* and one subconfiguration ("metadata", "data", or "repair"):

* "dublin_metadata"
* "dublin_data"
* "dublin_repair"
* "singapore_metadata"
* "singapore_data"
* "singapore_repair"


### Data Instances



### Data Fields



Some of the fields are configuration specific

* submission_id: a unique number identifying the submission
* user: a unique string identifying the (anonymized) student who submitted the solution
* date: the timestamp at which the grading server received the submission
* func_code: the cleaned code submitted
* func_name: the name of the function that had to be implemented
* assingment_id: the unique (string) identifier of the assignment that had to be completed
* academic_year: the starting year of the academic year (e.g. 2015 for the academic year 2015-2016)
* module: the course/module
* test: a human eval-style string which can be used to execute the submitted solution on the provided test cases
* Description: a description of what the function is supposed to achieve
* correct: whether the solution passed all tests or not


### Data Splits

#### Dublin

The Dublin dataset is split into a training and validation set. The training set contains the submissions to the assignments
written during the academic years 2015-2016, and 2016-2017, while the test set contains programs written during the academic year 2017-2018. 

#### Singapore

The Singapore dataset only contains a training split, which can be used as a test split for evaluating how your feedback
methods perform on an unseen dataset (if, for instance, you train your methods on the Dublin Dataset). 

## Dataset Creation

### Curation Rationale



### Source Data

#### Initial Data Collection and Normalization



#### Who are the source language producers?



### Annotations

#### Annotation process



#### Who are the annotators?



### Personal and Sensitive Information



## Considerations for Using the Data

### Social Impact of Dataset



### Discussion of Biases



### Other Known Limitations



## Additional Information

### Dataset Curators



### Licensing Information


#### Dublin 

#### Singapore

The data was released under a GNU Lesser General Public License v3.0 license






### Contributions


# Dataset Card for intro_prog ## Dataset Description ### Dataset Summary

IntroProg is a collection of students' submissions to assignments in various introductory programming courses offered at different universities.
Currently, the dataset contains submissions collected from Dublin City University, and the University of Singapore. #### Dublin

The Dublin programming dataset is a dataset composed of students' submissions to introductory programming assignments at the University of Dublin. 
Students submitted these programs for multiple programming courses over the duration of three academic years. #### Singapore 

The Singapore dataset contains 2442 correct and 1783 buggy program attempts by 361 undergraduate students 
crediting an introduction to Python programming course at NUS (National University of Singapore). ### Supported Tasks and Leaderboards #### "Metadata": Program synthesis

Similarly to the Most Basic Python Programs (mbpp), the data split can be used to evaluate
code generations models. #### "Data"

The data configuration contains all the submissions as well as an indicator of whether these passed the required test. #### "repair": Program refinement/repair

The "repair" configuration of each dataset is a subset of the "data" configuration
augmented with educators' annotations on the corrections to the buggy programs. 
This configuration can be used for the task of program refinement. In Computing Education Research (CER), 
methods for automatically repairing student programs are used to provide students with feedback and help them debug their code. #### "bug": Bug classification

[Coming soon] ### Languages

The assignments were written in Python. ## Dataset Structure

One configuration is defined by one source dataset *dublin* or *singapore* and one subconfiguration ("metadata", "data", or "repair"):

* "dublin_metadata"
* "dublin_data"
* "dublin_repair"
* "singapore_metadata"
* "singapore_data"
* "singapore_repair" ### Data Instances ### Data Fields



Some of the fields are configuration specific

* submission_id: a unique number identifying the submission
* user: a unique string identifying the (anonymized) student who submitted the solution
* date: the timestamp at which the grading server received the submission
* func_code: the cleaned code submitted
* func_name: the name of the function that had to be implemented
* assingment_id: the unique (string) identifier of the assignment that had to be completed
* academic_year: the starting year of the academic year (e.g. 2015 for the academic year 2015-2016)
* module: the course/module
* test: a human eval-style string which can be used to execute the submitted solution on the provided test cases
* Description: a description of what the function is supposed to achieve
* correct: whether the solution passed all tests or not ### Data Splits #### Dublin

The Dublin dataset is split into a training and validation set. The training set contains the submissions to the assignments
written during the academic years 2015-2016, and 2016-2017, while the test set contains programs written during the academic year 2017-2018. #### Singapore

The Singapore dataset only contains a training split, which can be used as a test split for evaluating how your feedback
methods perform on an unseen dataset (if, for instance, you train your methods on the Dublin Dataset). ## Dataset Creation ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information #### Dublin #### Singapore

The data was released under a GNU Lesser General Public License v3.0 license ### Contributions
TAGS
#region-us 
 # Dataset Card for intro_prog ## Dataset Description ### Dataset Summary

IntroProg is a collection of students' submissions to assignments in various introductory programming courses offered at different universities.
Currently, the dataset contains submissions collected from Dublin City University, and the University of Singapore. #### Dublin

The Dublin programming dataset is a dataset composed of students' submissions to introductory programming assignments at the University of Dublin. 
Students submitted these programs for multiple programming courses over the duration of three academic years. #### Singapore 

The Singapore dataset contains 2442 correct and 1783 buggy program attempts by 361 undergraduate students 
crediting an introduction to Python programming course at NUS (National University of Singapore). ### Supported Tasks and Leaderboards #### "Metadata": Program synthesis

Similarly to the Most Basic Python Programs (mbpp), the data split can be used to evaluate
code generations models. #### "Data"

The data configuration contains all the submissions as well as an indicator of whether these passed the required test. #### "repair": Program refinement/repair

The "repair" configuration of each dataset is a subset of the "data" configuration
augmented with educators' annotations on the corrections to the buggy programs. 
This configuration can be used for the task of program refinement. In Computing Education Research (CER), 
methods for automatically repairing student programs are used to provide students with feedback and help them debug their code. #### "bug": Bug classification

[Coming soon] ### Languages

The assignments were written in Python. ## Dataset Structure

One configuration is defined by one source dataset *dublin* or *singapore* and one subconfiguration ("metadata", "data", or "repair"):

* "dublin_metadata"
* "dublin_data"
* "dublin_repair"
* "singapore_metadata"
* "singapore_data"
* "singapore_repair" ### Data Instances ### Data Fields



Some of the fields are configuration specific

* submission_id: a unique number identifying the submission
* user: a unique string identifying the (anonymized) student who submitted the solution
* date: the timestamp at which the grading server received the submission
* func_code: the cleaned code submitted
* func_name: the name of the function that had to be implemented
* assingment_id: the unique (string) identifier of the assignment that had to be completed
* academic_year: the starting year of the academic year (e.g. 2015 for the academic year 2015-2016)
* module: the course/module
* test: a human eval-style string which can be used to execute the submitted solution on the provided test cases
* Description: a description of what the function is supposed to achieve
* correct: whether the solution passed all tests or not ### Data Splits #### Dublin

The Dublin dataset is split into a training and validation set. The training set contains the submissions to the assignments
written during the academic years 2015-2016, and 2016-2017, while the test set contains programs written during the academic year 2017-2018. #### Singapore

The Singapore dataset only contains a training split, which can be used as a test split for evaluating how your feedback
methods perform on an unseen dataset (if, for instance, you train your methods on the Dublin Dataset). ## Dataset Creation ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information #### Dublin #### Singapore

The data was released under a GNU Lesser General Public License v3.0 license ### Contributions