task_categories:text-generation task_categories:text2text-generation task_categories:summarization language:en license:apache-2.0 code ruby CodeSearchNet Summary region:us
---
dataset_info:
  features:
  - name: repo
    dtype: string
  - name: path
    dtype: string
  - name: func_name
    dtype: string
  - name: original_string
    dtype: string
  - name: language
    dtype: string
  - name: code
    dtype: string
  - name: code_tokens
    sequence: string
  - name: docstring
    dtype: string
  - name: docstring_tokens
    sequence: string
  - name: sha
    dtype: string
  - name: url
    dtype: string
  - name: partition
    dtype: string
  - name: summary
    dtype: string
  splits:
  - name: train
    num_bytes: 125295534
    num_examples: 53228
  download_size: 44007267
  dataset_size: 125295534
license: apache-2.0
task_categories:
- text-generation
- text2text-generation
- summarization
language:
- en
tags:
- code
- ruby
- CodeSearchNet
- Summary
pretty_name: Ruby CodeSearchNet with Summaries
---
# Dataset Card for "code-search-net-ruby"

## Dataset Description

- **Homepage:** None
- **Repository:** https://huggingface.co/datasets/Nan-Do/code-search-net-go
- **Paper:** None
- **Leaderboard:** None
- **Point of Contact:** [@Nan-Do](https://github.com/Nan-Do) 

### Dataset Summary

This dataset is the Ruby portion of the CodeSarchNet annotated with a summary column.  
The code-search-net dataset includes open source functions that include comments found at GitHub.  
The summary is a short description of what the function does.  

### Languages

The dataset's comments are in English and the functions are coded in Ruby

### Data Splits

Train, test, validation labels are included in the dataset as a column.

## Dataset Creation

May of 2023

### Curation Rationale

This dataset can be used to generate instructional (or many other interesting) datasets that are useful to train LLMs

### Source Data

The CodeSearchNet dataset can be found at https://www.kaggle.com/datasets/omduggineni/codesearchnet

### Annotations

This datasets include a summary column including a short description of the function.

#### Annotation process

The annotation procedure was done using [Salesforce](https://huggingface.co/Salesforce) T5 summarization models.  
A sample notebook of the process can be found at https://github.com/Nan-Do/OpenAssistantInstructionResponsePython  
The annontations have been cleaned to make sure there are no repetitions and/or meaningless summaries. (some may still be present in the dataset)  

### Licensing Information

Apache 2.0