# Dataset Card for "LogicInference_OA"

This is an re-produce of the dataset from LogicInference Dataset in paper: https://openreview.net/pdf?id=HAGeIS_Lcg9.


The github page of LogicInference Dataset: https://github.com/google-research/google-research/tree/master/logic_inference_dataset.


This dataset is aimed to offer more dataset for Open Assistant project, depending on their demands, there three columns: INSTRUCTION, RESPONSE, SOURCE.


The results in this dataset is a little different from which was introduced in the original paper:


1.For all three splits (IID/OOD/length), only IID is used. In the original paper, it seems that model can reach better performance with data generated by this split method.


2.In the original paper, there are two form of responses: LOGICINFERENCE<sub>b</sub> (with the answer at the beginning) and LOGICINFERENCE<sub>e</sub> (with the answer at the end). This dataset uses LOGICINFERENCE<sub>e</sub>, that means: for all questions, the model will first do logic inference, and give the final answer at the end.


3.The original paper, some parameters in generate_dataset.py are:

N_INFERENCE_PROBLEMS = 5000

N_VARIATIONS = 25

N_EXAMPLES = 200000

TRAIN_RATIO = 0.9

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 0


I choose some new parameters:

N_INFERENCE_PROBLEMS = 10000

N_VARIATIONS = 25

N_EXAMPLES = 55000

TRAIN_RATIO = 1

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 1111


The original script generated 4814 different inference problems and extended all those inference problems to around 200,000 Q-A pairs. My settings generated 5491 different inference problems and extended them to around 54,607 Instruction-Response pairs. I think for Open Assistant projects, maybe the number of different inference problems is more important, and generated many similar Instruction-Response pairs will only add training time and doesn't make much sense.
task_categories:question-answering size_categories:10K<n<100K language:en license:apache-2.0 Logic Inference region:us
{"language": ["en"], "license": "apache-2.0", "size_categories": ["10K<n<100K"], "task_categories": ["question-answering"], "dataset_info": {"features": [{"name": "INSTRUCTION", "dtype": "string"}, {"name": "RESPONSE", "dtype": "string"}, {"name": "SOURCE", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 30414202, "num_examples": 54607}], "download_size": 7588805, "dataset_size": 30414202}, "tags": ["Logic Inference"]}
TAGS
#task_categories-question-answering #size_categories-10K<n<100K #language-English #license-apache-2.0 #Logic Inference #region-us 

# Dataset Card for "LogicInference_OA"

This is an re-produce of the dataset from LogicInference Dataset in paper: URL


The github page of LogicInference Dataset: URL


This dataset is aimed to offer more dataset for Open Assistant project, depending on their demands, there three columns: INSTRUCTION, RESPONSE, SOURCE.


The results in this dataset is a little different from which was introduced in the original paper:


1.For all three splits (IID/OOD/length), only IID is used. In the original paper, it seems that model can reach better performance with data generated by this split method.


2.In the original paper, there are two form of responses: LOGICINFERENCE<sub>b</sub> (with the answer at the beginning) and LOGICINFERENCE<sub>e</sub> (with the answer at the end). This dataset uses LOGICINFERENCE<sub>e</sub>, that means: for all questions, the model will first do logic inference, and give the final answer at the end.


3.The original paper, some parameters in generate_dataset.py are:

N_INFERENCE_PROBLEMS = 5000

N_VARIATIONS = 25

N_EXAMPLES = 200000

TRAIN_RATIO = 0.9

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 0


I choose some new parameters:

N_INFERENCE_PROBLEMS = 10000

N_VARIATIONS = 25

N_EXAMPLES = 55000

TRAIN_RATIO = 1

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 1111


The original script generated 4814 different inference problems and extended all those inference problems to around 200,000 Q-A pairs. My settings generated 5491 different inference problems and extended them to around 54,607 Instruction-Response pairs. I think for Open Assistant projects, maybe the number of different inference problems is more important, and generated many similar Instruction-Response pairs will only add training time and doesn't make much sense.
# Dataset Card for "LogicInference_OA"

This is an re-produce of the dataset from LogicInference Dataset in paper: URL


The github page of LogicInference Dataset: URL


This dataset is aimed to offer more dataset for Open Assistant project, depending on their demands, there three columns: INSTRUCTION, RESPONSE, SOURCE.


The results in this dataset is a little different from which was introduced in the original paper:


1.For all three splits (IID/OOD/length), only IID is used. In the original paper, it seems that model can reach better performance with data generated by this split method.


2.In the original paper, there are two form of responses: LOGICINFERENCE<sub>b</sub> (with the answer at the beginning) and LOGICINFERENCE<sub>e</sub> (with the answer at the end). This dataset uses LOGICINFERENCE<sub>e</sub>, that means: for all questions, the model will first do logic inference, and give the final answer at the end.


3.The original paper, some parameters in generate_dataset.py are:

N_INFERENCE_PROBLEMS = 5000

N_VARIATIONS = 25

N_EXAMPLES = 200000

TRAIN_RATIO = 0.9

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 0


I choose some new parameters:

N_INFERENCE_PROBLEMS = 10000

N_VARIATIONS = 25

N_EXAMPLES = 55000

TRAIN_RATIO = 1

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 1111


The original script generated 4814 different inference problems and extended all those inference problems to around 200,000 Q-A pairs. My settings generated 5491 different inference problems and extended them to around 54,607 Instruction-Response pairs. I think for Open Assistant projects, maybe the number of different inference problems is more important, and generated many similar Instruction-Response pairs will only add training time and doesn't make much sense.
TAGS
#task_categories-question-answering #size_categories-10K<n<100K #language-English #license-apache-2.0 #Logic Inference #region-us 
 # Dataset Card for "LogicInference_OA"

This is an re-produce of the dataset from LogicInference Dataset in paper: URL


The github page of LogicInference Dataset: URL


This dataset is aimed to offer more dataset for Open Assistant project, depending on their demands, there three columns: INSTRUCTION, RESPONSE, SOURCE.


The results in this dataset is a little different from which was introduced in the original paper:


1.For all three splits (IID/OOD/length), only IID is used. In the original paper, it seems that model can reach better performance with data generated by this split method.


2.In the original paper, there are two form of responses: LOGICINFERENCE<sub>b</sub> (with the answer at the beginning) and LOGICINFERENCE<sub>e</sub> (with the answer at the end). This dataset uses LOGICINFERENCE<sub>e</sub>, that means: for all questions, the model will first do logic inference, and give the final answer at the end.


3.The original paper, some parameters in generate_dataset.py are:

N_INFERENCE_PROBLEMS = 5000

N_VARIATIONS = 25

N_EXAMPLES = 200000

TRAIN_RATIO = 0.9

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 0


I choose some new parameters:

N_INFERENCE_PROBLEMS = 10000

N_VARIATIONS = 25

N_EXAMPLES = 55000

TRAIN_RATIO = 1

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 1111


The original script generated 4814 different inference problems and extended all those inference problems to around 200,000 Q-A pairs. My settings generated 5491 different inference problems and extended them to around 54,607 Instruction-Response pairs. I think for Open Assistant projects, maybe the number of different inference problems is more important, and generated many similar Instruction-Response pairs will only add training time and doesn't make much sense.
passage: TAGS
#task_categories-question-answering #size_categories-10K<n<100K #language-English #license-apache-2.0 #Logic Inference #region-us 
