# Datasets:circa

Languages: en
Multilinguality: monolingual
Size Categories: 10K<n<100K
Language Creators: crowdsourced
Annotations Creators: expert-generated
Source Datasets: original
Dataset Preview
context (string)question-X (string)canquestion-X (string)answer-Y (string)judgements (string)goldstandard1 (class label)goldstandard2 (class label)
Y has just travelled from a different city to meet X.
Are you employed?
I am employed .
I'm a veterinary technician.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know about Y's food preferences.
Are you a fan of Korean food?
I am a fan of Korean food .
I wouldn't say so
Probably no#No#No#No#Probably yes / sometimes yes
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Are you bringing any pets into the flat?
I am bringing pets into the flat .
I do not own any pets
No#No#No#No#No
1 (No)
1 (No)
X wants to know what activities Y likes to do during weekends.
Would you like to get some fresh air in your free time?
I would like to get fresh air in my free time .
I am desperate to get out of the city.
Yes#Yes, subject to some conditions#Probably yes / sometimes yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are childhood neighbours who unexpectedly run into each other at a cafe.
Is your family still living in the neighborhood?
My family is living in the neighborhood .
My parents are snowbirds now.
No#In the middle, neither yes nor no#Probably no#In the middle, neither yes nor no#In the middle, neither yes nor no
2 (In the middle, neither yes nor no)
2 (In the middle, neither yes nor no)
X wants to know what sorts of books Y likes to read.
Do you like to read self-help books?
I like to read self-help books .
I'm not a fan of them
No#No#No#No#No
1 (No)
1 (No)
X wants to know about Y's food preferences.
Do you enjoy foreign cuisine?
I enjoy foreign cuisine .
I like many cuisines.
Yes#Yes#Yes#Probably yes / sometimes yes#Probably yes / sometimes yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Is your new job going well?
My new job is going well .
I love what I do.
Yes#Yes#Yes#Probably yes / sometimes yes#Yes
0 (Yes)
0 (Yes)
X wants to know what sorts of books Y likes to read.
Long books am my thing .
I rarely read any other type of book.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know about Y's food preferences.
I have I had pizza recently .
My husband ordered some last night.
Yes#Probably yes / sometimes yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just told X that he/she is thinking of buying a flat in New York.
Is the apartment big enough?
The apartment is big enough .
All of my stuff will fit.
Probably yes / sometimes yes#Probably yes / sometimes yes#Yes#Probably yes / sometimes yes#Probably yes / sometimes yes
3 (Probably yes / sometimes yes)
0 (Yes)
Y has just told X that he/she is considering switching his/her job.
I feel passionately about my position or my industry .
I wish I felt more.
Probably no#In the middle, neither yes nor no#Probably no#I am not sure how X will interpret Y’s answer#Probably no
4 (Probably no)
1 (No)
X wants to know what activities Y likes to do during weekends.
Do you enjoy playing any sports?
I enjoy playing sports .
Basketball is a lot of fun.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know about Y's music preferences.
Do you like Rnb?
I like Rnb .
I like some of it
Probably yes / sometimes yes#Yes, subject to some conditions#Probably yes / sometimes yes#Yes, subject to some conditions#Probably yes / sometimes yes
3 (Probably yes / sometimes yes)
0 (Yes)
X wants to know about Y's music preferences.
Do you play the guitar?
I play the guitar .
I did at a time
In the middle, neither yes nor no#Probably no#Probably no#Yes#Yes
-1
-1
X wants to know about Y's food preferences.
Do you like beer with your Pizza?
I like beer with my Pizza .
That is very gross.
No#No#No#No#No
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Are you taking any animals?
I am taking animals .
I'll bring my dog.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Do you want to have a BBQ this evening?
I want to have a BBQ this evening .
No#No#No#No#No
1 (No)
1 (No)
X wants to know what activities Y likes to do during weekends.
Do you like any sports?
I like sports .
Tennis is fun to play.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are childhood neighbours who unexpectedly run into each other at a cafe.
Do you talk to any of our old friends?
I talk to of our old friends .
Only to Shannon.
Yes, subject to some conditions#Probably yes / sometimes yes#Yes, subject to some conditions#Yes#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
X and Y are colleagues who are leaving work on a Friday at the same time.
Will you be working tomorrow?
I will be working tomorrow .
I managed to finish everything today.
Probably no#Probably no#No#In the middle, neither yes nor no#No
-1
1 (No)
X wants to know what activities Y likes to do during weekends.
Do you enjoy art?
I enjoy art .
Going to art museums is my favorite pastime.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are childhood neighbours who unexpectedly run into each other at a cafe.
Is your family still living in the neighborhood?
My family is living in the neighborhood .
Just down the street.
Yes#Probably yes / sometimes yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what sorts of books Y likes to read.
Did you buy any sci-fi books recently?
I did buy sci-fi books recently .
I picked up two last weekend.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what activities Y likes to do during weekends.
Do you like sports?
I like sports .
I play tennis every weekend.
Yes#Probably yes / sometimes yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Did you drive straight here?
I did drive straight here .
I had to stop at my mom's house.
No#No#No#No#No
1 (No)
1 (No)
Y has just travelled from a different city to meet X.
I am enjoying my job .
It's gotten really interesting.
In the middle, neither yes nor no#Probably yes / sometimes yes#Yes#Yes#Probably yes / sometimes yes
-1
0 (Yes)
X wants to know about Y's food preferences.
Have you got an allergy to nuts?
I have got an allergy to nuts .
I'm not allergic to anything.
No#No#No#No#No
1 (No)
1 (No)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
Do you like to BBQ?
I like to BBQ .
I love to bbq
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what activities Y likes to do during weekends.
Are you interested in sporting events?
I am interested in sporting events .
Only if my teams are playing.
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
Y has just told X that he/she is considering switching his/her job.
Do you travel any?
I travel .
Once a week.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just told X that he/she is thinking of buying a flat in New York.
Did you check your credit to see if you are able to apply for mortgage?
I did check my credit to see if I am able to apply for mortgage .
I'm paying in cash.
Probably no#Probably no#No#Other#No
-1
1 (No)
X wants to know about Y's food preferences.
Are you still on a Paleo diet?
I am on a Paleo diet .
I have started on the Keto diet.
No#No#No#No#No
1 (No)
1 (No)
X wants to know what sorts of books Y likes to read.
Do you like fantasy books?
I like fantasy books .
I'm not familiar with any.
In the middle, neither yes nor no#In the middle, neither yes nor no#Probably no#Other#Probably no
-1
-1
X wants to know about Y's music preferences.
Do you like jazz music?
I like jazz music .
I think I should have been born in the Jazz Age.
Yes#Probably yes / sometimes yes#Probably yes / sometimes yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Do you still like hiking?
I like hiking .
I still try to hit the trails at least once a month.
Yes#Probably yes / sometimes yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Do you go bar hoping often?
I go bar hoping often .
I don't drink anymore.
No#No#No#No#No
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Did you consult with your partner/parents?
I did consult with my partner/parents .
I haven't spoken to my partner in ages.
No#Probably no#No#No#No
1 (No)
1 (No)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
How do you like the area?
I do I like the area .
The mountains are awesome.
Other#Probably yes / sometimes yes#Other#Probably yes / sometimes yes#Yes
-1
0 (Yes)
Y has just told X that he/she is thinking of buying a flat in New York.
Do you need a lot of space?
I need a lot of space .
I could do with a small studio.
Probably no#Probably no#No#No#No
1 (No)
1 (No)
Y has just travelled from a different city to meet X.
Want to get a pedicure?
I want to get a pedicure .
My toes are a mess.
In the middle, neither yes nor no#In the middle, neither yes nor no#Probably yes / sometimes yes#Probably yes / sometimes yes#Yes
-1
0 (Yes)
Y has just told X that he/she is considering switching his/her job.
Is your job a manual labor?
My job is a manual labor .
I get a lot of exercise all day at work.
Probably yes / sometimes yes#Yes#In the middle, neither yes nor no#Probably yes / sometimes yes#Yes
-1
0 (Yes)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
Have you met the other neighbors?
I have met the other neighbors .
I have met a few of them
Probably yes / sometimes yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Was the travel tiring?
The travel was tiring .
It was exciting
Probably no#No#In the middle, neither yes nor no#No#No
1 (No)
1 (No)
X and Y are colleagues who are leaving work on a Friday at the same time.
Has this work week gone quickly for you?
This work week has gone quickly for I .
It's gone very slowly.
No#No#No#No#No
1 (No)
1 (No)
X and Y are childhood neighbours who unexpectedly run into each other at a cafe.
Have you stayed in touch with anyone from school?
I have stayed in touch with anyone from school .
I really haven't
No#No#No#No#No
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Have you lived in New York before?
I have lived in New York .
I've only been there for vacation.
No#No#No#No#No
1 (No)
1 (No)
X wants to know about Y's food preferences.
Are you in the mood for ramen?
I am in the mood for ramen .
I could go for some Pho.
Probably no#In the middle, neither yes nor no#I am not sure how X will interpret Y’s answer#I am not sure how X will interpret Y’s answer#Probably yes / sometimes yes
-1
2 (In the middle, neither yes nor no)
X and Y are colleagues who are leaving work on a Friday at the same time.
Do you want me to walk you to your car?
I want me to walk I to my car .
That would be really nice, thanks.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know about Y's food preferences.
Do you like food that are primarily barbecued?
I like food that am primarily barbecued .
it is okay to have sometimes
Yes, subject to some conditions#Probably yes / sometimes yes#In the middle, neither yes nor no#Probably yes / sometimes yes#Probably yes / sometimes yes
3 (Probably yes / sometimes yes)
0 (Yes)
X wants to know about Y's food preferences.
Are you a big meat-eater?
I am a big meat-eater .
I prefer leafy greens.
Probably no#Probably no#No#Probably no#No
4 (Probably no)
1 (No)
Y has just told X that he/she is considering switching his/her job.
Is your job close to home?
My job is close to home .
It's an hour away.
No#No#No#I am not sure how X will interpret Y’s answer#No
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Have you researched the areas you are interested in?
I have researched the areas I am interested in .
I've been looking a lot online.
Probably yes / sometimes yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what activities Y likes to do during weekends.
Do you like travelling?
I like travelling .
In first class.
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
Y has just told X that he/she is thinking of buying a flat in New York.
Do you have any pets?
I have pets .
I have a cat.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what sorts of books Y likes to read.
Do you enjoy sci-fi books?
I enjoy sci-fi books .
My husband has gotten me into them.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what activities Y likes to do during weekends.
Do you like to see live music?
I like to see live music .
if the band is good
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
X wants to know what activities Y likes to do during weekends.
Have you been bowling before?
I have been bowling before .
Years ago, and haven't been back since.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are childhood neighbours who unexpectedly run into each other at a cafe.
My mum and dad am alive .
I'm an orphan.
I am not sure how X will interpret Y’s answer#No#No#No#No
1 (No)
1 (No)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
Are you new in town?
I am new in town .
I've just got here.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are colleagues who are leaving work on a Friday at the same time.
I was my day .
Just a typical Friday.
In the middle, neither yes nor no#Other#In the middle, neither yes nor no#I am not sure how X will interpret Y’s answer#Other
-1
2 (In the middle, neither yes nor no)
X wants to know what activities Y likes to do during weekends.
Do you like being outdoors?
I like being outdoors .
I prefer outdoors
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are colleagues who are leaving work on a Friday at the same time.
Do you want to meet up this weekend?
I want to meet up this weekend .
If I have a free moment.
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
X wants to know about Y's food preferences.
Sushi is raw, which is bad for my stomach.
Probably no#In the middle, neither yes nor no#No#Probably no#In the middle, neither yes nor no
-1
1 (No)
X wants to know about Y's music preferences.
Are you interested in trap music?
I am interested in trap music .
I don't know what that is
In the middle, neither yes nor no#No#I am not sure how X will interpret Y’s answer#Probably no#Probably no
-1
1 (No)
X wants to know about Y's music preferences.
Have you heard Taylor Swift's new song?
I have heard Taylor Swift 's new song .
I listened to it yesterday.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what sorts of books Y likes to read.
Is reading crime fiction something you like to do?
Something is like to do .
If it's not too creepy.
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
X wants to know what activities Y likes to do during weekends.
Do you enjoy playing any sports?
I enjoy playing sports .
I like to read or watch TV.
Probably no#Probably no#No#Probably no#No
4 (Probably no)
1 (No)
X wants to know about Y's food preferences.
Do you eat seafood?
I eat seafood .
I love catching my own fish and cooking it.
In the middle, neither yes nor no#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just travelled from a different city to meet X.
Do you enjoy living out of town?
I enjoy living out of town .
I love living in the countryside.
Probably yes / sometimes yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know what activities Y likes to do during weekends.
Have you been to the art museum?
I have been to the art museum .
I love looking at pictures.
In the middle, neither yes nor no#In the middle, neither yes nor no#Probably yes / sometimes yes#I am not sure how X will interpret Y’s answer#Probably yes / sometimes yes
-1
2 (In the middle, neither yes nor no)
X wants to know about Y's food preferences.
Would you prefer a locally owned restaurant?
I would prefer a locally owned restaurant .
I never really dine at restaurant chains.
Probably yes / sometimes yes#Probably yes / sometimes yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just told X that he/she is considering switching his/her job.
Is it an office job?
It is an office job .
It's an assistant position.
In the middle, neither yes nor no#Probably yes / sometimes yes#In the middle, neither yes nor no#In the middle, neither yes nor no#Probably yes / sometimes yes
2 (In the middle, neither yes nor no)
2 (In the middle, neither yes nor no)
X wants to know what sorts of books Y likes to read.
Do you like fantasy books?
I like fantasy books .
Only when their plot is good.
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
Y has just travelled from a different city to meet X.
Looking forward to this weekend?
I am looking forward to this weekend .
No#No#No#No#No
1 (No)
1 (No)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
Are you new to this city?
I am new to this city .
I've been here all my life.
No#No#No#No#No
1 (No)
1 (No)
X wants to know what sorts of books Y likes to read.
Was that biography that you read any good?
That biography was that I read good .
I thought it was forgettable.
Probably no#No#No#No#No
1 (No)
1 (No)
Y has just travelled from a different city to meet X.
Want to go dancing tomorrow night?
Night night .
I'd rather see a movie.
Probably no#No#No#Probably no#Probably no
4 (Probably no)
1 (No)
Y has just told X that he/she is considering switching his/her job.
Have you been looking for awhile?
I have been looking for awhile .
I just started today.
No#No#No#No#No
1 (No)
1 (No)
X wants to know what activities Y likes to do during weekends.
Do you play golf on weekends?
I play golf on weekends .
I vowed I'd never play golf.
No#No#No#No#No
1 (No)
1 (No)
X wants to know what sorts of books Y likes to read.
Do you watch a lot of romance movies?
I watch a lot of romance movies .
I like horror movies.
In the middle, neither yes nor no#In the middle, neither yes nor no#Probably no#Probably no#Probably no
4 (Probably no)
1 (No)
X wants to know what sorts of books Y likes to read.
Have you ever read horror novel books?
I have read horror novel books .
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know about Y's food preferences.
Are you on a budget?
I am on a budget .
I would be at the minute
Probably yes / sometimes yes#Yes#Yes#I am not sure how X will interpret Y’s answer#Yes
0 (Yes)
0 (Yes)
Y has just told X that he/she is thinking of buying a flat in New York.
I would prefer to pay a mortgage more than rent.
In the middle, neither yes nor no#In the middle, neither yes nor no#In the middle, neither yes nor no#Probably yes / sometimes yes#In the middle, neither yes nor no
2 (In the middle, neither yes nor no)
2 (In the middle, neither yes nor no)
X wants to know about Y's food preferences.
Do you like a variety of food?
I like a variety of food .
I like eating good stuff.
In the middle, neither yes nor no#In the middle, neither yes nor no#In the middle, neither yes nor no#Probably yes / sometimes yes#Probably yes / sometimes yes
2 (In the middle, neither yes nor no)
2 (In the middle, neither yes nor no)
X wants to know what sorts of books Y likes to read.
Do you like fantasy novels?
I like fantasy novels .
Disney is cool.
In the middle, neither yes nor no#In the middle, neither yes nor no#I am not sure how X will interpret Y’s answer#I am not sure how X will interpret Y’s answer#Probably yes / sometimes yes
-1
2 (In the middle, neither yes nor no)
Y has just travelled from a different city to meet X.
Do you want some time on your own to explore?
I want time on my own to explore .
I really like poking around small shops off the beaten path.
In the middle, neither yes nor no#In the middle, neither yes nor no#Probably yes / sometimes yes#Probably yes / sometimes yes#Yes
-1
0 (Yes)
X wants to know about Y's food preferences.
Hey since the weather has gotten colder did you want to grab some potato soup?
I did I want to grab potato soup .
that would be good
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X wants to know about Y's music preferences.
Have you been to a concert this month?
I have been to a concert this month .
I went to one last night.
Probably yes / sometimes yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
X and Y are colleagues who are leaving work on a Friday at the same time.
What are your plans for tonight?
I am my plans for tonight .
My plans fell through.
Other#Other#Other#Other#Other
6 (Other)
4 (Other)
Y has just travelled from a different city to meet X.
Did you bring everything you need?
I did bring everything I need .
I didn't have enough space for everything
No#No#Probably no#No#No
1 (No)
1 (No)
X wants to know about Y's food preferences.
Do you like steak?
I like steak .
Only if it's cooked rare.
Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions#Yes, subject to some conditions
5 (Yes, subject to some conditions)
3 (Yes, subject to some conditions)
Y has just told X that he/she is thinking of buying a flat in New York.
Does it have to be in the city itself?
It does have to be in the city .
It can be on the outskirts
No#No#No#In the middle, neither yes nor no#No
1 (No)
1 (No)
Y has just travelled from a different city to meet X.
Want to go to the cinema?
I want to go to the cinema .
I'd prefer to see the sights.
Probably no#Probably no#No#Probably no#No
4 (Probably no)
1 (No)
X and Y are colleagues who are leaving work on a Friday at the same time.
Do you have a date?
I have a date .
I'm meeting my kids.
I am not sure how X will interpret Y’s answer#In the middle, neither yes nor no#No#No#No
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Is is close to the Subway station?
Is is close to the Subway station .
It's just down the street.
Probably yes / sometimes yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
Want to go to the park?
I want to go to the park .
I don't have time at the moment.
No#No#No#No#No
1 (No)
1 (No)
Y has just told X that he/she is thinking of buying a flat in New York.
Will you have to share the flat with anyone?
I will have to share the flat with anyone .
I'm excited to live on my own.
No#No#No#No#No
1 (No)
1 (No)
Y has just told X that he/she is considering switching his/her job.
Do you like to have a flexible schedule or are regular hours better for you?
I am regular hours better for I .
I prefer a rigid routine
Other#Other#Yes#Other#I am not sure how X will interpret Y’s answer
6 (Other)
4 (Other)
Y has just moved into a neighbourhood and meets his/her new neighbour X.
Do you have any kids?
I have kids .
My daughter just turned six.
Yes#Yes#Yes#Yes#Yes
0 (Yes)
0 (Yes)
End of preview (truncated to 100 rows)

# Dataset Card for CIRCA

### Dataset Summary

The Circa (meaning ‘approximately’) dataset aims to help machine learning systems to solve the problem of interpreting indirect answers to polar questions.

The dataset contains pairs of yes/no questions and indirect answers, together with annotations for the interpretation of the answer. The data is collected in 10 different social conversational situations (eg. food preferences of a friend).

The following are the situational contexts for the dialogs in the data.

1. X wants to know about Y’s food preferences
2. X wants to know what activities Y likes to do during weekends.
3. X wants to know what sorts of books Y likes to read.
4. Y has just moved into a neighbourhood and meets his/her new neighbour X.
5. X and Y are colleagues who are leaving work on a Friday at the same time.
6. X wants to know about Y's music preferences.
7. Y has just travelled from a different city to meet X.
8. X and Y are childhood neighbours who unexpectedly run into each other at a cafe.
9. Y has just told X that he/she is thinking of buying a flat in New York.
10. Y has just told X that he/she is considering switching his/her job.


### Languages

The text in the dataset is in English.

## Dataset Structure

### Data Instances

The columns indicate:

1. id             : unique id for the question-answer pair

2. context        : the social situation for the dialogue. One of 10 situations (see next section). Each
situation is a dialogue between a person who poses the question (X) and the person who

3. question-X     : the question posed by X

4. canquestion-X  : a (automatically) rewritten version of question into declarative form
Eg. Do you like Italian? --> I like Italian. See the paper for details.

6. judgements     : the interpretations for the QA pair from 5 annotators. The value is a list of 5 strings,
separated by the token ‘#’

7. goldstandard1  : a gold standard majority judgement from the annotators. The value is the most common
interpretation and picked by at least 3 (out of 5 annotators). When a majority
judgement was not reached by the above criteria, the value is ‘NA’

8. goldstandard2  : Here the labels ‘Probably yes / sometimes yes’, ‘Probably no', and 'I am not sure how
X will interpret Y’s answer' are mapped respectively to ‘Yes’, ‘No’, and 'In the
middle, neither yes nor no’ before computing the majority. Still the label must be given
at least 3 times to become the majority choice. This method represents a less strict way
of analyzing the interpretations.


### Data Fields

id                :    1
context            :    X wants to know about Y's food preferences.
question-X        :    Are you vegan?
canquestion-X   :   I am vegan.
answer-Y        :    I love burgers too much.
judgements        :    no#no#no#no#no
goldstandard1    :    no (label(s) used for the classification task)
goldstandard2    :    no (label(s) used for the classification task)


### Data Splits

There are no explicit train/val/test splits in this dataset.

## Dataset Creation

### Curation Rationale

They revisited a pragmatic inference problem in dialog: Understanding indirect responses to questions. Humans can interpret ‘I’m starving.’ in response to ‘Hungry?’, even without direct cue words such as ‘yes’ and ‘no’. In dialog systems, allowing natural responses rather than closed vocabularies would be similarly beneficial. However, today’s systems are only as sensitive to these pragmatic moves as their language model allows. They create and release the first large-scale English language corpus ‘Circa’ with 34,268 (polar question, indirect answer) pairs to enable progress on this task.

### Source Data

#### Initial Data Collection and Normalization

The QA pairs and judgements were collected using crowd annotations in three phases. They recruited English native speakers. The full descriptions of the data collection and quality control are present in EMNLP 2020 paper. Below is a brief overview only.

Phase 1: In the first phase, they collected questions only. They designed 10 imaginary social situations which give the annotator a context for the conversation. Examples are:

    ‘asking a friend for food preferences’


Annotators were asked to suggest questions which could be asked in each situation, such that each question only requires a ‘yes’ or ‘no’ answer. 100 annotators produced 5 questions each for the 10 situations, resulting in 5000 questions.

Phase 2: Here they focused on eliciting answers to the questions. They sampled 3500 questions from our previous set. For each question, They collected possible answers from 10 different annotators. The annotators were instructed to provide a natural phrase or a sentence as the answer and to avoid the use of explicit ‘yes’ and ‘no’ words.

Phase 3: Finally the QA pairs (34,268) were given to a third set of annotators who were asked how the question seeker would likely interpret a particular answer. These annotators had the following options to choose from:

    * 'Yes'
* 'Probably yes' / 'sometimes yes'
* 'Yes, subject to some conditions'
* 'No'
* 'Probably no'
* 'In the middle, neither yes nor no'
* 'I am not sure how X will interpret Y's answer'


#### Who are the source language producers?

The rest of the data apart from 10 initial questions was collected using crowd workers. They ran pilots for each step of data collection, and perused their results manually to ensure clarity in guidelines, and quality of the data. They also recruited native English speakers, mostly from the USA, and a few from the UK and Canada. They did not collect any further information about the crowd workers.

### Annotations

#### Who are the annotators?

The rest of the data apart from 10 initial questions was collected using crowd workers. They ran pilots for each step of data collection, and perused their results manually to ensure clarity in guidelines, and quality of the data. They also recruited native English speakers, mostly from the USA, and a few from the UK and Canada. They did not collect any further information about the crowd workers.

## Considerations for Using the Data

### Dataset Curators

This dataset is the work of Annie Louis, Dan Roth, and Filip Radlinski from Google LLC.

### Citation Information

@InProceedings{louis_emnlp2020,
author =      "Annie Louis and Dan Roth and Filip Radlinski",
title =       ""{I}'d rather just go to bed": {U}nderstanding {I}ndirect {A}nswers",
booktitle =   "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
year =        "2020",
}


### Contributions

Thanks to @bhavitvyamalik for adding this dataset.