[Call for participation] Interactive Grounded Language Understanding in a Collaborative Environment (IGLU) Competition@NeurIPS2021
Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research in this direction, we propose theNeurIPS IGLU competition: Interactive Grounded Language Understanding in a Collaborative Environment.The primary goal of the IGLU competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. Understanding the complexity of the challenge, we split it into sub-tasks to make it feasible for participants.This research challenge is naturally related, but not limited, to two fields of study: Natural Language Understanding and Generation (NLU/G) and Reinforcement Learning (RL). Therefore, the suggested challenge can bring two communities together to approach one of the important challenges in AI. Another important aspect of the challenge is the dedication to perform a human-in-the-loop evaluation as a final evaluation for the agents developed by contestants.The goal of our competition is to approach the following scientific challenge: How to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment? By the interactive agent we mean that the agent is able to follow the instructions correctly, is able to ask for clarification when needed, and is able to quickly adapt newly acquired skills, just like humans are able to do while collaboratively interacting with each other.Tasks and Application Scenarios:Given the current state of the field, our main research challenge might be too complex to suggest a reasonable end-to-end solution. Therefore, we split the problem into the following concrete research tasks:Architect Task:Given target structure, generate step instructions for the BuilderSubmission system:CodaLab - CompetitionBuilder Task:Given Architect-Builder conversation, build target structure:Submission system:https://competitions.codalab.org/competitions/33828Prizes:Architect Task:1st place - 5K $2nd place - 1.5k $3rd place - 500 $Builder Task:1st place - 5K $2nd place - 1.5k $3rd place - 500 $Timeline:July 26 – Stage 1 begins;(Tentative) October 15 – Stage 1 ends;October 22 – Stage 2 begins by deploying the top-3 performing agents for human evaluation;November 26 – The results of Stage 2 are posted, and the list of winning teams per task is released;December 6 – NeurIPS 2021 begins.Upcoming workshops:To make it even easier for you to onboard, we will arrange workshops per task:Architect Task on Sep 9, at 9 am PST: the link to sign up:IGLU - Workshop (iglu-contest.net)Builder Task on Sep 10, at 10 am PST: the link to sign up:IGLU - Workshop (iglu-contest.net)During the workshops, our team will walk you through setup, available baselines, and training environment (for Builder task). You will have a great opportunity to ask any questions, which we probably can answerGuest Lectures:If you have missed our guest lectures. Here are the links to recordings:IGLU - Guest Lecture by Marc (iglu-contest.net)IGLU - Guest Lecture by Jianwei (iglu-contest.net)For more frequent updates:follow us at Twitter@IgluContestThe news section at our website:IGLU (iglu-contest.net)For questions to organizers and mentors use the slack channel:Join IGLU on Slack | SlackRegister for the competition at CodaLab:CodaLab - Competitionhttps://competitions.codalab.org/competitions/33828