The Animal-AI Olympics

April 30th: The competition environment is now available!

Now with $30,000 in prizes! *

Competition environment available here

Full competition releasing into the wild on June 30th

AI has made significant progress in recent years, reaching superhuman performance on a wide range of tasks. Humans are no longer the best Go players, quiz-show contestants, or even, in some respects, the best doctors. Yet state-of-the art AI cannot compete with simple animals at adapting to unexpected changes in the environment. This competition pits our best AI approaches against the animal kingdom to determine if the great successes of AI are now ready to compete with the great successes of evolution at their own game.

The Playground (early version)

We are proposing a new kind of AI competition. Instead of providing a specific task, we will provide a well-defined arena (available at the end of April) and a list of cognitive abilities that we will test for in that arena. The tests will all use the same agent with the same inputs and actions. The goal will always be to retrieve the same food items by interacting with previously seen objects. However, the exact layout and variations of the tests will not be released until after the competition.

We expect this to be hard challenge. Winning this competition will require an AI system that can behave robustly and generalise to unseen cases. A perfect score will require a breakthrough in AI, well beyond current capabilities. However, even small successes will show that it is possible, not just to find useful patterns in data, but to extrapolate from these to an understanding of how the world works.

News

April 30th: We are excited to announce partnership with the Whole Brain Architecture Initiative who are sponsoring a standalone prize of $4,000 for the most biologically plausible entry. We hope this will lead to some exciting entries that not only behave like their biological counterparts, but are structured like them as well. The WBA prize winner will not need to be the best overall agent in the competition but will have to pass some simple baseline to be competitive. More details about this extra prize will be released closer to the competition start date at the end of June.

Competition Goals

Benchmark current AI against multiple animal species using a range of established animal cognition tasks.
Present tests towards identifying cognitive abilities of AI systems.
Determine which AI approaches are most promising for these types of tasks.
Create an ongoing benchmark and data repository for artificial cognition.
Determine which aspects of intelligence are challenging for current AI and which AI already excels at.
Create new experiments to feed back into the animal cognition community that can later be tested with animals.
Bring together two different disciplines to share methods and developments.

Experimental environments will be created in the Unity ML-Agents Toolkit, and the specification of the building blocks for all tasks will be freely available to all participants of the competition. Participants are allowed to use any methods and to experiment as much as they like in preparation for the competition, but the exact details of the tests will be kept secret. Participant performance will be measured on a range of tasks from simple combinations of the building blocks to complex tasks, each designed to probe certain cognitive abilities. Top prize will be for performance across the range of tests.

Timeline

January - June 2019: Research and Development
April 2019: 'Environment 0.1' release ✔
June 2019: Competition full release.
June 2019 - November 2019: Online submissions open with regular progress updates and leaderboards
1st November 2019: Deadline for final submission
December 2019: Results announced at NeurIPS 2019
2020+: