DSTC10

The Tenth Dialog System Technology Challenge

News

September 24, 2021 - Submission deadline for 'Situated Interactive Multimodal Conversational AI' has been extended. More details are here.

September 20, 2021 - Submission deadline for 'Automatic Evaluation and Moderation of Open-domain Dialogue Systems' has been extended to Sept 26th (AoE)

September 20, 2021 - Submission site for 'Reasoning for Audio Visual Scene-Aware Dialog' is open here

September 14, 2021 - Test set has been released for 'Reasoning for Audio Visual Scene-Aware Dialog' here

September 14, 2021 - Test set has been released for 'MOD: Internet Meme Incorporated Open-domain Dialog' here

September 13, 2021 - Workshop paper submission site here

September 13, 2021 - Test set has been released for 'Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations' task 1 and task 2

August 25, 2021 - DSTC10 workshop will be held at AAAI-22. Call for Participation announced

June 15, 2021 - Track websites along with training or validation datasets released for all tracks here

June 11, 2021 - 'Reasoning for Audio Visual Scene-Aware Dialog' track proposal updated here

May 28, 2021 - 'Automatic Evaluation and Moderation of Open-domain Dialogue Systems' track resources available here

May 28, 2021 - Second call for proposals announced with training/test release dates.

May 27, 2021 - Challenge registration has opened! Please fill out this form

April 12, 2021 - DSTC 10 Accepted Tracks announced. Call for Participation announced

December 15, 2020 - DSTC10 website launched

December 14, 2020 - DSTC10 call for proposals announced

Background and Motivation

Starting as an initiative to provide a common testbed for the task of Dialog State Tracking, the first Dialog State Tracking Challenge (DSTC) was organized in 2013, followed by DSTC2&3 in 2014, DSTC4 in 2015, and DSTC5 in 2016. Given the remarkable success of the first five editions, and understanding both, the complexity of the dialog phenomenon and the interest of the research community in a wider variety of dialog related problems, the DSTC rebranded itself as "Dialog System Technology Challenges" for its sixth edition. Then, DSTC6, DSTC7, DSTC8 and DSTC9 have been completed in 2017, 2018, 2019 and 2020 respectively.

In this tenth challenge, the call for task proposals has resulted into the following 5 different tracks: (1) MOD: Internet Meme Incorporated Open-domain Dialog (2) Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations, (3) SIMMC 2.0: Situated Interactive Multimodal Conversational AI, (4) Reasoning for Audio Visual Scene-Aware Dialog and (5) Automatic Evaluation and Moderation of Open-domain Dialogue Systems. The objective of these tracks is to invite interested organizations conduct dialog related challenges in specific areas of research and under the umbrella of the DSTC.