Last week, a team of students from the University of California, Davis, won the global 2018 Amazon Alexa prize and $500,000 at the AWS re:Invent 2018 conference. They created a chatbot Gunrock that can communicate with humans on topics such as entertainment, sports, politics, technology, and fashion. The chatbot was named after the University’s mascot. Gunrock maintained an average of 9 minutes and 59 seconds of conversation in the final round. It scored 3.1 out of 5. The second prize went to the Team Alquist from the Czech Technical University in Prague scoring 2.6/5 and winning $100,000 prize money.
This year, the finalists were announced on Twitch, live. The teams used Conversational Bot (CoBot) toolkit, Alexa Skills Kit, and the AWS cloud for creating socialbots for Alexa. The top teams were chosen on the basis of potential scientific contribution in the field of AI and research, the technical merit of approaches, the novelty of their ideas, and the team’s ability to execute against their plan. The finals were held at Amazon’s Seattle headquarters over two days in early November which involved three interactors who held conversations with the socialbots, and also academia experts and industry professionals who served as judges.
The Amazon launched the Alexa prize in 2016 to overcome the challenge of building agents which can carry multi-turn open domain conversations. The objective of this competition is to build agents that can converse coherently and engage with humans for 20 minutes.Last year nearly 3 million Alexa U.S. customers logged around more than 162,000 hours of conversation with the 2017 Alexa prize bots.
The Gunrock team programmed the chatbot using the conversational data from millions of Amazon Alexa users. The team of 11 students was led by Zhou Yu, an assistant professor in the Computer Science department.
The most striking feature about the Gunrock bot is that it uses language disfluencies or pauses such as “hm” or “ah.” This makes Gunrock more human like and different from traditional bots. The team worked on a natural language understanding model for breaking down the dialogue into self-contained semantic units and analyzing the language for determining the context. They further integrated structured knowledge bases such as Google knowledge into Gunrock. This helped the bot in handling a wide variety of user behaviors which includes topic switching and question answering.
Some people on Hacker News are skeptical of this win. A user pointed out, “Only the Gunrock team had an industry professional, Chun-Yen C, maybe the team won because of this reason.” People are confused as to what benefit is Amazon getting out of such competitions. A user said, “Since each of the teams was given around US$250,000, is it sort of a paid work?”
There are also questions raised on the idea of language disfluency. A user said, “ Google also recently faced a backlash over its feature in the Virtual Assistant.” As sentiments are something very natural and they might not sound convincing when coming from a bot.
To know more about this news, check out Amazon’s blog post.
Read Next
Amazon re:Invent Day 3: Lamba Layers, Lambda Runtime API and other exciting announcements!
Amazon announces the public preview of AWS App Mesh, a service mesh for microservices on AWS