I’m beside myself with joy. This year’s judges were absolutely wonderful. They knew Chip was a chatbot, but were understanding, indulgent, and looked for the right stuff.
I’m partly kicking myself that my prior cynicism with this contest stopped me from going to England this year, on the Turing centenary, but I was able to see the conversations in real-time at home for the first time ever and share the moment with my family.
After the competition, I was contacted by one of the judges, Tracey Logan, who shared her feedback with me. Even today, I’m walking on air when I think of what she shared with me:
Chip was, I think, the only Chatbot that really seemed to engage with me. ‘He’ apologised for not understanding a question. At one point Chip also suggested I might phrase a question differently so it would be more understandable to ‘him’. Chip didn’t try too hard pretending to be human but instead explained that it hoped to learn more so as to be able to answer my questions better in future. Chip made me realise that I really don’t care whether I’m talking to a human or a computer as long as the conversation is in some way rewarding or meaningful to me. Chip realised that conversations are a two-way street. Give and take. I don’t think any of the other finalists quite got that. One insulted me quite a lot. Another fired so many questions at me I didn’t have time to keep up. And the third was off on cloud cuckoo land somewhere pretending to be a cat. Go figure….? Generally-speaking the chatbots didn’t like answering questions, though Chip was way better than most. However, it couldn’t answer my question about the secret of its success as a human-like chatbot. Can you, Mohan?
Causing paradigm shifts in people is what I live for. (While we’re at it, Go Vegan!.)
People – I’m still in shock with what just happened here. One of my main criticisms of this contest is that despite Hugh’s vision, the judges’ slavish interpretation of the contest rules favored fake backstories, canned responses and other trickery over real effort.
Well, Chip won:
– without spelling mistakes or fake backspaces to correct fake errors
– despite saying stuff like “I didn’t understand what you just said” and “I can’t deal with that syntactic variant yet – instead of “Jim likes peaces?”, use “Does Jim like peaches?”
– despite his inability to say what his profession is (let alone, mother, father, brother, dog’s name, sister-in-law)
This flies in the face of a lot of long-cherished beliefs people have about this contest, including my own. I don’t know if it’s a fluke, but all of the judges were pretty consistent in how they approached this.
I want to emphatically make one point: more than the win, the thing that pleased me the most is that I won on my terms, without a fake backstory, fake typing errors and a preponderance of canned responses. Ironically, when contest organizer David Levy had contacted me and asked me to provide a 200-word-bio for the contest booklets he would be printing, I gave him this:
My name is Mohan Embar and my chatbot entry is Chip Vivant. You can read more about Chip at www.chipvivant.com. Chip and l are from Milwaukee, Wisconsin in the U.S.
I’ve been interested in the Loebner Prize since the early 2000s and finally got around to submitting an entry in 2008. This year’s Loebner Prize Competition is Chip’s fifth attempt and the third time he’s made it to the finals.
When I first entered this contest, I naively believed that a SHRDLU-type entry would handily win the contest because its attempt to respond with genuine, non-canned responses within a limited problem domain would justifiably be construed as “most human”. Several years later, however, I have become more cynical and believe that the current incarnation of this competition is nothing more than a glorified creative writing contest, not due to Hugh’s original vision, but because of most judges’ slavish intepretation of the contest rules despite chatbots being in their infancy. Given that any savvy interrogator can bust any bot with five minutes of training, my hope is that this contest will eventually reward entries that attempt to innovate in ways other than fake backstories and canned responses.
Not surprisingly, he refused to print this, instead opting to scrape my website for the relevant information and chiding me for criticizing a contest I was choosing to participate in. I have mixed feelings about his refusal, but it’s all well and good now because my concerns turned out to be invalid. (For the record, my admiration for his accomplishments and work in organizing this contest crushingly outweigh my thoughts on this matter.)
It’s all good. It’ll take more than my allotted fifteen minutes to wipe this smile off my face.