This October I had the privilege of participating in the IBM-Watson-Conversation Hackathon in London, as part of a five person BBC team. We eventually won out of the fifteen teams that participated, with our combination of technology, ‘human interest’ and humour being noted.
These guys, right here….
The brief was simple: use IBM’s Watson powered Conversation engine to create a chatbot, integrating with Watson’s other Artificial Intelligence based APIs (e.g tone analysis, image recognition, context based news etc).
Conversation is a Natural Language Processing (NLP) engine, that allows the construction of non-linear, non-brittle dialogs. It’s integrated into a wider eco-system of IBM and Watson based products, using the IBM BlueMix cloud platform as its bedrock, so getting off the ground is as easy-as-pie. It also enables integration with select external services such as Foursquare and Twilio.
Our idea was a mobile based dating assistant, called wingApp! The idea was that you could enter things like ‘He comes from Italy’ and it would suggest new stories from Italy, or something like ‘she really like me’ and it’ll suggest nearby venues to move onto. All this was done in a very tongue in cheek way, with our own little mascot picture; the app would even give you a ‘get out’ call on your phone if the date goes south!
Each of us had our own roles: one of us did UX design (including the mascot!), another of us integrated with Facebook messenger and so on. I managed to get the hang of the Conversation dialog and training engine very quickly, so I was charged mainly with turning the teams ideas into plausible statements and responses.
Conversation In Detail:
The Conversation platform works by looking for Intents in the users inputs, matching any Entities that it recognises, and responding accordingly. So, the input phrase ‘She works as a Doctor’ could match the Intent of ‘looking for something to talk about’ (as in the user intends to find something related to their date), and the Entity of ‘Doctor’.
The main trick here is that the Intents are not hardcoded phrases; rather, you give Conversation a list of example phrases for an Intent, like so:
- She works as a Doctor
- She’s a Doctor
- I think she’s a scientist
- He’s Musician
Eventually Conversation learns the semantic meaning behind these phrase, and variations thereof, so if it encounters a new permutation of the phrase, it still reads the intention correctly. It has its own built in dictionary of words and idioms to draw from too; for example, I trained it to tell a joke in response to “I want to make them laugh”, or “I need a joke”, but it also recognised the intention from “I want to crack her up!”.
This is conceptually simple to grasp, and works well. Instead of relying on a linear flow for the conversation, Conversation finds the Intent behind the message, and uses that as it’s starting point. Further interactions and conversational branches can then be followed.
In my experience the engine worked well in this regards. Obviously we couldn’t explore it’s true potential in the 48 hours that we had to build the chatbot, but once the basics were established we made good steady progress.
The actual interface was very much in ‘beta’, but the underlying system gave us plenty of scope for more advanced things; for example, there is a shared context object between Conversation and other parts of the Watson platform, so we could easily pass information between the various services. Integration with Facebook messenger was obstacle free as well.
One problem I did encounter was that the training cycle (which was kicked off everytime you entered a new Intention or corrected a mis-matched one) was long, about 20 minutes. This probably wasn’t helped by the fact there were 15 other teams in the same building doing the same thing, however I have used other chatbot platforms with similar concepts where there was no learning time (however, I did feel that the quality of Conversations learning was better).
So what did I think overall? If I was to build an NLP based chatbot Watson-Conversation would be my first port of call, just for ease of integration with the other Watson services, the wider BlueMix workflow and toolset, and the ease of setup. Wit.ai (another chatbot platform) seems more integrated into Facebook, and is possibly quicker to get off the ground with, but it doesn’t have access to the wider IBM ecosystem, and doesn’t work as well in my (limited) experience.
The Human Factor:
One thing that is missing from my description above is that our chatbot was funny and engaged with the user ‘in a human way’; this was picked up on by the judges, and indeed is arguably the whole reason behind Conversation and NLP.
‘Information Technology’ is, at the end of the day, always about trying to solve a human problem, and both Developers and Business managers would do well to remember that.