Six questions to break through the AI hype.
The AI hype-cycle is in full swing. But many of the so call game-changing innovation promoted by AI companies have not actually come to fruition or ended up the way they were promoted. Just take Google Duplex for example - a huge announcement for a digital assistant that could understand natural language and sound like a human speaking. That once consumer-facing could only reserve you a table at some restaurants, some of the time.
Whenever you hear about a supposed success in artificial intelligence. Here's a list of six questions you should ask
Stripping away the rhetoric, what did the AI system actually do?
Here to how general is the result. e.g Does an alleged reading task measure all aspects of reading or just a tiny slice of it.?
Is there a demo where I can try out by examples, be very sceptical if there isn't
If the researchers or their press people allege that AI system is better than humans, then which humans, and how much better?
How far does succeeding at the particular task reported in the new research actually take us toward building genuine artificial intelligence?
How robust is the system?. Could it work just as well as other data sets, without massive amounts of retraining? e.g. Could a game playing machine that mastered chess also play an action-adventure game?. Could a system for reorganising animals correctly identify a creature, if it had never seen before as an animal? Would a driverless car system that was now trained during the day be able to drive at night or in the snow.?
