What AI can and cannot do today

A two-day artificial intelligence (AI) conference could overlook the opposing point of view, especially held in San Francisco—the epicenter of technology innovation and the center of over-hyped technology.

But by adding Gary Marcus to the speaker roster of the MIT Technology Review’s EmTech Digital conference, we got a balanced view about AI, including engaging criticism about where AI works, where it does not, and why Marcus says the direction of R&D in the AI field will not lead to artificial general intelligence (AGI). AGI is a theoretical machine intelligence that equals human intelligence.

Marcus, a neural science professor at New York University and a leading figure in AI, had a special credibility as a critic because he just sold his 2-year-old AI startup Geometric Intelligence to Uber.

3 things AI can do:

• Speech recognition
• Image recognition, when the number of objects in the image is limited
• Natural language understanding in narrowly bounded domains

6 things AI cannot do:

• Conversational interfaces—ask Siri something off script, and it breaks down
• Automated scientific discovery
• Automated medical diagnosis
• Automated scene comprehension for blind people
• Domestic robots
• Safe and reliable driverless cars

A couple of Marcus’ points are arguable, such as using image understanding to read radiological charts to diagnose disease or read retinal images to diagnose diabetic retinopathy that has achieved equal and, in some cases, superior accuracy to human clinicians. But generally, he is correct. He chose these six areas to make a point of comparison: for these problems with machine intelligence, machines will have to learn more like a child learns language than the way machines are trained today.

Marcus used the example of his nearly 3-year-old daughter Chloe’s common sense to explain what a child can do, that artificial intelligence cannot do. He describes a conversation with his daughter in which he tells her that he will put her artwork into mommy’s armoire. Chloe deduction—mama does not see the artwork, but when she opens the armoire she can see it—as an example of the limits of machine intelligence. He asserted that Chloe could infer what he meant with just a few words without having been trained on 10 million similar situations like most AI systems are trained today. 

AI based on machine learning

With the exception of a few examples of reinforcement learning, such as the Libratus pokerbot and Google’s AlphaGo-bot, most of the AI is based on machine learning that predicts a result using neural networks.

A good example of this is image recognition. Millions of images are shown to a neural network that puts them into categories to train the computer model. Then a set of correctly categorized images is shown to the computer model that mathematically corrects errors made by the neural network using gradient descent, a sophisticated form of averaged error reduction, to improve the precision of the model to correctly predict the objects in an image.

For dramatic effect, Marcus reworded a quote from AI luminary Andrew Ng categorizing the capability of machine intelligence to imitate human intelligence.

Marcus qualified Ng’s statement that anything a person can do in one second can be automated with neural networks and machine learning today. But that’s possible only when there is an enormous corpora of training data to create models to understand topics such as images, translate one language to another, or recognize the meaning of natural language. If the model is to work with high precision, though, the models should be applied to fairly narrow applications that are also not subject to too much change requiring frequent retraining, and the application of the models must have a tolerance for errors, albeit sometimes very low error rates. These systems do not really understand, asserted Marcus, but predict the most likely meaning.

The state of the art in machine learning prediction is about 98 percent. Marcus said this type of precision is OK for systems such as Amazon’s machine learning recommendation system that might recommend another one of his books to someone who bought one of his other books. If the recommendation does not lead to a satisfying read 2 percent of the time, the consequential cost is just $20. He questions the consequences, though, of a self-driving car’s pedestrian detector that is 98 percent accurate or an eldercare robot that drops geriatric patients only 2 percent of the time.

The talk was not all criticism. Marcus says one day intelligent machines will be able to perform complex tasks such as read research papers about a disease like bladder cancer and propose a novel treatment, but not yet—and perhaps not in his lifetime.

Marcus recommends a worldwide, government-funded AI research program like CERN to focus on the long-term AI research needed to create an AGI. CERN operates a nuclear accelerator that cost $4.6 billion to build and is shared among researchers studying particle physics.

Fundamental scientific questions that might take a decade or two of research to produce results need to be answered before an AGI can be created. Blue-sky researchers engaged in fundamental AI research are limited in budget and computational resources. Private sector research is too focused on the technologies that achieve shorter-term financial goals, which Marcus says will not lead to machines with common sense and AGI.