It’s uncommon to open the technology section of a newspaper or specialised blog and not see an article about Artificial Intelligence (AI) or Machine Learning (ML). Most of the time, both terms are used as interchangeable buzzwords, removing much of their technical meaning. This fact may leave many wondering if they are somehow synonyms and, if not, what the difference is between them. In this post, I will try to shed some light on this topic.

First, in order to understand how we got here, let’s review a bit of AI history (or click here for a comprehensive list).

A bit of history

Humans have always been captivated by understanding how intelligence works and, in the quest for a conclusive answer, lots of machines that tried to emulate human capabilities have been designed and built. However, the history of AI has been a chain of boom-and-bust cycles. We’re currently immersed in the third cycle of optimism (maybe, misplaced optimism) which, if not understood properly, may lead to a cutback in enthusiasm and funding as seen twice before. (You may find it interesting to read Michael Jordan, the AI expert’s, reflections on this).

The first boom took place in the late 50s and 60s when efforts focused on answering if machines could actually think: the search for the so-called general or strong AI. Some breakthroughs consisted in having machines that could prove mathematical theorems, solve high school algebra word problems, search through mazes or even keep a human-like conversation as ELIZA was able to. The invention of the perceptron (an early example of artificial neuron or machine learning classifier) in 1957 was, for some, an unambiguous indication that general or strong AI was very close. However, subsequent works were not able to meet the expectations that had been created and this caused substantial cuts in funding from governments and companies, which brought AI research almost to a halt. This period is known as the first AI winter.

It was not until the 80s that the emergence of expert systems put AI in the limelight again. In this case, the developments revolved around the idea of creating knowledge bases that an inference engine (following logical rules) used to answer questions about a specific domain of knowledge, e.g. medical diagnosis. Recurrent neural networks and the backpropagation algorithm (a common method of training artificial neural networks) were also developed. However, the computational power at the time hampered remarkable improvements which brought the second AI winter.

In the 90s, a new vision brought fresh air to AI (see Moravec’s paradox). Scientists realised that, for decades, they had focused on the wrong problem: they were trying to design machines to solve problems that today’s humans find challenging (playing chess or proving mathematical theorems). The more difficult tasks, however, were indeed those that we do innately, almost effortlessly, like recognising faces and moving around. They advocated building intelligence “from the bottom up” and taking into account the role of “the body” in human intelligence. Consequently, the quest for general AI lost momentum and efforts were redirected to solve specific isolated problems. This gave rise to the so-called narrow or weak AI.

This more cautious approach and an increase in computing power happened to be really successful. Some celebrated milestones were:

• IBM Deep Blue beating the reigning world chess champion Gary Kasparov (1997)
• Driverless cars completing a 212 km off-road course in the second edition of the DARPA Grand Challenge (2005) (the first, only one year before, was a complete fiasco)
• IBM Watson destroys the world best’s Jeopardy! champions (2011)
• DeepMind AlphaGo defeates the 18-time world Go champion Lee Sedol (2016)

It was also the time when advanced ML algorithms like Support Vector Machines, Random Forests, and the area of Reinforcement Learning were developed. Also, the discipline of AI benefited from other areas of knowledge when the economist’s definition of a rational agent was married to the computer scientist’s definition of an object. This shaped what is nowadays known as an intelligent agent.

The actual difference

You may have already spotted that Machine Learning is, indeed, a subset of AI. In short, ML is the mechanism a machine uses to learn. That is, to improve its behavior based on experience. If you want to get into the details, I recommend having a look at this post. A lot of people depict this relationship between ML and AI by means of a Venn diagram like the following, which indirectly answers our original question:

According to this diagram, when asked for the difference between AI and ML, the proper set-theoretic answer would be the items included within the shaded area in the figure. In other words, the other tasks that constitute AI along with ML.

So, what other intelligent capabilities, apart from learning, would we like a machine to have? What follows is my attempt to enumerate them.

Problem solving

Humans, as intelligent beings, are able to solve problems. One such example would be that of solving a Sudoku. Machines can take different approaches to the problem:

• Classical search: one solution method is to perform an ordered searching among all possible solution candidates. Well-known algorithms include BFS, DFS, A*, backtracking, the use of heuristics as a model of intuitive decision-taking, etc.
• Adversarial search: two-player games are usually more difficult in the sense that search must take into account the other player’s decisions. Some example algorithms are minimax search, alpha-beta pruning, expectiminimax, etc.
• Optimisation: a very different kind of search came to prominence in the 90s, based on the mathematical theory of optimisation. In some problems, we can take a guess and then refine it incrementally until no more refinements can be made. Some optimisation algorithms are hill climbing, simulated annealing, beam search, random optimisation and many others, including those usually classified under the umbrella of evolutionary computation. This category includes, mainly, swarm intelligence (ant colony, particle swarm optimisation) and evolutionary algorithms (genetic algorithms, gene expression programming, differential evolution, etc.).

Knowledge management

Humans are good at acquiring knowledge and storing it in a way that is appropriate for its use in future reasoning. In particular, intelligent machines would need to be able to accomplish:

• Knowledge retrieval and extraction: obtain knowledge from different information sources such as text, images, audio recording, videos.
• Knowledge representation: the ability to represent knowledge as entities and formalise the relationships among them.
• Collaborative filtering: in many areas, it is necessary to extract knowledge from data coming from multiple, sometimes contradicting, sources of information. This problem is common in the financial industry when the same data is retrieved from several financial sources.

Reasoning

Once knowledge has been retrieved and properly represented, a set of rules can be used to derive new knowledge:

• Logical reasoning: when knowledge is certain, machines can leverage propositional logic rules to obtain new statements that add up to the existing knowledge base.
• Uncertain reasoning: in the presence of uncertainty, it is still possible to reason by using tools like Bayesian Networks. Some other tools like Hidden Markov Models, Kalman Filtering (that are examples of Dynamic Bayesian Networks) can be regarded as particularisations of Bayesian Networks. These are especially suitable for reasoning over time and solving problems such as filtering, predicting and smoothing of time series. Decision making is an area that has benefited from this type of reasoning, and also from mathematical advancements in an economy such as utility functions (for preference encoding) and game theory (for multi-agent problems). Some celebrated models for sequential decision problems are Markov Decision Processes (and their Partially Observable counterpart).

Interaction

As we previously mentioned, endowing intelligent machines with body-parts (or alike) is necessary for them to interact with humans and the real-world. Furthermore, our perception of intelligence improves if it’s able to keep a conversation, hear, see, grab objects or navigate the world. Here are some pointers to start researching these topics:

• Human-computer interaction:
  • Natural language processing covers reading, understanding, translating and writing texts.
  • The ability to hear and speak is termed speech recognition and synthesis, respectively.
  • Social intelligence involves recognising, interpreting, processing, and simulating human affects. This may be useful to predict human actions.
• Real world interaction (which encompasses things usually associated to robotics):
  • Perception: the ability to use input from sensors (cameras, microphones, tactile sensors) to deduce aspects of the world. Recall that machines can also have superpowers, in the sense that they may exceed human capabilities when equipped with more exotic sensors such as sonar, infrared cameras, etc.
  • Motion/manipulation: the ability to act on surroundings, grab/move objects, walk/move around, etc.

All this said, it leads me to a couple of philosophical questions.

How intelligent do you think this machine is? Are we that close to the holy grail of movie-like artificial intelligence? Place your bets in the comments below!