Turing Test

Player C, the Interrogator, is given the task of trying to determine which player – A or B – is a computer and which is human, based on the responses given to written questions.
Wikimedia Commons

The Turing Test, also known as the imitation game, was suggested by the English mathematician and computer scientist Alan Turing as a way to determine whether or not a machine is capable of adequately simulating human cognition. Introduced in his seminal paper “Computing Machinery and Intelligence” (1950), published in the journal Mind, it is widely but erroneously believed to be the benchmark for artificial intelligence (AI), and remains the most cited article in the field.^[1]

A computer program called Eugene Goostman was claimed to be the first to pass the Turing test, in 2014, but it has been argued that all that was really proven was how easily humans can be fooled.^[2]

Background

The world’s first electronic stored-program computer, the Small-Scale Experimental Machine
World’s first electronic stored-program computer. (SSEM) ran its first program in June 1948. In announcing it and its successor, the Manchester Mark 1
One of the earliest stored-program computers, developed at the Victoria University of Manchester from the Small-Scale Experimental Machine which went operational in 1948. , the British press used the description “electronic brain”,^[3] a phrase that had used earlier by Lord Louis Mountbatten in a speech delivered to the British Institution of Radio Engineers on 31 October 1946.^[4] The description provoked a negative reaction from Sir Geoffrey Jefferson, professor of neurosurgery at the University of Manchester. On being asked to deliver the Lister Oration on 9 June 1949 he chose as his subject “The Mind of Mechanical Man”, in which he argued that:

Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain – that is, not only write it but know that it had written it.^[5]

Turing, on the other hand, considered questions such as “Can machines think?” to be “too meaningless to deserve discussion” arguing that his test pragmatically addressed a more meaningful question;^[6] is it possible for a machine to exhibit intelligent behaviour?^[7]

How the test works

The essential idea is to communicate with an unknown entity using either a keyboard and/or screen, and to decide on the basis of answers to questions, whether the responding agent is a human or a computer.^[8]^[a]In Turing’s original imitation game Player A is a man and Player B a woman, and the Interrogator’s task is to determine which is which. The role of the man is to try and trick the Interrogator into making the wrong choice, whereas the woman tries to be helpful.^[9]

The Turing test starts by supposing that there are two rooms, one of which contains a person, the other a computer. Both rooms are linked by means of text‐only communication to an Interrogator whose job it is, by means of questioning the entities in each room, to determine which room contains the computer and which the person. Any and all questions that can be transmitted via typed text are fair game. If after a lengthy session of no‐holds‐barred questions, the Interrogator cannot distinguish the computer from the person, then the computer is declared to be intelligent (i.e. to be thinking).^[1]

There are variations on this standard interpretation of the Turing test, but what they all have in common is that their purpose is not to determine whether a computer is able to fool an interrogator, but whether it can imitate a human.^[9]

A computer is considered to have passed the test if it is mistaken for a human more than 30% of the time during a series of 5-minute keyboard conversations, a milestone that was reached by a program called Eugene Goostman during Turing Test 2014, held at the Royal Society in London.^[10]

Commentary

The fundamental principle underpinning the Turing test is that whatever acts as if it is sufficiently intelligent is intelligent,^[1] but many experts in the field, such as the cognitive scientist Gary Marcus, take the view that the test only demonstrates how easily humans can be fooled.^[2]

What Goostman’s victory really reveals, though, is … the ease with which we can fool others … When Goostman is out of its depth – which is most of the time – it attempts to “change the subject if possible … asking questions, steer[ing] the conversation, [and] occasionally throw[ing] in some humour.”
— Gary Marcus^[2]

In addition, when considering questions such as “Is holding a gulp of Coke in your mouth more like having pins and needles in your feet or having cold water poured on your head?”, any computer’s lack of a human body or human experience would make it extremely difficult for it to answer such sub-cognitive questions convincingly. This has led to the suggestion that the Turing test could be adapted to provide a test for human consciousness, graded by how similar the responses given are to those obtained from a random sample of people.^[9]

Notes[+]

Notes
a	In Turing’s original imitation game Player A is a man and Player B a woman, and the Interrogator’s task is to determine which is which. The role of the man is to try and trick the Interrogator into making the wrong choice, whereas the woman tries to be helpful.^[9]

References

p. 236