I FOUND myself in a college library last week, surrounded by philosophers and experts on computer science: one participant, grinning behind his wild beard, placed his phone on the table and had it play a song written — both words and music — by an AI that he had trained.
It was fairly dreadful. Like most of these demonstrations, it seemed more of a reminder of how little creativity there is in most human productions than a proof of the creativity of which machines are supposed to be capable. It certainly wasn’t proof of consciousness, which was the subject of the symposium.
It is 27 years now since I went to a conference on the scientific study of consciousness in Tucson, and, back then, almost everyone expected that this was a problem that would soon be solved. Dan Dennett still stalked the earth, proclaiming that a mystery was simply something that we did not yet understand or had not yet reduced to physics; Francis Crick sent an emissary, Christof Koch, to explain his theory that consciousness was associated with the presence of 40Hz resonances in the brain.
None of those ideas went anywhere, of course, and, over the years, I became a convinced “mysterian”: the derogatory term for those who believe that consciousness is a primary category that cannot be reduced to anything else. Dennett’s strongest argument seemed to me the claim that there had been a time before consciousness, when the earth was just a lifeless coalescence of space debris, so that its evolution and appearance could be entirely accounted for by physical processes that we could, in principle, understand.
LATER, I came to believe that time does not work for conscious beings as it does for clocks, and that, for us, the awareness of time comes with a sense that we can, in some sense, step outside it. This makes sense of the otherwise mysterious epigraph to Auden’s “Memorial for the City”, taken from Julian of Norwich. “In the self-same point that our soul is made sensual, in the self-same point is the City of God ordained to him from without beginning” — although the scientist would have thought of this as a problem of a creature with senses acquiring a soul, rather than a pre-existent soul acquiring a body, as Julian sees it.
In any case, the progress that has been made in the study of consciousness since the ’90s has been to complicate the problem rather than to simplify it. For most of us, the kind of consciousness that is interesting is not simply the ability to respond to internal states, in the way that my laptop might be said by a strict behaviourist to be conscious of the programme that it is running and the keys that I am striking. It is the kind of thing suggested by Thomas Nagel’s famous question “What it is like to be a bat?” Only a creature that can understand that question, and that knows what it is like to be something, can be considered conscious in an interesting way.
Much of the discussion at the seminar that I attended circled round the question whether a machine could ever get that far, even if it seems clear that none has so far done so.
But, at that point, the discussion moved from abstraction to unreality. Everyone took for granted that to acquire consciousness of this sort would change the moral status of the creature or the machine that did so; yet I am reasonably certain that not all the participants were vegans or even vegetarians. Using conscious beings as means to an end is an entirely human activity: babies do it as soon as they can scream, long before they can know that they are doing it.
You don’t have to be Donald Trump to enjoy the infliction of pain or humiliation for its own sake. We all have a little monster inside. AI boosters talk about “aligning” their programmes with moral aims, as if this were an uncomplicated process to be solved by reference to some form of Californian utilitarianism.
But the Israel Defense Forces are already using an AI system to identify targets in Gaza, and then have them killed without any human intervention at all. I am sure that most of the operators believe that they are acting for the good. Even if it were possible to construct a machine that would refuse to carry out such orders, it is impossible to imagine any government that would deliberately build it.
