Notes from a Room

Q: You’re seen as the Godfather of this industry. Do you have any concern about what you’ve wrought?

A: I do a bit. On the other hand, I think whatever going to happen is pretty much inevitable. One person stopping doing this research wouldn’t stop it happening. If my impact is to make it happen a month earlier, that’s about the limit of one person can do.

Q: We haven’t touched on job displacement. Is this going to eat up job after job?

A: I think it’s going to make jobs different. People are going to be doing more of the creative end and less of the routine end.

Q: This is the biggest technological advancement since… is this another industrial revolution, or how should people think of it?

A: I think it’s comparable in scale to the industrial revolution or electricity, or maybe the wheel.

Q: And sentience? I think you have complaints about how you even define that?

A: When it comes to sentience, I’m amazed that people can confidently pronounce that these things are not sentient, and when you ask them what they mean by sentient, they say they don’t really know. So how can you be confident about sentient if you don’t know what sentient means?

Q: So maybe they are already?

A: Who knows. I think whether they’re sentient or not depends on what you mean by sentient. So you better define what you mean by sentient before you answer the question of whether they’re sentient.

Q: Does it matter what we think, or does it only matter whether it effectively acts as if it is sentient?

A: That’s a very good question.

Q: And what’s your answer?

A: I don’t have one.

Q: Because if it’s not sentient, but it decides for whatever reason that it believes it is and that it needs to achieve some goal that’s contrary to our interests but it believes is in its interest, does it really matter in terms of any human reflection?

A: I think a good context to think about this thing is an autonomous lethal weapon. It’s all very well saying it’s not sentient, but when it’s hunting you down to shoot you, you’re going to start thinking it’s sentient.

Q: Or not really caring, not an important standard any more.

A: The kind of intelligence we’re developing is very different from our intelligence. It’s an idiot-savant kind of intelligence. It’s quite possible that if it is at all sentient, it’s sentient in a somewhat different way from us.

Why do I love the Bot? Because it frees me up. It appeals to the laziest parts of my mind. It does my work for me. It will wipe out my doubts. It will think and decide for me. It will kill who I thought I was, which I always secretly wanted.