AI safety is an engineering problem: containment and out-of-band monitoring

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 30:43

   You've been highly critical of the Doomer narrative. You can only get there if you build the technology responsibly. That's right. And so the first thing is build the technology responsibility. Notice the word build. It's an engineering word. It's about technology. It's about engineering. Engineers, scientists, researchers, we have to build this technology safely, responsibly. And the question is, can you?

2. 31:10

   And the answer is absolutely yes. Because if you can't, then you shouldn't build it at all. Of course. And so, we either have to believe we can or we shouldn't build it, okay? And so, I believe we can. And so, let me just reason about it with you. So, the first part of it is they say, some people say that AI is uncontrollable because it's impossible to align perfectly. Alignment means this for all the audience.

3. 31:39

   You tell somebody, I want you to do this. And so, the mission is clear. The mission is clear. and it's clear to you because you said it in English, okay? And so it could be, I want you to solve this quadratic equation. And so the mission is to solve the quadratic equation. The alignment is how do you do it. do it. There are many ways to do it. For example, you memorized it. And so the answer is whatever it is. Or you saw the answer in a textbook, so you looked it up. You refer to a reference.

4. 32:15

   Or you went to a drawer and you know that drawer had all the answers. You open the drawer, you picked out the answer. Some people call that cheating. I call that opening the drawer and retrieving the answer. And that's what computers do. There's a lot of different ways you could solve that problem. Now, of course, that one is very explicit. Here's another one, be successful in life. The mission is easy to say. And in fact, we say it to each other all the time. We say to our children, we say to our friends, let's do our best to contribute to society. Well, now the question is how? There are so many ways. How do you reasonably align in everybody's culture and background? And some people think that contributing to society means to be generous, to be kind. Some people might mean that contributors to society make a lot of money

5. 33:04

   and then be philanthropic. Some people might so on and so forth. Isn't that right? Yes. Okay. And some people might just think that building a company alone or doing something alone is a sufficient contribution to society. Creating a beautiful piece of art or all of it is true. Correct. And so the question is, what's the alignment? And it just depends on your background, your culture, maybe has something to do with where you live, where you came from.

6. 33:30

   There's just infinite number of ways. And so my point is this, alignment is going to be a forever problem. If we depend on perfect alignment in order to ensure safety, then we're never going to get there. And so the answer is this, the way we treat, the way we think about babies and when we're raising kids and we have pets or whatever it is, employees, okay, you start with containment. You give them the rights, the access to tools, the access to information, the access to whatever responsibility, capabilities that they have, funds, resources.

7. 34:06

   You give them access based on the job that you ask them to do. You start out with zero trust. When you hire somebody, you start out with zero trust. When you have a baby that's one year old, you put them in a playpen, you assume zero trust. And so zero trust, you give them minimum rights And you expand that boundary as you learn more. And so whenever we have a new AI, we have to assume the same thing.

8. 34:31

   Whenever you're testing an AI, zero trust, minimum boundaries, minimum rights, and you expand it as you test and give yourself confidence. So containment, very tight containment is really important to any autonomous system. Second thing you do, monitoring. And the monitoring has to be out of band. And the reason for that is because if you tell an intelligent person, hey, listen, I'm going to sit here and watch you do your work to make sure you're doing it right.

9. 34:59

   Right. Okay? I'm going to sit here and watch you make sure that you're serving customers properly and that you're working hard enough. Okay? That autonomous system is going to behave differently than when you're not watching. It's like Brad's in the room with me or not. Exactly. Exactly. Exactly. Okay? And so the monitoring has to be out of band, and it has to have the ability to observe that the autonomous system is doing something that was inconsistent with the policy or inconsistent with alignment, and you have the ability to either escalate or shut it down, just turn it off.

10. 35:33

    And so those kind of capabilities have to exist. That's engineering, that's technology, and it's possible. And we announced it recently called Open Shell. We have over 100 companies now joining us. It's got lots and lots of activity around the world because everybody wants the same thing. Starting at the highest level, when I looked around the table at the White House, everybody