In the ongoing consternation about the rapid advance of artificial intelligence, one of the most immediate and grounded fears is that it will replace humans in endeavors that were previously thought to be essential to the human experience. But we still have a long way to go before this happens in many areas. For one thing, human intelligence developed around having a human body, and having a sense of cause and effect in the real world that a machine might never be able to discern. How does someone working deep within the industry balance these concerns about human uniqueness with our endless hunger to know how our own brains – and how our machines – work?
Dr. Peter Stone is the Chief Scientist of Sony AI. He is the founder and director of the Learning Agents Research Group (LARG) within the Artificial Intelligence Laboratory in the Department of Computer Science at The University of Texas at Austin, as well as associate department chair and Director of Texas Robotics.
(This post is part of Sinai and Synapses’ project Scientists in Synagogues, a grass-roots program to offer Jews opportunities to explore the most interesting and pressing questions surrounding Judaism and science. Dr. Stone led one in a series of post-Tikkun Leil Shavuot classes titled “Jewish Ethics, AI and the Future” at Congregation Agudas Achim in Austin, TX; this post is adapted from the transcript.)
Starting with Vision
Peter Stone: So how did I get into artificial intelligence? There was one faculty member in my undergrad days – I don’t even remember who it was – who sort of introduced us to the problem of computer vision and the idea at that time, this was, like, in the late mid 1980s or so, he made the point that if I draw a square on a piece of paper or a circle or a triangle, it’s very easy for any kid, any person, to say, identify the shape. But at that time, there weren’t good computer programs that could point a camera at it and tell you even what shape it is.
At the famous conference called the Dartmouth Conference in the 1950s, where the term “artificial intelligence” was coined, one of the people, sort of famously, thought that computer vision would be a really good summer project for a graduate student – “Why don’t you go and solve a computer vision problem?” It turned out to be more like a 60-year kind of project.
Kirk Simon, at the time, predicted that it would take about ten years for a computer to beat the human chess champion. And it took about 50 years, but it happened. We’ve moved a lot since then with computer vision, but he introduced the world, through that example, through that example of chess, to the question of, “Can we use computers to replicate or understand what’s going on in the human brain?” In some ways, it’s one of the big scientific questions of our lifetimes – of this century.
I figured I wanted to try to get at this question, and there were a number of different ways to do it. So I went to some neuroscience classes, and I was sort of frustrated that they could tell me what was going on chemically between two neurons – you know, the synapses, the neurotransmitters. But it was very, very low-level. There was nothing close to understanding what happens in your brain when you look at a shape on a piece of paper and identify that it’s a square, or you tell me your phone number, and I remember it. So then I tried going to some psychology classes, which were sort of getting it in the nature of intelligence from the other direction. “Let’s look at behaviors. Let’s come up with theories for how these work at a very high level.” And I found those interesting, but also not really getting at the question of “What’s the nature of intelligence?” in a satisfying way to me.
And then I went to a simple computer science class, and that was connected to my interest in mathematics as well, but was introduced to some artificial intelligence classes where the premise was that if we can create intelligence in a machine, or behaviors that we would consider intelligent if a person didn’t know, that would give us some insight into what the nature of intelligence is. And so it wasn’t about replicating the human brain exactly. Like neuroscience, it was interested in not necessarily replicating it, but understanding exactly how it works.
The analogy that’s often given is that it was more like trying to understand flight not by building a bird, but by building an airplane. An airplane doesn’t fly the same way as a bird, but when you build the airplane, you learn some of the principles of flight, the Bernoulli principle of air and lift and things like that, that birds also use. So, if we could create intelligence in a machine, we would learn something that would tell us about the nature of intelligence as an abstract thing and maybe get some kind of insight into how intelligence works.
Artificial intelligence was a fairly small field at the time with a rich history, and there have been ups and downs. We were in the middle of an “AI winter” of sorts when I started. It was a field that had had its heyday already and was in decline. People sort of looked at me strangely, like, “Why are you going into AI? Everybody knows that was tried, and it failed.” but I wasn’t going into it for the commercial applications.
Sony AI was launched because they acquired my startup company, Cogitai, which was founded in 2015. Sony AI is now an organization with a couple hundred people working on projects having to do with video games, which is the group that I’ve been most involved in recently, but also in robotics, and in artificial intelligence, and even in a gastronomy project, to help chefs.
I think Google and sort of the big companies that people think of as the AI powerhouses here, like Amazon as well, get a lot of their revenue, basically, from getting people to buy things, and click on ads. I really value being at Sony, partly because that’s not the thing we’re trying to strive for – it’s more to help make people more creative, to create more, and to help both artists and to help people who consume art.
Neil Blumofe: I really wanted you to be the first one on the panel because you represent, more than the other speakers in this series, a perspective that really thinks about AI as a good – not without things we should watch out for, but overall, your feeling is really very positive. Is that fair to say about a future with AI?
Peter Stone: Yeah, I mean, I think I wouldn’t be doing what I’m doing if I didn’t think that artificial intelligence is, on balance, making the world a better place. But I think it’s a complicated story. As with any technology, I think there are fantastic things that technologies like the automobile have done for us, and on the other hand, 40,000 people a year are killed in the United States alone in traffic fatalities. And airplanes have done a lot of good things for us, but they’re also used as weapons. Social media serves to connect us and yet is causing lots of problems among our youth. So I think it’s not a news story that there are technologies that have possible positive uses and possible negative uses.
Artificial intelligence, I think, is no different. There are huge opportunities in areas of health care, in climate science, science in general – fantastic results. Recently, it’s been used in protein folding, which is changing the whole nature of biology, research and drug discovery. Even better weather forecasting, traffic mitigation, reducing congestion… there are all kinds of positive potential implications of artificial intelligence. And unless you’ve been under a rock recently, there are also possible negative implications that people are worried about. Things like bias and spreading misinformation, deepfakes, and possibly being used in military realms without check.
And so, like any other technology, there’s a responsibility for us as a society to understand and think about how we shape our technologies and their regulations to maximize the chances that the benefits will outweigh the harms.
There are a few differences from previous technologies. The pace of progress recently has just been mind-blowing. I’ve been surprised by a lot of the things that have happened over the last few years. When I was interviewed on NPR, I used this analogy: when the Model T Ford was introduced in 1908, to get from that moment to the point where there were 100 million cars on the road took somewhere between 50 and 70 years. There were about 50 million cars in the 1950s and 200 million cars in the 1970s. And during those 50 years, lots of regulations were put into place. Roads were paved, stop signs were introduced, traffic signals, seat belts, a little later, airbags, emission standards, things like that. When ChatGPT was introduced, it took a month for it to get 100 million users. So, instead of those 50 years, there is this widespread dissemination of a technology in a really, really short period of time – change that’s much faster than our typical political and governmental systems are, used to dealing with.
But all of these things are the reason why, relatively recently, people have started talking about ethical artificial intelligence. When I was a graduate student, nobody thought that it was at all important for a computer scientist to learn anything about ethics, because AI’s time had come and gone. But now we’re in a time where people are recognizing that computer science students should be educated in the humanities and in social sciences – and that humanities students and social science students should know a little bit about artificial intelligence. For that reason, actually, I taught a course last fall called “The Essentials of Artificial Intelligence.” “AI literacy” is another way of thinking about it. And I’m teaching the course again in an expanded version this coming fall, because I actually think now, at this point, every student who graduates from a university should have at least basic literacy in artificial intelligence – and that includes the music majors and the English majors.
So we’re trying to make AI accessible to everybody, and at the same time, engage the computer scientists. I co-taught a graduate course for computer scientists with a humanist on ethical artificial intelligence, because that’s the reality of where we’re at. And I think it’s exciting, but it’s also a little scary.
Ray Kurzweil and the Singularity
Neil Blumofe: This quote is from Dr. Ray Kurzweil: “When you talk to a human in 2035, you’ll be talking to someone that’s a combination of biological and non-biological intelligence.”
Peter Stone: Yeah, so the notion of the singularity, for those who don’t know, is based on what’s called Moore’s Law. It’s not really a law, but historically, the amount of computational power you can get for the same cost has doubled roughly every year and a half. That’s sort of an exponential advance.
AI programs are also helping the programmers. Just as they’re automating the tedious parts of the creative process, in many ways, they’re automating the tedious part of computer programming. If you want to write some code, sometimes it’s code that’s been written before, or you’re going to write the same thing over and over again, you don’t have to type it yourself – it can just be imported. Still, there is a creative process to it that’s being driven by people. It’s making people more efficient.
Dr. Kurzweil’s projection was that if that trend continued by the year 2030 – I think that is what he usually said, but maybe he’s expanded it now to 2035 – that computers would have the same computational power as the human brain. The human brain has been a constant in making computational power double over time. Human computational power being matched by computer power would mean that all of a sudden, the next generation of computation would not be designed by people. Now it would be computers designing computers, and that would give you almost instantly infinite computing power.
And once you have that, then everything is possible. And you can predict what’s going to happen; you can be optimistic that the whole world will be a utopia, or if you’re pessimistic, it will be a dystopia. Dr. Kurzweil has tended to be more on the pessimistic side, but that’s been a concept I think he’s still standing by. Many people are saying it’s sort of playing out that way he predicted, especially the people who think that we’re on that cusp where artificial intelligence is possibly going to get to the point where it can’t be controlled. Again, I don’t think that’s the mainstream view, but there are still people who are saying, “See, Kurzweil was right all those years ago.”
I think that he misses some other things. Intelligence isn’t just about computation. It’s not just, “Do you have the same amount of computing power as the neurons in the human brain?” It’s also, “Do we understand the algorithms? Can we replicate intelligence?” And we haven’t fully gotten there yet. I think the technologies that have been developed in the last few years are really surprising and shocking in what they can do. But there are also some really basic things that they can’t do – that people can do and make.
Also, I tend to discount this perspective, but I think at least it can’t be completely dismissed: there’s the possibility that artificial-intelligence technologies, if not checked, could pose a sort of serious existential threat to humanity – and on the kind of scale, I think, that is much less likely than existential threats from things like nuclear weapons and pandemics and climate change. But there are people who say this is something we should be worried about.
So that’s another difference – nobody was saying that automobiles were going to destroy the world; some people are saying that about artificial intelligence. And that changes the discourse around it a little bit.
Sanctity and Irreplaceability
Neil Blumofe: I want to jump in with a text from a selection by a very revered scholar in the Jewish world, Rabbi Dr. Jonathan Sacks, who passed away a couple of years ago, In Covenant and Conversation: Genesis: The Book of the Beginnings, which was published in 2009. I brought this just as a conversation starter.
“[Names are what gives] human life its dignity and preciousness. Without it, we would not know love – for love in its primary sense is always directed to a person. One who truly loves does not love abstractly… Love lives in particularity. This is what also gives human love its pathos and vulnerability. We know that like us, our beloved will eventually grow old and die, and that they can never be replaced. If we know we would never die, we would need no intimations of eternity. But because we know we will one day die, one of the great things we can experience is the moment beyond time (the one we know we will never forget) when two souls touch and between them form a bridge over the abyss of mortality. That is the meaning of the verse, love is as strong as death, its passion as unyielding as the grave (Song of Songs 8:6). On another level, this is what gives human life its sanctity. A single life, teach the sages, is like a universe. However lifelike robots may one day become, there will always be this fundamental difference between a machine and a person. Machines can be replaced. Persons cannot.” (2009)
Peter Stone: I think the sentiment resonates with me a lot, but the reasoning behind it is maybe a little bit questionable at this time, and he may not have written the same thing if he wrote it today. At that time, in 2009, I was very used to the notion that a computer program is something that can be saved, written to a disk, and reloaded. You can do that when you’re playing a video game, too. You can get to a point, you can save it, go to bed and reload it, start again, something bad happens, you can go back to the place you would save – which is how we think of computer programs.
But the generation of computer AI programs that has come about in the last five years or so – generative artificial intelligence programs – are moving much more towards that irreplaceability. Now you can have a conversation with the chatbot and it stores the state, but if you erase it and start over and start that same conversation, it’s going to go differently. We’re now in a position where computer programs can also be thought of as irreplaceable. Especially if you’ve spent a lot of time interacting with one, it’s learned something about you. It becomes your personal assistant in some way. You might get to that same sense of irreplicability.
Then, at the same time – I think it’s not all the way there, but people talk about cloning technologies that, on the biological side, make certain animals arguably more replaceable. So I totally agree that love should be directed towards a person, not towards a robot. But I think we need to find a different way to justify and explain that than was the case 50 years ago.
Neil Blumofe: I think it’s a great challenge. And I would say that this, too, that mortality, or the issue of living or not, he says, gives human life its sanctity – which is a very strong theological word. This idea that perhaps machines are not as replaceable as we think has us not only question why we explain that love needs to be directed towards a person, but even the language that we use for holiness, or kedusha. This idea of sanctity wouldn’t make sense today when speaking about a computer program that cannot be replaced.
So it’s interesting to me, as well, that this doesn’t just call into question how we live our life and adjust to a new gadget, but rather how we, in fact, bring our sense of ethics, truly, but our sense of value to the moment that we’re in. This isn’t like a “get off my lawn” speech, but those who are connected or addicted to social media have much more trouble in the real world, and to a degree, and I think that giving sanctity to the machine versus to the person is something you helped me notice.
Peter Stone: Yeah. There’s now debate: can computer programs be sentient? What does it mean to be sentient? There was the Google employee, Eric Demaine, who was fired for complaining that “you can’t turn this off. It’s sentient. Here’s my evidence for it.” That kind of perspective is not the mainstream by any stretch in the AI community, but there are definitely people who are saying, “I’m interacting with a machine in a way that’s somehow functionally indistinguishable from the way I interact with people, and therefore I should ascribe everything I do to a person to this machine.”
I got sort of approached out of the blue recently by somebody who said, “We’re forming a startup company to use generative AI programs like ChatGPT or Gemini or Lala to play the role of a therapist, and then we’ll use this to replace human therapists.” I said, “I want nothing to do with that. Leave me out of this entirely. I think that’s a horrible idea.”
In the 2016 report of the 100-year study on intelligence, we identified that caregiving is something that we do not expect artificial intelligence to sort of replace as a role in the near term – and this returned as a theme in the 2021 report. There’s something very human and intimate about the nature of caring for another person. There may be AI programs that can make the act of caregiving easier or more efficient, but at the end of the day, this is a deeply human act. So I didn’t want anything to do with this AI therapist project. That’s an example of something that I hope doesn’t take off. It’s not a matter of how good it is most of the time – this is something I don’t think we have any business trying to fully automate.
Neil Blumofe: In my world, there is discussion of the practice of preserving with willingness the voice of someone who has passed on, and then hold it for bereavement purposes – to continue a relationship with someone who’s dead, based on the generative aspects of knowing who that person is. So on one hand, it keeps someone alive, whatever that looks like, but on the other hand, it may impede someone’s grief as well. So, I think that’s an interesting point. It may not be as ethically fraught, but maybe it is. So, I’m sure you’ve heard about that and thought about that.
Peter Stone: Yeah. And this goes to that question from Rabbi Sacks, also, of “What does it mean? Is it replaceable?” To create that program, you need a sample of that person’s voice to dialogue with. And then if that were to be unplugged or taken away, that would feel like a loss – and it is representing a person that you had a relationship with. And so, yeah, I think the ethical question would be like, can you do this without the person’s permission? It’s more of a mental health question than a bereavement question of, “is this a good thing to do?”
I think the fact that you can generate things in other people’s voices can be used for very good purposes. It’s an open question whether it’s good or bad for it to help in the bereavement process.
Neil Blumofe: I do believe using it in the bereavement process is a good thing, but in the same way that, like, looking at pictures is a good thing – if you lose yourself in the pictures of yesterday, I think that there’s an issue with that, and I would think the same, at least at this point, from losing oneself in a conversation with someone who’s dead.
Replaceability and Human Uniqueness
Neil Blumofe: I’d like to look at this passage, from a blog called Jewish Action. There’s a question that Rabbi Dr. Ari V. Zivotofsky asks Rabbi Yosef Zvi Rimon. He says, “Can AI help us perform mitzvot? By the way of the speakers?” I’m not so familiar with these quotes, but I thought it was interesting.
Q: Can AI help us perform mitzvot? A: …The actual shechitah must be done by a God-fearing Jew. The Shulchan Aruch rules that the shechitah is only acceptable if it is done by a human being. It also explicitly states that shechitah may not be done by a machine that is not directly operated by a human.
Can AI help us perform mitzvot? Wouldn’t that be great? “Hey, AI, do all 613,” right? So according to the Shulchan Aruch, the killing of the animal to make it kosher must be done by a God-fearing Jew. It also rules that the shechitah, the ritual slaughter, is only acceptable if it is done by a human being – and explicitly states that shechitah may not be done by a machine that is not directly operated by a human.
Creativity
Peter Stone: What can artificial intelligence bring to the table to help, maybe possibly automate, the mundane parts of the creative process, and allow creators to get into the flow or spend more time in the flow and be more creative in various ways?
Sony has a very top-notch AI ethics team. If you are going to create artificial intelligence for creators, there are big questions: are you training with data that people want it to be trained on? Can we just grab people’s pictures from the Internet, or do we need their permission? And so we have a fantastic AI ethics team there. There’s a robotics group, there’s a group that’s contributing to the imaging and sensing business of Sony, where they’re developing great camera equipment where AI can help photographers.
Neil Blumofe: Sony AI helping complement artists or creators is more reassuring than the people who are sending me AI-composed jazz songs, which are hard to listen to. Or with my son, my oldest, who’s a poet, would be heartbroken if any AI program could just “outpoet” a poet. So I’d much prefer the human connection and knowing that the Shulchan Aruch knows how ineffable and important taking a life is.
I thought that that would be an interesting thought about how that could be parallel in the work that you do.
Peter Stone: There are several different things that can be said, but like I said, at Sony AI, we do try to augment human creativity. I was just at a symposium in London at the Royal Society, where there was a pop artist who’s been working with technology for 20 years in the creation of his songs. He did a demonstration where he, along with a producer, spent about 45 minutes creating in front of us, the audience – taking cues from us for suggestions for lyrics, and voting on what the background track should be. He created a really good-sounding song with his voice singing all the words, even though he had just sung a few chords or notes. The AI had generated his voice singing the lyrics that were generated.
He said that in 45 minutes, what was produced was as good a quality as what he used to have to spend three to four weeks in the studio doing. And furthermore, he thought his seven-year-old daughter could do most of the things that he had just done in front of us. And this was presenting an existential crisis for him. He’s said, “I’ve spent my whole life trying to become an expert. I’ve put in my 20,000 hours to become an expert in this thing that is now becoming a commodity.” And so how does he, now that anybody can do that using generative AI tools?
And so he asked the question to the audience, “If Dune 3 were created by artificial intelligence with no human involvement, but when you watched it, you experienced the same range of emotions – you cried, you laughed. Would you care that it wasn’t produced by a person?” So we’re thinking about this. What is it that makes the arts? And I think there’s a big part of it that is the human story behind who created it and the experience that they came from. So even if an AI program generates something just as good, it may not be the same. But I think there is this upheaval coming, and we’ll have to see how that goes.
The fact that generative AI seems so easy to use has also caused an uproar. Teachers have been all up in arms about with the introduction of ChatGPT about how, “Oh, all of a sudden students are going to cheat in their classes,” as if no student ever cheated before. I think AI makes it a little easier, but it was unethical to cheat when we were students, and now the fact that it’s easier doesn’t make it more or less unethical. This is an overused analogy, but that’s like saying that the calculators were bad because they allowed students to cheat on their math. It’s a new technological tool.
I think there are going to be ethical uses of the technology and unethical uses. We need to educate all of our students and citizens about what the appropriate ways to use them are. And I don’t think ease of use has any implication about whether it’s ethical.
Sony is a big IP company that has a lot of rights to songs and music. What intellectual property can you use when training an AI model? Sony AI is being very careful to try to sort of be on the right side of this question, but then it’s hard to define what is the right side.
One of the fuels of generative artificial intelligence models is the data. And a lot of people are training data based on just “the web,” whereas at Sony AI, we have a project to try to gather data that can be used for training generative models. That is, data that we have rights to, permission to – that people have been paid for, compensated for, and then trying to put data sets out there that would be more ethical to train with.
With regards to the shechita, you know, it says, the quote is that the machine has to be directly operated by a human. There are starting to be gray lines – is it sufficient, then, for a person to press a button and then the machine does the rest? What does it mean for a human to be directly controlling it?
There’s an article in the New York Times – today, actually – about how in the Ukraine-Russia war, people are developing killer robots. There are people in my field who have long been calling for making international treaties that any kind of robot that could take a human life would have to be under the control of a person. But it’s starting to blur the line; what does it need to be under control of a person? One of the themes of the article in today’s paper was, this is existential for Ukraine. They’re running out of people. Regardless of whether this is ethical or not, they’re going down this path – and then the only response becomes other countries going down this path. So I think that’s a scary aspect.
At Sony, there’s an ethical AI team led by Alice John, who’s fantastic. She had both a law degree and is an expert in AI, and thinking very much about these very questions. But there are no settled answers.
Ethics and the Good Systems Initiative
And that’s why our initiative at UT Austin is purposely called Good Systems, not Ethical Systems – because there could be things that are completely ethical, but still not good in the sense of “you should do them.” You can do them, they’ll be ethical to do, but maybe you shouldn’t. I think this might be one of those things. I’m not a mental health expert. You, Neil, probably are much more qualified to say than I am whether this is good for the bereavement process, but it’s certainly something that’s possible to do, and it’s a pertinent question: should you do it?
The Good Systems Initiative actually grew out of the 100-year study on AI that I have been a leader in. There was a call by the Vice President of Research at UT Austin, Dan Jaffe, for faculty to think of some “grand challenge” problems that will come up over the coming decades. There have been three “grand challenges” launched as a part of it. One called Community Full Health, sort of a holistic medicine grand challenge, one on. It’s called Planet Texas 2050, on climate change, and then Good Systems, which is about what are going to be the social implications of artificial intelligence.
And it’s not just there to sort of comment on or study where the field is right now, which is what sort of the AI 100 reports were about, but actually to define what it means for an AI system to be good, to figure out ways to evaluate that, and to develop best practices for moving in that direction.
A common theme between AI 100 and Good Systems is that they’re cross-cutting organizations not just populated by technologists like myself, but also collaborate closely with humanists, social scientists, political scientists, ethicists, legal experts, and others. The one I’m most involved in is called “living and working with robots,” which is studying what it would be like if we put robots in our communities. How should they add to enhance our experience?
I think this is an example of something that would not have been really appropriate to try to do, or there wouldn’t have been the groundswell of interest – when I was first getting into the field. But now, because of the way the field has advanced and the potential implications on society, it’s very pertinent. I think it’s fantastic that we’re getting these cross-disciplinary dialogues – not just fantastic, I think it’s essential.
Embodying AI
The generative AI systems that are most famous today, that everyone’s playing with, are disembodied intelligence. And that’s one of the reasons that I’m director of robotics at UT and have always worked in robotics, is that I think intelligence can’t be separated from embodiment. Intelligent robotics is about trying to make these AI systems embodied in some way. Many people have said, “I need a robot to clean my apartment, not to write my poetry.” And the kinds of things, the kinds of intelligence, that come from writing a poem or writing a song from a disembodied program are very different than the kind of intelligence that’s needed to hold laundry or put dishes away in your dishwasher, which turn out to be very easily tasks for people and are still beyond the capabilities of AI.
I’ve been involved in an organization called RoboCup, which has a goal by the year 2050 to create robots that can beat the best World Cup soccer team – but it also has another sort of “league” called RoboCup At Home, where it’s people trying to create general-purpose service robots that could put away your groceries, or put dishes in the dishwasher, or fold your laundry. So, we’re working on putting AI in a robot that can move around the world and pick things up and manipulate them. This is a very different type of intelligence and still, in many ways, way behind what people can do – and still a different embodiment.
We can’t completely separate intelligence from embodiment. And that’s the premise of my work in intelligent robotics. People used to say, “Oh, intelligence is playing chess – if you can beat people at chess, then you’re intelligent.” But then it turned out that we could beat people at chess with an algorithm that can’t do many other things. Part of the reason is that there is actually no generally accepted definition of artificial intelligence, partly because there’s no generally accepted definition of intelligence.
So what does artificial intelligence even mean? The definition that we wrote in the AI 100 study is: “the science and practice of creating machines that do the kinds of things people do with their brains, bodies and nervous systems, but not necessarily in the same way.” That’s one definition of artificial intelligence, but the connection to intelligence and bodies is, I think, an important theme of defining it.
The things that people think are difficult for people are actually easy for machines and vice-versa. Hans Moravec called this the “AI Paradox.” When you think about the evolution of life on this planet, the amount of time it took to get to organisms that can just walk and navigate and avoid obstacles, that’s like millions and millions of years. And the time to get from animals that could do that to animals that can play chess is a minuscule amount of time in the context of evolution. There’s a famous article by Rodney Brooks called “Elephants Don’t Play Chess.” Most of the intelligence we have in people is also found in elephants, at least from the evolutionary time-scale perspective.
The Alignment Problem
In the AI field, another component of developing intelligence is incentivizing and rewarding machines in a way that accounts for ethical considerations and even societal norms. This is called The Alignment Problem. The sort of famous example is that if you had a self-learning computer and you gave it the task of trying to make paper clips as efficiently as possible – because you own a paperclip factory, and you want the AI system to help optimize it – that it might decide that the best way to make many, many paper clips is to get all the resources of the world, and by the way, people consume resources: “So first I’d better kill all the people so I can get all the resources, so I can make more paper clips.”
If you’re using machine learning systems, how do you define a reward function so that if you optimize it, you get what you want? Then there are people who criticize the study of alignment: “Well, aligned with who? Whose preferences? Who gets to decide what you’re aligned with?”
I’ve actually been trying to popularize a slight shift of terminology. Rather than calling it the alignment problem, I’ve been trying to call it the alignability problem. We shouldn’t be trying to create aligned computer programs, we should be trying to create alignable computer programs – so that it’s not the AI programmer in the basement who decides what objective the program is trying to achieve, but rather that can be decided by the end user – whether it’s an individual or a democratic process an autocratic process or something else. If there are autonomous cars, for example, maybe the way they’ll drive in Austin should be different from the way they’d drive in Stockholm or the way they’d drive in Tokyo. And that should be decided not by the Fords or the Hondas or the Teslas, but by the people who are using them in that society. So the autonomous cars should be alignable, so that people can give those different objectives. But it’s an active area of study right now, and I’m doing some research in this space as well, especially in reinforcement learning: how do you create reward functions where if you optimize them, they give you what you want, what you meant – not what you literally said?
And I think that’s the theme here as well, is that there’s a bit of an AI hype going on, that expectations are way beyond reality, and that we should settle down, relax, and try to get a more realistic perspective on it.
AI Backlash
There has been a long history of technological upheavals. And in many ways, I see AI being part of a long history of technological upheavals. It’s like the industrial revolution. It’s like social media. It’s like mobile phones. It changes lots of different things. Changes the way we respond to technology. When I went on my road trip after college, I had maps that I folded out and unfolded and refolded – and now my son’s on a road trip after college, and just looking at his phone.
And it is changing the workforce – there are people who are nervous about losing their jobs, just like how there was with other technologies. Of course, there are new jobs being created as well. I think there are people who are scared of the upheaval that it will cause. AI is like many other technologies: there will be professions that are no longer as relevant as they used to be, and that will undercut the livelihood of some people and cause backlash.
On the other hand, historically, new jobs have been created. Technologies have made the economy more robust and created more productivity in society as a whole. As long as the pie gets bigger, and as long as it can get divided in a fair way, then you tend to avoid the backlash. There’s a very real danger that there’ll be a concentration of wealth. So many people are saying that’s already happening as a result of this, and that sometimes is the fodder for revolution.
And so, I think this is, again, there’s, this is more of an economics question, but there are AI economists. There’s a colleague of mine, Erik Brynjolfsson, who has been involved in the AI 100 study with me, who thinks about these questions as well. I think these are very pertinent questions.
I feel strongly that we need to find a way of making sure that the increased productivity that we get from artificial intelligence isn’t all concentrated into a few hands. In the boom of AI in the 1980s, most of the advances came from academia, but many or most of the advances now are coming from a small number of private companies that are the only ones that have the resources to train the models. And it’s sort of a race right now for each of them. There are potential problems with that. I have some concerns about that as well.
There are also some movements to have publicly-funded generative AI models, sort of like – maybe you would need more resources than any one country would contribute, but sort of like the particle accelerator in Europe, the CERN particle accelerator, that’s like a consortium of many countries to create a publicly funded scientific resource. There are people who are sort of moving towards having a collectively-funded resource of an AI model that could be at the same scale that private companies are able to create. I think that’s in many ways something that’s worth striving for and would be good.
Ongoing Challenges
Earlier, we talked about computer vision, and how for 30 or 40 years that was the biggest challenge in the industry to overcome. We used to think that if you could write a computer program that could do the things that computer vision programs do now, you would have to solve the entire intelligence problem. And now it turns out that you can solve it with methods that don’t do everything.
We’ve talked a little bit about some of the biggest open challenges with AI – embodiment, robotics, manipulation, being able to have a model of the world – but I think that’s partly tied up with one of the other really big missing pieces right now, which is causal reasoning. The computer models that are trained right now are basically correlation machines. They’re basically trained to predict the next word, but they don’t “understand” (whatever that means) what it is they’re describing. There’s no model of “the pen is on the table. And if I push the pen off the table, that it will fall because of gravity. What’s causing this?”
And so there’s sort of a sub-community in artificial intelligence that’s working on ways to do causal modeling. But it’s pretty easy to mess up, to get failure cases, from the current AI systems that are really good at computer vision if you give them tasks that require that. In fact, there’s a famous one that’s sort of circulating on social media. They gave a generative AI model the task of creating a video of gymnasts. If you’ve seen it, it’s sort of grotesque – the person is doing flips, but it has no model of the human body. So there are motions that we can look at and say, “That’s completely impossible, that doesn’t look like a human.” And there’s extra limbs. And you would need to have a sort of a physical model of the world to understand that that doesn’t look right, and that’s just not there in AI systems right now. And I don’t think we don’t know how to do that.
It’s been really surprising what you can do by just doing essentially what people sometimes describe as “auto-completion on steroids,” because that’s what these models are trained to do. They just predict the next word or predict, you know, create an image that sort of looks plausible, but to go that next step of being able to have an understanding of why something’s working, what causes something to happen – that’s not embedded in any of the AIs.
I think there’s a quote in the last AI 100 study that says, “the closer we get to sort of human level intelligence, the farther it seems away, because we realize what the gaps are still in the current models and what are the things that we would really need fundamental breakthroughs still to need to achieve.” I think it still can’t be dismissed that there is this sort of inflection point coming because of the rise of computation power to sort of unprecedented levels, and that we’re getting to some tipping point – but that’s not what it feels like from where I stand. Certainly the musician that I saw present was fascinated that he could just sing a few notes and then the AI system could sing the song in his voice. He said, “And it sounded better than if I had sung it live. It’s in tune.”
0 Comments