AI and Surveillance Capitalism
AI and Surveillance Capitalism the Rev. Edmund Robinson Unitarian Universalist Meeting House April 29, 2018
Some of us may be inclined to say, “what’s the big deal about artificial intelligence? I love being able to look anything up I want on Google, I love that I can ask Siri, the robotic voice in my smartphone, to find the nearest dry-cleaner and get results right away. I love that I can feed in a sentence in Serbo-Croatian and have it translated into English for me. Someday soon AI may drive my car. What’s not to like about AI?” Here in 2018, the developments in computer science which go under the name Artificial Intelligence, or AI, have become big business. Most of the giant corporations such as Google, Amazon, Spotify, Facebook are investing heavily in AI. Elon Musk, CEO of Tesla, warns against AI and urges that it be regulated. But what is AI? It’s a wide variety of systems which try in one way or another, to duplicate some of the capabilities of the human mind. And in the attempt to shove some fo our mental work off onto machines, we have learned a lot abut the way the human mind works. For example, human babies at a very early age learn to recognize faces, and this ability sits naturally in our brains throughout our lives. When you walk into a group of people whom you know, such as this Meeting House, you will effortlessly be able to put faces to names. But it turns out to be very hard to teach a machine to do this basic task. It is also hard to get a machine to recognize that a person seen from the side is the same person who was seen a minute ago from the front. The kind of adjustments that humans learn effortlessly and naturally over the course of growing up can be very daunting to program into a machine. Now the classic scare value of Artificial Intelligence is the idea of robots run amok, artificial beings who turn on their creators and kill them. We see this in Mary Shelley’s Frankenstein in the nineteenth century, probably the first science fiction novel. We see it in the 1920 Czech play R.U.R. by Karel Capek, which gave us the term robot. We see it in the film 2001: A Space Odyssey by Stanley Kubrick and Arthur C. Clarke, which came out 50 years ago this month. We see it in the Blade Runner films and more recently in the films Her and Ex Machina and in the HBO series Westworld. It goes back in legend to the myth of Pygmalion or Pinocchio or the Golem. Creatures made by humans somehow obtain sentience and the ability to think and feel for themselves and then turn on and harm the humans who gave them life. What I want to say today is that as interesting as the robots run amok scenario is, that scenario does not encompass all we have to fear from Artificial Intelligence. We are a long way from creating machines that actually think and actually feel and actually have wills in ways that humans do. But artificial intelligence has led to other problems than a robot uprising, problems that are here with us today. Seven years ago, I preached a sermon here called “Is Google God,” because I saw a real similarity between what Google has come to know about me and the view of God set forth in the 139 Psalm, which I read just now. James Luther Adams, one of the leading UU theologians of the twentieth century, described God as located at the intersection of “the intimate and the ultimate.” It’s a grand conception: a being who sets the galaxies in order, who knows where the black holes are and propagates the gravity waves also knows each of us well enough that he knows our lies and evasions, deepest fears and fondest wishes. O LORD, you have searched me and known me. 2 You know when I sit down and when I rise up; you discern my thoughts from far away. Google can also discern our thoughts from far away, for it has a record that last night at 7:30 pm I looked up an article about Augustine and free will. There may be no human being who is aware of that search, one among billions taking place at the time, but with the right access, a human being could find that search of mine. Well, you say, so what? If no one is actually watching, what does it matter that there is a data trail which evidences the wandering interests of my mind as I interact with Google? The answer is it doesn’t in the individual case, but when you add up billions of us, you get the greatest advertising machine ever created. And in politics, as the Cambridge Analytica scandal has shown, the big data mined through Artificial Intelligence gets you the 2016 elections in America and the Brexit vote in England. In addition to the usual sources of liberal critique, I subscribe to a wonderful podcast called Radio Open Source with the cultural and political journalist Chris Lydon. What is a podcast? It is like a radio show you can listen to any time you want. I have been a fan of Lydon’s since I first moved to Boston in the 1990s. The day that Mark Zuckerberg appeared before Congressional committees, Lydon did a program on surveillance capitalism, which featured a scholar at Harvard Business School named Shoshanna Zuboff, who coined that challenging term. As professor Zuboff described it, surveillance capitalism is the current system by which our data, our choices, our interests, our political reflexes, becomes the product of commerce, becomes that which is bought and sold in the marketplace. She says it started at Google in the early 2000s, where the company’s old business model had come under criticism, and they were at a crisis point. Here is how she described this new kind of capitalism as she was interviewed by Chris Lydon: Zuboff : Capitalism has products and customers, capitalism has reciprocities with its populations. It needs people to be customers and it needs people to be employees. Surveillance capitalism doesn’t have that. We’re in a different zone now. One of the problems with recognizing surveillance capitalism, and I’ll describe some of the key mechanisms ... Lydon: I want to know who done it. Zuboff: OK, surveillance capitalism was born at Google, it was discovered and invented at Google 2001, 2002. It was discovered under a state of emergency at Google, as the dot-com bust was happening. All these nice little startups were imploding, and even Google’s investors were threatening to withdraw their support. And there was a turning point back then where the company decided to use its own caches of data about its users. Data that was sitting in its servers that had really been ignored up until that point. It was called data exhaust. Because of some accidental things, more or less, that happened at the company, there were a few insights. You do a search, every time you’re interacting with the computer in any way, you’re leaving these little collateral traces. Lydon: right, little cloud out of your iPhone, everywhere Zuboff: That’s right. So, anyway, some people started piecing together the fact that these forgotten pieces of data actually had some significant predictive value to them. And under the pressure of this state of emergency, it was the founders of Google who decided that we’re going to change the way that we sell ads. So instead of the advertisers picking their keywords and we charge them for space by the keyword, what’s going to happen now is that we’re going to turn to these forgotten sources of data, and we are going to use those to do our fine-tuned predictions with our state-of-the-art computer science. And those predictions are going to tell us which ads are more likely to get a click-through on which page. “...The breakthrough was realizing that they could gather behavioral data that was more than what they needed to improve products and services. That extra behavioral data, that began with this insight from data exhaust, that’s what I call behavioral surplus. That surplus was set into their manufacturing processes. Back in those days at the beginning, it was called computer science. And then a few years later, they began calling it artificial intelligence, and now it’s called machine intelligence, but it’s the same structure. The behavioral surplus was put into the manufacturing process to fabricate what I call prediction products. We’re getting predictions about our behavior. And the first group of people who were really interested in those predictions happened to be online advertisers. “That’s how it all came together. And Google created these special marketplaces which I call behavioral futures marketplaces. They are marketplaces where people are trading on what we are going to do now, soon and later.” As I am understanding it, what Professor Zuboff is calling surveillance capitalism is using these forgotten bits of data, the trails we each make when we pursue our random curiosity, and massively extracting and compiling them and then loosing on them the algorithms of artificial intelligence to make predictions about our future behavior in the marketplace. Predictions, accurate predictions about market behavior are immensely valuable. If I had a copy of the June 1 stock market page and a thousand dollars today, I could be sitting very pretty when June rolled around. The rest of the story is that this predictive system spread from Google to Facebook and the other giants of internet commerce. In the course of his interview with Professor Zuboff, Chris Lydon raised a very interesting question which he did not end up pursuing, but which fascinates me: what does this have to do with free will? If Artificial Intelligence can crunch big data to make fairly accurate predictions about our individual behavior in the marketplace, is free will a delusion? It is an ancient concern. Psalm 139 not only asserts that God knows our inmost parts, but also knows our every thought and word before we say them. “Even before a word is on my tongue, O LORD, you know it completely. 5 You hem me in, behind and before, and lay your hand upon me. “...In your book were written all the days that were formed for me, when none of them as yet existed.” Traditionally God is called Almighty because of three traits: omnipotence, or unlimited power; omniscience, or unlimited knowledge, and omnibenevolence, or unlimited goodness. Of course, there is a problem with this formulation, the problem of evil. If God is all powerful and all good, why doesn’t he/she eliminate evil? St. Augustine’s answer was that God had to create humans with free will, and this free will allowed humans to do evil, but God was not responsible for the evil that humans did. Thus God told Adam in the Garden of Eden not to eat the fruit of the tree of the knowledge of Good and Evil, and Eve and Adam ate anyway. So free will was an essential part of Augustine’s explanation of how there could be evil in the world. However, the Roman pagan thinker Cicero, two centuries before Augustine, had reasoned that God could not know the future. Because if God knew the future, then there would be no free will. If God already knows that I will have a plate of fried oysters for lunch today, the die is already cast, and I don’t have any freedom not to have them. Augustine took on Cicero’s argument, and basically tried to have it both ways. Yes God can see the future and in that sense everything is predestined, but you still have human free will as the reason that God is not responsible for evil. I don’t know what Cicero or Augustine would have made of the ability of Artificial Intelligence systems in the twenty-first century to predict the behavior of internet users. It gives me a creepy feeling when I mention in an e-mail that I am thinking of buying some curtains and five minutes later ads for curtains start rolling through my sidebar. It gives me an even more creepy feeling when someone thinks they’ve figured out who I’m going to vote for. Now this is not just a worry about the 2016 election and Cambridge Analytica and the Russian trolls. The Obama Campaigns of 2008 and 2012 used Artificial Intelligence to try to predict who the likely voters were and which ones would support their candidates. That is just an outgrowth of what campaigns have always done, but, like internet advertising, it’s much more effective because it’s targeted to the individual voter. But what has emerged from the analysis of the 2016 campaign shows something quite different. The DNC was hacked and its e-mails selectively published. But there was also a campaign on social media from Russian “troll farms” to create completely bogus news stories and feed them to vulnerable voters who would then circulate them. Factually, it appears, we believe what we are pre-set to believe. Way back in the eighteenth century. the Enlightenment gave us a notion that reason would lead to truth in a free marketplace of ideas if debate was open and transparent. Liberal arts education is bottomed on this marketplace of ideas. But what we’re seeing with this AI surveillance capitalism is that it’s not a free marketplace of ideas, we just all jump to believe whatever meme will make us feel good. Mobile devices that most of us carry in our pockets have honed this to an art. There is actually a specialty called Persuasive Design which studies how your brain pursues the things it desires, and designs features of your so-called smartphone to keep you hooked, to keep you checking every ten minutes. Have you checked yours since the service began? Take the dot-dot-dot of text messaging. Text messaging has become for many of us a preferred means of communicating with certain people, because, unlike a voice call, the other person doesn’t have to be actually available to talk at the exact moment you want to contact her. But the phone company wants your brain to be in something of the state of interest of a live conversation, and so when you’ve sent your message, you can look for three little dots on the screen which indicate that the person on the other end is typing a reply. This keeps your attention hooked on this conversation, and your brain will eventually be delighted when the other person types back, “fine, I’ll see you ate 12:30 for lunch.” So artificial intelligence is not just stand-alone robots who are taking over our jobs, though that is a part of it. Artificial intelligence, at a more sinister level, is a way for other humans, motivated by profit or political aims, to interface with us and manipulate our desires and beliefs to get us to spend our money the way they want or to vote the way they want. Let me wrap up here by getting back to the idea of the God at the intersection of the intimate and the ultimate. We are all hungry for intimacy in our world. Many of us have been separated from those we have loved most by geography, by death, by estrangement, and we cling to the connection that social media and smartphones and our other devices can offer to those with whom we still can communicate. One role of AI is in making these connections, but have these giant Internet companies created, in effect, false gods? Listen again to Psalm 139 and ask yourself how much of what the Psalmist described about God could also today describe Google, Facebook, Twitter, Instagram or the messaging capacity of your smartphone? O LORD, you have searched me and known me. 2 You know when I sit down and when I rise up; you discern my thoughts from far away. 3 You search out my path and my lying down, and are acquainted with all my ways. 4 Even before a word is on my tongue, O LORD, you know it completely. 5 You hem me in, behind and before, and lay your hand upon me. 6 Such knowledge is too wonderful for me; it is so high that I cannot attain it. 7 Where can I go from your spirit? Or where can I flee from your presence? 8 If I ascend to heaven, you are there; if I make my bed in Sheol, you are there. 9 If I take the wings of the morning and settle at the farthest limits of the sea, 10 even there your hand shall lead me, and your right hand shall hold me fast. 11 If I say, "Surely the darkness shall cover me, and the light around me become night," 12 even the darkness is not dark to you; the night is as bright as the day, for darkness is as light to you. 13 For it was you who formed my inward parts; you knit me together in my mother's womb. 14 I praise you, for I am fearfully and wonderfully made. Wonderful are your works; that I know very well. 15 My frame was not hidden from you, when I was being made in secret, intricately woven in the depths of the earth. 16 Your eyes beheld my unformed substance. In your book were written all the days that were formed for me, when none of them as yet existed. 17 How weighty to me are your thoughts, O God! How vast is the sum of them! 18 I try to count them — they are more than the sand; I come to the end — I am still with you.” Are you still with me? Amen!