For most the idea that we’re simulated is basically a joke. But sometimes, and perhaps of late increasingly, a situation can seem too extreme to fit the facts that we take to be true. Maybe we really are in the Matrix? Maybe this is a test. At the edge of what can be emotionally or mentally accommodated, the simulation hypothesis seems less like a joke and more like a symptom, or the expression of a feeling that something important has become unhinged.
In his recent book, Reality+: Virtual Worlds and the Problems of Philosophy, philosopher David Chalmers argues for the relevance of the simulation hypothesis. The best of simulation technology is advancing quickly. It’s more difficult than ever to distinguish the virtual from the ordinary. Chalmers estimates that we’ll be able to engineer indistinguishable virtual realities within a century, which makes it more plausible that we already have. We may already be simulated, sim-living, say, in one of the many simulations that Elon Musk is running to counterfactually model his rise as sim-god. The fact that a metaverse is being built seems to suggest that we’re already in one. How would we know? If simulators are more likely to simulate people who think about simulations, then an interest in Reality+ may be a sim sign. If things seem absurd or inexplicable with some regularity, and if no one else seems to notice or pay it much mind, this may be a sim sign.
“Virtual reality is genuine reality,” says Reality+. The virtual worlds we create are just as real as our world, which, for all we know, may be a virtual world. It’s not only the case that VR is real reality, according to Chalmers. It’s also the case that our lives in VR worlds can be in principle just as good, if not better, than our lives in ordinary reality.
It’s true that the hardware is bulky now. Haptics, smell and taste are hard to get right. But these are technical hurdles. Eventually VR could afford us many more experiences than the ordinary world. An engineer recently told me that he had always wanted to be a monk. Now in VR he could be a monk. Over time we could come to prefer our virtual lives. “As our minds speed up in the technological future, physical reality may come to seem unbearably slow.” Virtual space is effectively unlimited. “Everyone can have a virtual mansion or even a virtual planet.” “Everyone can have a personal idyllic virtual island if they choose.” Along with these benefits, VR poses some novel ethical challenges, Chalmers concedes. He devotes one of his three chapters on value to the question of the moral worth of simulated beings. Do simulated lives matter? He quotes MLK. The arc of the universe bends toward justice for sims. This seems to be sincere.
Many will say that Chalmers is out of touch. It’s easy to say that he’s too in touch with big tech. “If we create simulated worlds ourselves,” Chalmers writes, “we’ll be the gods of those worlds.” But Chalmers seems out of touch at least partly because of his contact with real technology. The idea that we’re simulated does seem more persuasive as our simulations become more lifelike.
While the idea that our reality may be computer-generated was popularized in the early 2000s, the form of the idea is very old. The simulation hypothesis is a skeptical hypothesis, a variation on the theme that we can’t be sure about things that seem certain. For all we know, the world, its objects and our bodies and minds may be an illusion. This may be a dream. You may be a passing thought in the mind of God. Skeptical questions are interesting, at least historically, for being both devastating and irrelevant. The questions are devastating insofar as they’re basic but unanswerable. The questions are irrelevant insofar as most people don’t notice them, or care. Certain forms of skepticism seem incompatible with care. Whether a handed person actually has hands is not a relevant concern, for example, when someone asks for help.
What’s tone-deaf or weird about Reality+, then, has less to do with the person of David Chalmers, analytic philosopher, founding member of the Association for the Scientific Study of Consciousness, than with the fact that the skeptical concerns his book raises are now also urgent concerns—like concerns over what we owe each other under the real conditions of our times.
The technical term for VR and AR (augmented reality) devices is “XR,” or “extended reality.” I recently attended an XR conference. Its keynote speaker suggested that people—and by people, perhaps meaning Americans—spend on average eight hours per day on digital media. The XR community would like to improve those hours. Ideally, that is, everyone is eventually uncrumpled from under their computer screen. While even the conference attendees struggled to define “metaverse,” or “metaverses,” there is a general sense that the way we interact with digital media is transforming, and for the better. Soon making a change in the digital world will not require three clicks or one click but zero clicks. Buying something online may be a matter of pointing or thinking. Engineers are working toward a greater integration of the physical and the digital: more intuitive interfaces between the body and the digital and augmented worlds.
One hears jokes at such a conference about how many holes will be needed to be drilled to fit the brain-computer interface. One hears the usual clichés. We will go beyond reality, our collective creativity will be unleashed. “The digital future is ours.” One wonders at the jokes and objects to the clichés, and thinks of how the “our” in “our collective creativity” refers simply to the people in the room, who have the resources to decide what our digital future will look like, resources that include money, intelligence and infrastructure as well as imagination. They are imaginatively rich. Outside of Silicon Valley, many assume that a future of convincing XR is either the plot of a Black Mirror episode or fantasy, in the sense of being a waste of time to think much about. When I talk to friends who don’t live in Palo Alto, they suggest that I have been here too long. I hear things like, you have drunk the Kool-Aid. No one wants this, they say. No one will use these devices.
Meanwhile, a lab at Stanford has already manufactured an effective retinal implant. The clunkiness of existing VR headsets is beside the point. How our lives will become more digital is undecided. That they will become more digital seems to me basically inevitable. To gesture to Meta’s slumping stock price in order to clinch the argument for VR’s irrelevance is to draw attention away from the question of who’s steering the ship, to what end, and why.
People worry about VR technologies for some of the same reasons that they worry about other media: overuse and addiction, misplaced priorities, negligence of ordinary-world responsibilities, ordinary-world social isolation, access to pornographic or radicalizing content. Worries of surveillance and privacy are intensified. Identity theft may take new forms: someone stealing your avatar or committing crimes with a digital rendering of your physical body. Bad actors may adopt minority avatars to induce or deepen minority biases. The digital personas with the ability to liberate us may end up, for the most part, solidifying social preferences. Everyone has a choice, VR says, and the vast majority choose the same Instagram face.
Perhaps there’s nothing new to worry about. Is VR so different from a video game? Mass media tends to provoke hysterics. Sociologist Jeremy Bailenson, for one, believes that there are new concerns. VR is useful for training doctors and soldiers because it can be used to teach us embodied skills. Unlike video games, then, VR can teach us how to use weapons effectively. Regarding psychological impact, says Bailenson, VR is different in kind and not degree from other media. “VR engulfs us,” he writes in his 2018 book, Experience on Demand. Bailenson’s lab studies the cognitive and behavioral effects of virtual technologies. It has a standard method for demonstrating the psychological power of VR. I put on a headset. Soon I’m standing on a platform above a chasm. There’s a second platform at the far side of the chasm, which is spanned by a thin plank. I’m asked to walk the plank. I know that I’m in a nondescript room with other people on a university campus. I can hear them laughing. I know that there’s no danger of falling. And yet—and yet this is not like the fear that one feels while watching a horror movie. The best of VR technology is very persuasive. Even while you’re wearing a headset, VR will persuade you that the facts of the world are the facts that it presents. The fear of falling feels real.
VR is especially well suited for provoking and directing people’s emotions. That it will be abused for ideological ends is obvious. That it is an effective tool for propaganda and torture is obvious. Chalmers suggests that it ceases to be an illusion machine for experienced users, who can correctly identify digital objects as digital. This is a surprising suggestion for at least two reasons: first, because the fact that VR technology is heading in the direction of indistinguishability is a fact that motivates Chalmers’s book; and second, because there’s evidence to suggest that precisely the opposite is true. We have reason to believe that the more time one spends in VR, the more uncertain one is as to the nature and location of one’s experiences. According to Bailenson, the once world-record holder for continuous time spent in a virtual environment reported afterward that he was often unsure, at any given moment, of whether he was in VR. Children easily acquire false memories. It’s suggested that they should be in some sense protected from this technology. But reality discernment doesn’t seem to be exclusively a problem for users under the age of thirteen.
The computer scientist and veteran VR theorist Mel Slater has offered a menu of worst-case scenarios for VR technology. The concerns increase as virtual experiences become more real-feeling. One concern is “lack of ground truth,” which threatens to undermine “some core elements of social fabric.” With AR, especially if AR devices become integrated with our bodies, physical-seeming digital objects will exist alongside ordinary ones in our ordinary environment. Sensory experience will be less reliably coupled to ground truth when the stuff of perception may be digitally rendered by a hacker. Say you see a cat on the mat. Say you see someone abuse a child. Why, with advanced AR, would a court of law believe in the integrity of that observation?
Chalmers assures us that such convincing deepfakes are “a long way off” (despite devoting most of Reality+ to the idea of perfect universal simulations). And besides, he says, an experienced user would be able to use her critical thinking skills to break down the machine’s illusions, correctly identifying digital objects as digital. One questions if the technology is so far off. One wonders if Chalmers overestimates our critical thinking skills. “Once deepfake virtual realities are perfected,” he writes, “we can expect that both computer security and brain security will be growth industries.”
It’s not certain what will happen to the public sphere when people opt to spend significant amounts of time in other realities. Chalmers leaves our world under-described, or described only negatively, in opposition to VR: our reality is “ordinary” reality, or better, “nonvirtual” reality. Perhaps “shared” does a better descriptive job. To use Hannah Arendt’s image, our world is like a table, which brings us together and keeps us separate. “To live together in the world means essentially that a world of things is between those who have it in common, as a table is located between those who sit around it,” Arendt writes. That we’re stuck here together can induce a feeling of resignation. Others look at billionaires, and say, at least we live under the same sky. It isn’t clear what will happen when the sky isn’t ours anymore, or the universe, when there is no common ground, or common table, because another has been made, which some will call just as real.
Suppose there’s a piano in the park in Facebook Reality, Chalmers writes. It will be true of Facebook Reality, and not, say, of Google Reality, that there’s a piano in the park. Chalmers calls this a harmless form of frame-of-reference relativism. Facts are true of one reality but not necessarily of others. The existence of other realities doesn’t undermine “the cold, hard facts about ordinary reality.” But for practical purposes, the facts of base reality are only as cold and hard as people report them to be. Frame-of-reference relativism seems less harmless when the existence of a base reality is unacknowledged or forgotten. One hears talk that this is already the case. It’s already the case that people live in their own worlds, in their own camps, with their own facts, tromping about the campfire chanting their camp orthodoxy. But these are metaphors.
I recently attended a Zoom event titled “Mindful Metaverse.” The instructor began with a PowerPoint. History progressed from the industrial age to the information age. We are now in the experience age, which is about immersion. Think of the metaverse as “coreality.” The metaverse, or coreality, is shared, spatialized, social, sensual, soulful and storied. A link to the metaverse was dropped in the chat. We were invited to join our instructor for a moon garden mindfulness meditation in the metaverse. I click the link. I have at least ten tabs open in one browser and four browsers minimized. Someone at my library table gives a dry cough. The instructor speaks over Zoom. I place my avatar where I feel comfortable, where I can feel a sense of flow, and ease. The fluorescents are burning. My computer’s fan is whirring. I attend to the grass at my feet. Moon-white pixels fall softly over the other avatars. There is the Cheshire cat. There is Mary Shelley. I am allowing myself to be here fully present, with my body, in coreality.
Great efforts are being made to recreate in VR a sense of social presence, the feeling of being together with someone in space. Some describe this as the attempt to offer digitally what digital devices have taken away, but with a difference. The difference is that digitally rendered social presence is offered on private platforms, where data is stored and then sold. Plurality is pushed through private channels like playdough through a spaghetti maker, and to the profit of the spaghetti maker’s manufacturers. I am told that there is something to be gained.
What is the motivation to render it all digitally? In addition to offering wearable NFTs, some luxury brands are now buying metaverse land to open virtual boutiques and luxury experiences. According to some estimates, the market for skins, or video-game character accessories, is over $40 billion. “If it seems strange,” Bailenson writes, “that people would shell out real money for avatar adornments and symbolic representations of status, then I encourage you to pause for a moment, look around you, and contemplate the pointless real-world consumer behavior that is a feature of our modern economy.” Supposedly there is something to be gained. Conspicuous consumption usually leads to mounds of plastic in the world’s oceans. But then, data centers need to be kept cool, and material for hardware needs to be extracted, and the devices need to be built. Perhaps, socially and environmentally speaking, it all comes out in a wash.
With consumer VR improved and more evenly distributed, experiences will be more widespread and less expensive. Not only the employees of tech companies, but most, soon, will be able to live in new Majorca or new Tokyo or deep in the new Sierras. Many more will be able to scuba dive, or climb tall mountains, or test an open relationship. And while some will note that, when every experience is on demand, it seems difficult to choose among them in any meaningful way, or rather, difficult to find meaning in any of them, and while these critics will find in the flittering from experience to experience a simple need to be stimulated, backlash to the experience machine can be easily accommodated by the experience machine. If one wants to homestead and pickle radishes then one can pickle in VR.
Perhaps a more defensible concern is that the technology’s designers aren’t being thoughtful enough about what they’re doing. When I speak to engineers about XR technology and ethics, they usually respond in one of two ways: either they find our discussion fascinating, in large part because they have never had it; or they say a version of, I just make chairs. You’re worried about the erosion of ground truth? All I do is make realistic digital chairs. The decentralization of accountability is certainly a concern. There’s also a concern that reality’s extenders will leave something out: something that we can’t account for, but that our bodies nevertheless register. Something like whatever is in excess of our expectations, or whatever makes it so that we can still be surprised, or pulled in a new direction, or feel so strongly after so much time. Whatever makes it so that we do not want bespoke fantasies but something else entirely. One has an intuition that there is something at stake in reality being found, and not made or chosen.
It’s true that what was once a fanciful hypothesis now seems more like a serious hypothesis. A reality TV show host was president but isn’t anymore. The world is burning. I see that the roses of Palo Alto are always blooming. I see radical Marxist critics eating raspberry truffles in the shopping district of Palo Alto, where a new store, “The RealReal,” just opened, and ask, is that a sim sign? With what probability? When everyone is streaming and none of the acronyms mean anything anymore, one is tempted to wear pajamas everywhere one goes.
A few holographers once told me that the holy grail of holography is a realistic window. What they meant was that holography will have made it, technically speaking, when a screen looks like a portal to another world. What I took them to mean was that windows are uniquely difficult to render digitally. I began to look at windows more intently. There a pot is floating in the trees. There a little picture is sitting on the garden ledge. Now a speckle, now a double-pane, now a spidery scratch of sudden brilliance. On a sunny day color gathers in the creases of a room, and many of the window’s reflections are difficult to locate in real space. Surely this could not be rendered digitally.
Or maybe it could be. Perhaps it’s all digital—who knows. Perhaps consciousness is simply a matter of our brain doing certain things, and those things could be done on a computer, instead of a brain. Perhaps we could be uploaded to the cloud and still be conscious. Perhaps our identities could somehow persist through this process, so that it makes sense, post-upload, to say—this is me. Perhaps in this form of reality there will be no scarcity, and thus no justice, and no death. It’s hard to see how we could rationally choose to enter such a state. It’s hard to see how there would still be meaning. On what grounds could we want to be a disembodied, computational mind, if as a computational mind our current conditions of wanting are all gone? As it stands, our values seem bound up with the human condition, and the human condition, with a nod again to Arendt, bound to things like birth, death, embodiment, the earth and its natural cycles, and the man-made, durable world of things that persist much longer than the labor it takes to make them, and ideally much longer than us.
The roses of Palo Alto are the city’s most surreal feature. I have never seen roses this large, in such abundance. I moved to the Bay in the summer of 2020, when it was impossible to breathe and just before the world turned orange. The air was once so bad that we needed to weatherproof the house, despite the heat, and wear our N95 masks indoors. But once the smoke passed and the layers of plastic insulation were removed, when the weather seemed to smile on us again with its usual benign smile, it seemed to confirm for us that the world is fundamentally a benign place, a place that wants you to thrive as its roses seem to thrive, big, abundant and without season.
This essay is part of our new issue 29 symposium, “What is tech for?” Click here to see the rest of the symposium.
Art credit: Rachel Rossin, Gaslit-stonetree, 2020. Acrylic airbrush and oil paint on panel with holographic display fan. Courtesy of the artist.