A full interview with Donald Hoffman, Cognitive Psychologist at the University of California, follows this book review

Does the moon still exist when you’re not looking at it? In a provocative new book titled The Case Against Reality: Why Evolution Hid the Truth from Our Eyes, psychologist Donald Hoffman suggests that the answer is no. Evolution, the argument goes, may have shaped our perceptions to create an interface analogous to a computer desktop interface. Trains, snakes, and even the moon may be mere icons that guide our behavior, much like the icons on your computer desktop. That doesn’t mean that they don’t represent something real out there—just as dragging a blue email icon to your trash really does delete an email file in your computer, stepping in front of a train really does kill you. But, Hoffman argues, just as the email file in your computer is not actually a blue pixelated rectangle, the actual threat posed by the train is not actually a large, elongated object occupying three dimensions of space and one of time. We must take the train seriously, but not necessarily literally, his reasoning goes.

Indeed, just as direct access to the bits and voltages inside a computer would be unlikely to help us quickly write an email, Hoffman claims that direct access to the complex information in our real environment is unlikely to help us pass our genes to the next generation. With playful references to The Matrix (“There is no spoon …”), Hoffman argues that perceptions that are veridical—that is, truthfully reflecting objective reality—are less likely to help us survive and raise offspring than false perceptions. Hoffman argues that false or non-veridical perceptions actually serve as an interface, compressing vast amounts of information to help us survive.

The basis of this claim lies in evolutionary game theory, simulations that show how creatures survive based on whether or not their perceptions are veridical. Using these simulations, Hoffman and several collaborators have put forth a mathematical argument that there is virtually zero probability that our perceptions are a true or even almost-true reflection of reality. As a concrete example, consider an organism that can only survive in an environment with an intermediate concentration of some particular substance. Perhaps this substance is oxygen: high and low concentrations are lethal, so our organism must find the sweet spot in the middle. As argued in Chapter 4 of the book, if the organism directly sees the concentration of oxygen, then it will probably move towards the highest concentration and die. But if it instead only sees intermediate levels of oxygen, while being blind to high levels, then it can survive by moving only towards what it can see.

While the logic of this argument is rooted in evolution, Hoffman also invites us to consider some curious features of modern physics that also suggest that our perceptions are an interface. For instance, quantum mechanics, which governs the world at the atomic and subatomic scales, shows us that particles may not actually exist in a definite location until they are directly observed. You might think of this like a video game that only renders the visual graphics needed by the observer at any given moment. So, could quantum mechanics also suggest that our perceptions exist only to compress a far more complex reality?

Like Morpheus from the Matrix, Hoffman offers us a red pill with the promise of understanding the truth about the outside world. Yet, as the reader, I found this red pill difficult to swallow. Occam’s razor, a principle for choosing between competing scientific theories, tells us to select the simplest theory with the fewest parts. Isn’t the simpler theory that our perceptions are more or less true, rather than a mere interface hiding a higher dimensional reality? I was lucky enough to interview Hoffman and raise this objection to him; an abbreviated transcript of our conversation follows this book review.

In both our conversation and in the book, Hoffman argues that his interface theory of perception (ITP) is needed to address the gap between physical processes that take place in the brain and our conscious experiences. Even if we fully understand the neurophysiology of color perception, why, for example, does it feel like something to behold a rainbow? Why don’t the neurological processes that allow the brain to perceive color simply take place in the dark, so to speak, without subjective experience? This conundrum, known as the hard problem of consciousness, has baffled many scientists and philosophers. Hoffman believes the way forward is to assume that consciousness precedes brains and is fundamental, and that reality is, at the most basic level, a network of interacting conscious agents.

Whether or not you agree with ITP or Hoffman’s network of conscious agents theory, The Case Against Reality is not only a highly enjoyable read but also a stimulating exercise that challenges us to consider how true our perceptions really are. Nearly all neuroscientists agree that even if our perceptions are largely veridical, we don’t always see the truth. Illusions like the Necker Cube, for instance, show that our minds can be tricked into perceiving things that aren’t there. In the image below, is the yellow side the far or the near side of the cube? Looking at the cube, you can willfully cause it to flip between both orientations.

The Necker Cube has no true orientation, only that which exists in the mind. Neuroscientists like Anil Seth at the University of Sussex have used such illusions to argue that all our perceptions are hallucinations based on what our brains predict to be true. That is to say, what we see is not merely the light that hits our retina, but a far more user-generated experience, if you will, based on predictions such as “faces are always convex and never concave.” In that particular example, we know faces in our environment nearly always curve outward, but the occasional exception may result in a powerful illusion. While Seth believes that the predictions that drive our perceptions are usually true, Hoffman diverges in his thinking by arguing that space and time are themselves only an interface that exists in the human mind. Although I’m more in agreement with Seth, there’s no doubt that many of the brain’s predictions regarding color, motion, and orientation are sometimes off the mark. With new illusions being discovered each year, the exact extent to which our wakeful, sober perceptions are veridical may remain an open question for years to come.

Interview with Author, Donald Hoffman:

KN: How long have you had this intuition about reality?

DH: I first had the intuition in 1986. I was working with Chetan Prakash and Bruce Bennett on a mathematical model of perception. And it was looking at the mathematics of that model and having a conversation with them about it that I remember one day I suddenly realized that what we’re seeing might not be real, that we might just be making this all up. And it hit me all the harder because, it was my own theory. And I hadn’t been trying to build it into the theory, I was just trying to build a mathematical model of perception.

That was the red pill moment for you?

That’s right, the mathematics opened up the red pill and handed it to me. I still remember, I was so stunned I had to sit down.

Do you see your theory of conscious agents as a solution to the hard problem of consciousness?

Yes, that’s why I’m proposing it. I’ve been prompted by how little progress we’ve made in physicalist approaches to the hard problem of consciousness. You know, integrated information theory, microtubule theory, global workspace theory, and so forth can’t explain a single conscious experience.

I think I have difficulty accepting ITP because Occam’s razor suggests that, all other things being equal, we should favor the simplest hypothesis. Doesn’t Occam’s razor favor the hypothesis that objective reality is more or less what we see, rather than something beyond three dimensions of space and one of time?

Well, I would agree with you and I would have been happy to go that direction if we could solve the hard problem of consciousness from within that physicalist framework. It’s the utter failure to solve the hard problem of consciousness that’s sort of forced my hand here. And I mean, I could have had ITP and just say, well, there’s a more complicated physical universe. Maybe there’s ten dimensions instead of just three spatial and one temporal. I could have done something like that. But the Occam’s Razor is about theories that can explain the phenomena, right? Of all the theories that can explain the phenomena, choose the simplest. Well, that theory, that we’re seeing reality as it is, cannot explain the most critical phenomena that we have, namely our conscious experiences. So, Occam’s Razor fails to apply here because Occam’s Razor is only about theories that can do the explanatory work. Now you’re choosing among them. Well, physicalism utterly fails to explain anything about consciousness. It just can’t do it.

If we only perceive fitness payoffs, why can I see the Milky Way Galaxy under a dark sky at night? I don’t see any obvious fitness payoff there.

Right. So the idea will be that evolution has shaped us with a very simplified interface that’s been shaped mostly to report the stuff that’s going to keep us alive. But if you look up at the stars, you’ll notice that you’ll see you see a mountain in the distance and then you see the moon and the stars coming up behind the mountain. The mountain looks far away. And the stars and the moon look a little bit further away. But, not that much further.

Right, you have no intuition for how far they are.

No intuition. And that’s why we believed for a long time that there was this little shell around the earth and things weren’t that far away. And that actually is an interesting aspect of the interface idea. And that is, what is space from the point of view of the interface theory? Well, it’s just a data structure. Space and time are just a data structure. They’re there to represent fitness payoffs. The distance from me to an apple, say, here like two meters away versus another apple, you know, 20 meters away—that distance is coding the percentage of my caloric resources that I currently have that would be required to be expended to get the resources in the apple at two meters versus 20 meters. In other words, distance is a calorie expenditure representing fitness cost. And so, it’s no surprise that the stars look about as far away as the mountain, because they’re both at infinity given my caloric resources.

In your book you talk a lot about visual illusions like the Necker Cube. What about hallucinations? Do you think there’s any possibility that hallucinations might be more veridical than our sober perceptions?

Not the normal hallucinations if you get DT—delirium tremens¹—or something like that. Yeah, those are probably not going to be any more veridical.

What about Aldous Huxley’s idea expressed in The Doors of Perception, that our conscious mind filters sensory input and that psychedelic drugs release this filter on reality?

I am absolutely open to the idea, that there could be [drugs] like DMT² opening us up to new levels of reality that are genuine insights.

The way that people describe DMT, that almost sounds like something outside of spacetime, right?

That’s right, and people who’ve done those [DMT] come back saying that it felt more real than our everyday reality. They end up being more convinced that it’s real. They’re interacting with these guides that are actually telling them relevant stuff that’s important to them, and every time they go back on another trip, they pick up where they left off and are learning new stuff. I mean, that doesn’t sound like the musing of a mind addled by alcohol. I’m not saying I’m sure, but I can’t dismiss the possibility that, in some sense, as Huxley puts it, we’ve opened the doors of perception, or as James puts it, this filmiest of screens, between us and other forms of conscious experience has been split, and we’re seeing these other forms of conscious experience that could be interface representations other than the one that we’ve got. On the other hand, it could also be that we just have addled brains.

What do you think of Anil Seth’s idea that we hallucinate our reality?

He likes the idea that we construct what we see. But he still assumes that in the normal case we are reconstructing some aspect of a physical reality. He’s a really bright guy. And a really nice guy. But the question is, what is controlling the hallucination? And he’s saying an objective physical reality that we need to track in order to stay alive and be fit. So it’s an objective reality that’s controlling the hallucination. And I’m saying that, no, all the stuff that we’re seeing, it’s a hallucination in the sense that it’s just a visualization tool that doesn’t resemble reality.

Quantum physics seems to suggest that subatomic particles don’t have a definite location in space until we observe them, almost as if we construct our own reality. Would you have come up with this idea if you didn’t know what you knew about quantum physics?

Oh, absolutely. It was entirely the theory of evolution by natural selection that sort of nailed down the ITP idea for me. I was aware of quantum theory in the 80s, so no doubt it influenced me. But I think I interpreted the quantum stuff the way most physicists interpret it, not in terms that space and time are just a user interface. So I was aware of the quantum weirdness, but I think it was my study of perception that really tipped me over to realizing that maybe spacetime itself is just a user interface. And then I could go back and look at the quantum stuff and realize, well, there is a way of interpreting the quantum stuff that’s compatible with the interface theory of perception.

Let’s talk about your idea that reality is, at the most fundamental level, a network of conscious agents. How do we get natural selection from there? If minds exist before genes, what is natural selection actually acting on?

So the idea is that reality is this vast social network of interacting conscious agents. And there is a sense of “fitness”, in vast social networks. The more connections you have, in some sense, the more fit you are. And the less connections you have, the less fit you are. So Google has tons of connections and the owners of Google are billionaires, Hoffman has a very few and he’s not a billionaire. And then you see this relationship between the number of social connections and fitness. What we may have to do is the following: we may have to say, OK, let’s look at evolution by natural selection. Let’s try to pull that back, do a reverse interface. Pull it back through the interface out to the realm of conscious agents. There’ll be a number of dynamics of conscious agents that are compatible with it. Let’s look at those dynamics and see which one makes sense. And then that will give us some insight into what’s going on in the realm of conscious agents, what they’re up to, and then when we project that back into our spacetime interface, we should get evolution by natural selection or hopefully a generalization that makes new predictions beyond evolution by natural selection. Same thing for quantum mechanics and general relativity.

Does ITP have spiritual implications?

I want a scientific spirituality in which we begin to explore a world beyond space and time. But we do it with mathematically precise models, and we start to address the big questions about why are we here and what is human consciousness about? Where did it come from? What’s the meaning of life? And so forth. Topics that scientists have in many cases said we don’t need to address. But in fact, they did address them. From the physicalist framework, the answer was, there is no life after death. There is no deep meaning to life, because once your brain dissolves, that’s it. And so they really did have a theory of life and transcendence: there is no such thing, there is no transcendence. But now all sorts of possibilities open up for exploration. And I’m pretty excited about it. So science and spirituality, I think, could really start to collaborate. But scientists have to let go of spacetime and spiritual traditions have to let go of dogmatism. Not easy.

Donald David Hoffman (born December 29, 1955) is an American cognitive psychologist and popular science author. He is a Professor in the Department of Cognitive Sciences at the University of California, Irvine, with joint appointments in the Department of Philosophy, the Department of Logic and Philosophy of Science, and the School of Computer Science.

Joel Frohlich is a postdoc studying consciousness in the lab of Martin Monti at UCLA

Footnotes

[1] Hallucinations that result from alcohol withdrawal after excessive drinking
[2] N,N-dimethytrypatmine, a drug that causes powerful hallucinations and is an active ingredient ayahuasca, a brew made by the indigenous people of the Amazon basin. DMT is illegal in most countries.