Frank Jackson | Photo @David Braddon-Mitchell (2018)

Mary’s Room is infamous, the stuff of philosophical legend. It is one the most famous thought experiments ever, designating Frank Jackson as one of the most influential philosophers of the second half of the twentieth century. It has spilled considerable ink, with more than a thousand research papers published on it alone. But what is it and why is it so influential and controversial?

The thought experiment goes like this:

Mary is a brilliant scientist who is, for whatever reason, forced to investigate the world from a black and white room via a black and white television monitor. She specialises in the neurophysiology of vision and acquires, let us suppose, all the physical facts and information there is to obtain about what goes on when we see ripe tomatoes and thus the colour red as it pertains to light waves, retinas and neurological processing etc. (situation one) …. What will happen when Mary is released from her black and white room or is given a colour television monitor and actually sees and experiences the colour red in real time (situation two), will she learn anything or not?

Careful now and think this through, your answer has mammoth implications. Part of the beauty of the thought experiment is its simplicity in conveying the complexity regarding the two situations. If you think Mary learns something new when seeing colour in real time, physicalism could be in jeopardy because Mary already knew all the facts related to colour and its vision, so how can she learn or experience anything new in situation two? If she doesn’t learn or experience anything uniquely new when seeing colour in situation two, then physicalism can still be intact.

The thought experiment’s aim is to challenge the metaphysical notion of physicalism and how it relates to subjective experience and qualia. If Mary knew all the facts and had all the knowledge whilst reading/studying about the colour red (situation one), then nothing superfluous should happen when she does see and ‘experiences’ the colour red in real time when she leaves the room (situation two). But most people would intuitively agree that reading and knowing all the facts about colour and vision is not the same as actually seeing and experiencing the very same colour in question. So, what is going on? Is physicalism therefore, false? Can it not explain experience? Does Mary really gain new facts and knowledge when leaving the room? Just what is going on in the two situations in Mary’s Room and what does it say about the nature of knowledge, experience, and representations.

Tony Sobrado: Mary’s Room attempts to dispel physicalism. After much controversial debate people have settled in broadly three camps:

1.  The thought experiment does not refute physicalism but does prove the presence of qualia (the subjective feels of 'what it is like').

2. Mary does not actually gain any new knowledge or facts in situation two; she only gains a new ability or that the knowledge present when Mary sees the colour in situation two is simply representational knowledge.

3. The premise of the thought experiment, that physicalism is false, is valid and stating that Mary does gain new facts when seeing colour in real time in situation two means that situation one is incomplete and there is an issue with the paradigm of physicalism.

Since its publication, you have argued that what occurs in situation two is actually representational knowledge and no longer claim that physicalism is false because Mary's Room is really about the representationalist terms of phenomenal experience. Basically, seeing colour in situation two is the representation of things as being a certain way and when you think in those terms it’s a mistake to wonder where the 'special redness' is in traditional qualia.

So please explain from your perspective how representationalist knowledge is related to experience,  how it relates to 'qualia' and what your general thoughts are on qualia today, and if qualia even exists in any form?

Frank Jackson: To answer your questions about qualia, I need to do a bit of scene setting.

So, let’s start with perceptual experiences. Perceptual experiences help us navigate our surroundings and to report on how these surroundings are. The obvious explanation of this acknowledged fact is that perceptual experiences are causal responses to our surroundings that represent, not infallibly but with reasonable accuracy, how they are. This relatively non-controversial thought suggests a way of thinking about a highly controversial matter - the ‘what it is likeness’ of perceptual experiences. Of course, there is such a thing as seeing something as bright red, as moving from left to right, as moving towards an object etc. But, despite the ‘of course,’ there is something deeply puzzling about this. In describing what it is like, we use terms we know do *not* apply to perceptual experiences. We talk of something’s being an especially bright red, of yellow dots moving from left to right or towards one, and of whether or not the rate of movement remains the same over time and so on.

But no-one thinks that the experiences themselves are moving from left to right, or are bright red, or are moving towards one, etc. So, what on earth is going on?

Representationalism says that when one reports on ‘what it is like’, one is accessing in a special way how one’s perceptual experiences are representing things to be. The experiences aren’t moving etc., but when it seems so right to use ‘moving’ etc. to describe what it is like to have them, what one is describing is how one’s experiences are representing things to be.

What’s the special way of accessing how one’s experiences are representing things to be? It is the exercise of a *recognitional capacity*. We are able to recognise that our perceptual experiences are representing that things are thus and so. It is one thing to be in a state that represents that things are thus and so, but another to be aware that one is in a state that represents that things are thus and so. When we see something *as* bright red, or *as* moving, or *as* approaching us, etc., we are recognising that this is how our perceptual experiences are representing things to be, and, thereby, believe that this is how our experiences are representing things to be.

So, do I believe in qualia? If by that you mean do I believe that we have a distinctive way of accessing how our senses are representing things to be? The answer is yes. But if you mean by that question do I believe that the ‘what it is likeness’ of perceptual experiences - the feature of them that underlies much of the thinking about the hard problem of consciousness - outruns our access to how our experiences are representing things to be, then the answer is no.

Tony Sobrado: So, you believe in what might be called ‘weak qualia,’ that we can access how our senses are representing things to be. For example, 'that appears to be a dark shade of red as opposed to a lighter shade of red'. But ‘stronger’ forms of qualia, or formal qualia, relating to the 'what it is likeness' of perceptual experiences - that outruns our access to how our experiences are representing things to be - you deny this. Now illusionism argues that our sense that it is like something to undergo conscious experiences is due to the fact that we systematically misrepresent them as having phenomenal properties. You are saying we can access how our senses are representing things to be but there is nothing significant about qualia, formally, that outruns our access to how our experiences are representing things to be. This is a similar argument to denying that there really is any kind of phenomenal property in experience itself - something above and beyond the direct experience itself. This is also echoed by Dennett's famous paper 'Quining Qualia' where he advocates that there is no uniquely distinct quality behind experience itself, nothing above and beyond it and he has examples such as 'inverted qualia' to demonstrate this.

Others in this space elaborate, I think of Susan Blackmore specifically, on the fact that when you are going about your daily life, processing and absorbing information in real time, that in that moment you don't think 'what it is like to see red'... you just see red. These accounts also tie into what you say about representational experience; that these experiences and sensations are just representing the exterior world to us and that there is nothing special about our experiences beyond this; and functionalist arguments behind consciousness would agree.

This is all well and good. However there still is experience regardless and David Chalmers insists that experience itself is the key issue. So, when representationalism says that when one reports on ‘what it is like’, one is accessing in a special way how one's perceptual experiences are representing things to be, it is a recognitional or functional capacity. But 'experience’ is still there and so is its problem. So, whether you are an illusionist, a functionalist or a representationalist about experience; there is still an experience de facto so its looks like the hard problem of consciousness, even if just employing ‘weak qualia,’ is still there. The hard problem being, in its simplest form, how and why a physical brain produces a subjective experience? This issue is still there. What are your thoughts on this?

Frank Jackson: Let me start with some remarks that echo some of what you say early on in that question. (I may here sound a bit like an illusionist - like Daniel Dennett and Keith Frankish, no bad thing in my book.) This will lay the ground for my response to the worry you put in David Chalmers’ mouth, which, of course, might equally have been put in my mouth back in the day. I agree, in much company and with you, I take it, that experiences as such are not conscious. When I reach a drop shot in tennis, I rely on a whole range of perceptual experiences of which I am unaware. When we theorists discuss the ‘what it is like-ness’ or the phenomenal nature of seeing something as red or as approaching us, we are discussing what we might call the experience of experience, where the first occurrence of ‘experience’ is the sense you have in mind when you use ‘experience itself’ to characterise Chalmers’ concern.

I think it should be common ground that there is an element of illusion in our thinking about what it is like to be in various mental states. We tell the doctor that the pain is moving down our arm, but no-one thinks (or should think) that the pain literally travels down their arm. Some amputees describe their pains in terms of where their leg would be but for the operation, but no-one thinks that pains are literally located in empty space. And, as we noted earlier in our discussion, we describe our perceptual experiences using terms that we know do not apply to them - for example the ball is travelling towards you but the experience is not.

Representationalism gives us a way of understanding what’s going on here. The key idea can be introduced by reflecting on a well-known visual illusion. In the Müller-Lyer illusion, two lines of the same length look to have markedly different lengths. This is an illusion because one’s awareness of the two lines looking to be different lengths goes hand in hand with one’s awareness of how things have to be to be in accord with their looking to be different lengths - the lines need to be different lengths in fact, and they aren’t. The key point is that the phenomenal nature of one’s experience delivers how things have to be to be in accord with the experience. Now we can say what’s illusory in our thinking about conscious mental states. When we describe what it is like to have an experience E in terms of property P, what it’s like to have E doesn’t come from E’s being P; it comes from our recognising that, in order for things to be as E represents them to be, something needs to be P. In a bit of philosophers’ jargon, we confuse an intentional property for an instantiated property.

Now for the worry you put in Chalmers’ mouth. What about the nature of the experience itself? But now we know how to characterise the nature of the experience itself; it is in terms that are not true of the experience itself. Pain does not literally move down your arms. Experiences of something looking red or moving towards one are neither red nor moving towards one, and so on. But, of course, there is something it is like to, e.g., see something as red and as moving towards oneself. It is for one to be aware that that is how things in front of one *putatively* are, which in turn settles how things have to be for one’s experience not to be an illusion. When we contemplate an instance of the Müller-Lyer illusion, we can sensibly reflect on whether the lines look to be 10% different in length, or maybe nearer 20%, or maybe……? But these reflections are not sensible ones to have if thought of as pertaining to the visual experience itself; they make sense only in as much as they are read as reflections on what it would take for the lines to be as they look to be. Do they need to be 10% different in length, or 20%, or maybe...?

Tony Sobrado: ‘When we describe what it is like to have an experience [...] we confuse an intentional property for an ‘instantiated property’ I like that and well put.

Now what that does is it essentially addresses the bigger question of overall experience that is to say, like an illusionist would say, there is nothing that’s really going on in terms of experience above and beyond the direct experience itself. There is no innate phenomenological property behind experience itself and as we've already discussed, this is covered off by elements of representational knowledge and functionalism etc. which can lend themselves to illusionist frameworks.

I think your example of pain travelling down the arm is excellent in terms of pain and its experience simply being a representation in our mental schema so to speak. It's not that pain is literally and physically moving down your arm but that's how the experience is represented to you. There’s a whole host of work that bears this out in neuroscience. Michael Graziano at Princeton argues that the ‘mental world’ so to speak (and I say that in inverted commas because some people think there isn’t actually a mental world just like some people don't think there is phenomenological experience) is the brain (the hardware) modelling itself to itself. It is a sense of how the brain gets to understand itself and navigate the world. So that example of pain moving down the arm is a great example of that because pain doesn't literally move down the arm; it's simply how it’s represented to you internally as a mental state. That example goes a long way in clarifying some of the misconceptions that people have about illusionism. For example, Galen Strawson says illusionism is absolute nonsense, how can our sensations, inner world and experience be an illusion? We experience it so it's real, but you've just elucidated on to what extent it is an illusion and Keith Frankish says similar things in terms of phenomenological properties.

Now David Chalmers is going to say something else about experience that I won't get into right now and you've addressed that in terms of the difference between an intentional property and an instantiated one. That might not be sufficient for some. As someone working in this area myself and having spoken to many people across the spectrum in this area there are undoubtedly going to be some people that say illusionists, representationalists and functionalists etc. are just moving the goal posts here! Yes, there is an element where there may not be a formal phenomenological property of experience; and experiences are not what we really think they are. However, the issue of experience whichever way you cut it is still there. Now you've gone into great detail already from your position stating that well yes there's an experience but it's not really what you think this experience is. You have said of course there's something ‘what it's like’ to undergo an experience even though this isn't anything above and beyond the direct experience itself - something that outruns our experience.

OK, so let's narrow down and frame this discussion more specifically: We're left with subjectivity and sensations. We don’t have formal qualia so to speak but we do have subjectivity, sensations and an inner world (disregarding the discourse encompassing potential zombies) so let’s talk about that subjectivity and sensation. So, what is left of the hard problem of consciousness if anything at all? Because you've already noted that you don't believe in qualia in the traditional sense that underpins the hard problem, so is there a hard problem at all? Even in a weak sense? Essentially given a physical brain, is there a problem combining physicalism with any sort of experience - even if it’s just representational experience? Furthermore, what about explanatory gaps as espoused by the likes of Joseph Levine? That if ‘experience’ Y is the neural fibre X in the brain that is fine, but we need an explanatory mechanism explaining why neural fibre X produces the experience and sensation of Y, so what do you think of explanatory gaps?

Frank Jackson: Let me start by saying a bit more about the view I like (I say it this way to highlight my debts to others). This will allow me to respond to the important issues you’ve raised. Experiences represent how things are around us. We have an ability to recognise how our experiences are representing things to be, and that’s the key to understanding what it is like to have such and such an experience. You might well ask for clarification on what I mean by exercising this ability. Here are two illustrative examples that may help. One way to learn that the face in front of you is a face you have seen before is to hear the words, ‘We have met before.’ Another way is to exercise your ability to recognise faces (this way is denied to those suffering from prosopagnosia). The second way is what I am talking about, so here's another illustration: One way to learn that your knees are bent is to see your reflection in a mirror that shows your knees as being bent. Another way is to exercise what is sometimes called the sixth sense, proprioception. The second is an example of what I am talking about.

Now we have a way of thinking about the view you mention in connection with Michael Graziano, but I highlight that I do not know his work. What I describe in the above paragraph can be thought of as the brain modelling itself; it is the brain capturing how it itself is representing things to be.

We also have a way of understanding the famous explanatory gap, at least as it applies to colour experiences. We representationalists hold that colour experiences are states that represent that objects have various highly distinctive surface properties. (Let’s stick to surface colour). This is how we learn the words for colours: We learn to tag the properties our visual experiences represent those surfaces as having. This is, however, only possible if we are aware of how our experiences are representing those surfaces to be. We need to recognise how those experiences are representing things to be. Otherwise, we’d be learning words for we know not what, to paraphrase John Locke. This means that we are not tagging spectral reflectance profiles, despite the fact that spectral reflectance profiles of surfaces causally determine the colours surfaces look to have. What then are we tagging? The answer is that we are tagging locations in a complex similarity space. More particularly, we are tagging locations along three axes, hue, saturation, and brightness-lightness, as depicted in the well-known colour solid.

These locations are occupied by reflectance profiles, but the similarities and differences our visual system responds to are highly unobvious ones amongst those reflectance profiles. Neuroscientists sometimes talk of the brain inventing colours. If what I am saying is correct, our brains discover highly unobvious similarities and differences between, as it happens, the reflectance properties of the surfaces we view. Our brains do some discovering, not some inventing. The ‘what it is likeness’ of seeing some particular surface as coloured is, then, the recognition that one’s visual system is representing that surface as occupying one or another location in that similarity space, a location that is deeply obscure if one looks simply at graphs of reflectance profiles (percentages of light reflected at frequency points). Little wonder, then, that there is a kind of gap between the physical account of the reflectance profiles and our perception of colour.

There remains the question of how to understand the ‘what it is likeness’ of seeing a surface as coloured. But we know - or so say I - how to understand the ‘what it is likeness’ of undergoing the Müller-Lyer Illusion. Recognising that one’s visual system is representing that the lines differ in length presents itself as a kind of direct acquaintance with two lines differing in length. Similarly, recognising that one’s visual system is representing the location of a surface in a similarity space presents itself as a kind of direct acquaintance with a surface quality that, by its very nature, has that location. This is why pink presents itself as being *between* red and white, where else could it be? Red is very *different* from green; how could it be red unless it were very different from green? Yellow is *brighter-lighter* than brown; how could it fail to be and still be yellow? And so on. Now we can, hopefully, say a bit more about the ‘illusion’ question. Of course, we see things as yellow; that’s not an illusion. But the attractive, or attractive to many, thought that in seeing something as yellow we confront a property differing in kind from the properties discussed by optical science is an illusion. Instead, we are recognising a location among those properties in the same way we recognise a face as one we have seen before. Mary in the black and white room knows about this location of course, but her knowledge comes from a complex calculation of spectral reflectance, and not the exercise of recognitional capacity.

Tony Sobrado: So just to be clear here and to make sure I understand what you are saying, because representationalists hold that general experiences are states that represent objects and the external world to us,  we tag the properties to our experiences and in so doing we are aware that our reality is representing things to us - this is generally the representationalist view of experience and this for you actually dissolves the explanatory gap - as Levine and Kim would say, why is it that brain state X produces sensation Y? You believe that this issue is dissolved by stating that it is representational knowledge. Is that correct?

Frank Jackson: For the explanatory gap question, yes representationalism dissolves that gap.

Tony Sobrado: OK, as you do not believe in qualia in the traditional sense that underpins the hard problem of consciousness - as something that outruns our direct experience of the world around us - do you actually believe in the mental realm itself or are you more partial to mental eliminativism like Patricia Churchland and if so why? And if not, why not?

Frank Jackson: Two comments on eliminativism. First, I should emphasise that I am not an eliminativist about ‘what it is like.’ I am someone who has a certain view on its nature. In the case of seeing something as yellow, the view comes to: Seeing something as yellow is representing that it is a certain way, combined with recognising that one is representing it to be that way, where recognising that one is representing it to be that way is one way of believing that one is representing it to be that way.

I know that many think that any representationalist treatment of ‘what it is like’ must really be a (covert?) version of eliminativism; somehow or other, the vibrant, experienced nature of what it is like to see something as yellow, say, is being denied. As Dennett expressed the thought many years ago, ‘they wonder where the yellow went?’ I think (following Dennett, I trust) that they are overlooking the fact that the debate about ‘what it like’ is driven by our *beliefs* about what it is like, and our *memories* of what it is like (often when we write about what it is like to see something as yellow, we are not at the time seeing something as yellow). But belief and memory are classic examples of representational states so it had better not be the case that representational states cannot capture ‘what it is like.’

My second comment on eliminativism relates to the Churchlands’ position. I think of their eliminativism as focussed on propositional attitudes. We distinguish the belief that snow is white from the belief that grass is green by appealing to the difference in the propositions - that snow is white versus that grass is green - which are their objects. One might then go on to think that these two beliefs correspond to distinct internal states playing distinctive causal roles. I think of the Churchlands as challenging this line of thought about internal causal architecture, but you may well know more about their views than I do.

Tony Sobrado: The reference you made to memory is a telling one with consequences for formal qualia - the ‘what it is like’ that goes above and beyond the direct experience itself in a phenomenological manner because formal qualia encounters an issue when faced with memory and belief. This is because it's debatable that anyone has any ‘what it is like’ moments in real time. Instead, they just do and feel the event in real time. So again, this lends credence to the fact that qualia isn’t this phenomenological property that outruns the experience itself and that we have direct acquaintance with it. This is also borne out in memory. Memory is a working function of the mind itself and ‘what it is like’ also occurs in a memory but this again is another representational experience and thus a device of a functioning brain with a mental schema. And so, for you, representational knowledge serves two purposes:

1. There is no formal qualia and experience above and beyond the experience itself, in a formal phenomenological sense, and so you don't believe there's a hard problem of consciousness at all. 
2. Representational knowledge also dissolves the explanatory gap.

But is there anything lurking in the back of your mind that sometimes gets you when you think of experience? Why do we have experience? Or do you just instantly recall the representational view of knowledge?

Frank Jackson: That’s a good question and as a onetime and self-described ‘qualia freak’, I feel the force of the views of my past self. So, yes, I do sometimes need to remind myself of the powerful case for the representationalist way of thinking about the debate over ‘what it is like.’

Tony Sobrado: It’s interesting that you described yourself as a once ‘qualia freak’. This is because one of the things that I find fascinating in any discussions and debates regarding profound matters is the ability for people to change their mind. Many people have given you plaudits for changing your perspective on something that you had such a great impact on. Mary’s Room was very influential so for you to retract your original position illustrates a primary dedication to intellectualism.

But something that I find just as fascinating is when people state that you were right the first time - it’s the second time when you advocate representational knowledge where you’re wrong. So, what is impressive is that:

1. You changed your mind.
2. Some say you didn’t need to; you were initially right all along.

The second of these points is echoed by Torin Alter who thinks that seeing and experiencing colour in real time (situation two) is a new fact itself and Philip Goff, the leading panpsychist, argues that Mary’s Room does not refute physicalism but does prove the presence of qualia. In all honesty I happen to agree with Goff. Despite representational knowledge’s attempt, along with illusionism, to set aside phenomenological qualia, which might be the case regarding phenomenology, the issue of sensation and experience is still there. It’s too late Frank! The cat is out of the bag! Now I know you will say that you have put the cat back in the bag with representational knowledge and that is fine so with that in mind let’s discuss property dualism, emergence and identicalism.

Do you subscribe to property dualism - that the mind and brain are two properties but part of the same substance - physicalism? Do you believe that the mind is an emergent property (weak or strong emergence)? Or do you simply endorse identicalism - that mental states are brain states, or do you propose something else when it comes to the ontology and potential metaphysics of the ‘mind’?

Frank Jackson: My physicalism/materialism belongs in the ‘Australian’ camp. I think that mental states are identical to brain states, and that what makes it the case that some brain state is identical to some given mental state is that the brain state in question plays the right kind of functional role. Given my remarks earlier in this exchange, you will not be surprised to hear that the relevant functional roles are very often those that make the brain state represent that things are thus and so. What does this imply about the relationship between the mind and the brain? It depends on one’s account of the relationship between a mind and the states of a mind, but if you think of a mind as a suitably integrated array of mental states, then the mind will be a suitably integrated array of brain states.

What does this imply about emergence? A car piston lying on the ground doesn’t play any interesting functional roles. It might trip someone up, but that’s not very interesting. However, a piston rightly aggregated with the many other parts that make up a functioning car engine plays important, interesting functional roles. Ditto for the parts of a functioning computer. They get to play important, interesting functional roles once, but only once, they are rightly integrated with all the relevant bits and pieces. There is, therefore, a sense in which we can talk of emergence in both cases. Assembling aright leads to new, important, interesting properties. Ditto for the brain states that are mental states, say we materialists who belong in the functionalist camp. Assembling aright - the result of the process described in medical texts about the path from fertilisation to embryo formation to birth to development into humans like you and me – leads to brain states that play the functional roles needed to be mental states. Is this a strong or weak emergence? I trust I have said enough to make the view clear without the need for a decision on the label. You also ask about property dualism. In the view I have just sketched, neither minds nor brains are properties, but minds are literally made out of the same ‘stuff’ as brains.

Tony Sobrado: Basically, you're an identicalist and a functionalist that sees mental states as being the same as brain states hence your ideas on representational knowledge.

As for emergence, this is always a tricky concept especially when we talk about realisability, supervenience, causation and reductionism. The way you have elucidated it there is fine but that's neither weak nor strong emergence. Weak emergence states that it's ultimately realisable in the physical yet the emergent property, in our case the ‘mental’, is an autonomous property in its own right - and you don't subscribe to that. Secondly, therefore, you definitely won't agree with strong emergence, which is it that the emergent property can't even be reduced to the physical.

Frank Jackson: You are right Tony, yes, I'm not a believer in emergence in either of those senses.

***

Frank Cameron Jackson is an Australian analytic philosopher and Professor Emeritus in the School of Philosophy (Research School of Social Sciences) at the Australian National University. *** Tony Sobrado is a social scientist and freelance writer focusing on philosophy, science, atheism and current affairs.

***