Blindness, Shape, and Modality
“The eye sees only what the mind is prepared to comprehend.” Henri Bergson
A few years ago I was showing my Nana pictures on my phone and though she looked at the phone it wasn’t obvious she understood what she was looking at. It was as if she was just looking at the phone and couldn’t see the pictures. She has good vision so it wasn’t a problem with vision. I think it was a modality problem related to ‘Molyneux’s question’.
324 years ago William Molyneux wrote a letter to the British philosopher John Locke that asked a question that has only recently been answered by Pawan Sinha, a professor at MIT. Molyneux wrote, “Imagine that a man blind from birth who has learned to identify objects – a sphere and a cube, for example – only through his sense of touch is suddenly able to see. The puzzle is whether he could, by his sight, and before he touch them, know which is the globe and which the cube?”
The short and counter-intuitive answer is “no”. The long answer is really interesting and perhaps a bit more intuitive. The Bergson quote is really profound and if you have time I highly recommend his book Matter and Memory. Bergson argues that we must have some type of context or scaffolding upon which to understand what we are seeing. Learning is about forming and connecting this complex neural network. In fact, genius is usually seen as making leaps between seemingly unrelated contexts – things that the rest of us could not comprehend until given the proper context. For example, we all understand that mass equals energy now because it’s within our mental framework (E=MC2) but 100 years ago it wouldn’t have made any sense at all. We just were not ready to comprehend it.
So, why would the man – just given sight – not be able to distinguish a sphere from a square? The most important reason is that the shapes are out of context. Since you cannot hear, smell, or taste a shape, you are left with only one modality: touch. If you are not blind, you’ve been connecting shape between the two modalities of touch and sight for your entire life. It’s impossible to imagine it being otherwise because you’ve made those connections for all time. It’s the same as the game where you are blindfolded and asked what food you are smelling. It’s very difficult even with very familiar things like bananas, oatmeal, and tomatoes because we require context and both vision and smell for identification.
I think the older we get the more difficult it is to overcome these cognitive barriers. I’m not sure 85 year old Nana will ever understand that the photo on my iPhone is the same photo she can hold in her hand – or the concept of a mobile phone at all. I’ve noticed this with other members of my family that have been slow to adopt technology changes. The first time I showed them a newspaper on an iPad it’s as if they didn’t quite “get it”. But over time, they rewired their tactile and visual neural network to accommodate for this shift of medium. Of course, children pick this up instantly because it’s all new to them and their brains are still amazingly plastic.
One practical implication is for the designers of new technology – especially human interface hardware. As our senses become more closely melded with digital hardware, the harder it will be to undo those relationships. Teachers already complain that without Google students wouldn’t know anything. Without our phones we wouldn’t know anyone’s phone number. When these services are integrated into a contact lens or implanted into our corneas, the (e)book will have to be rewritten on how our brain processes and links up our perceptions into thoughts and sensations.
When I’m 85 and my grandson pops a contact lens in my eye and takes me back into a cherished virtual reality memory with such realism that I can taste, touch, see, smell, and hear as if I was really there again, I wonder if I will even recognize what’s going on. I’ll probably ask if he has the photo on an iPhone.
