I was listening to a talk by Alan Watts about some aspect of Eastern mysticism. I can’t remember the exact context. I think he was describing the impossibility of truly understanding the nature of one’s own mind. He said that trying to use one’s mind to understand one’s own mind was ‘like pointing the camera at the monitor’.
I was immediately struck by this. Partly I was surprised at his using such a simile, which involves common enough concepts in 2017, in a talk that he gave in the sixties, when computers only existed in large research establishments and occupied enormous rooms. There was certainly no such thing as a webcam back then. I realised later that he probably had in mind a closed-circuit television arrangement, which they did have in the sixties.
But beyond that, I was struck by the fact that it’s actually a very interesting question – what does happen when one points the camera at the monitor? It’s a classically self-referential problem. But unlike some self-referential problems, like the question of the truth of the statement ‘This sentence is false’, it must have a precise answer, because we can point a camera at a monitor, and when we do that the monitor must show something. But what will it show?
There are a number of practical considerations that can lead us towards different types of answers. While each of those considerations leads to an interesting problem in its own right, I tried to remove as many of them as possible to make the problem as close to ‘ideal’ as I could. So here it is.
Imagine we have a computer connected to a monitor and a digital video camera. A webcam is a digital video camera but, since the camera we are imagining here needs to be extremely accurate, a high-quality professional video camera would be more suitable. The monitor uses a rectangular array of display pixels to display an image and the camera uses a sensor that is a rectangular array of light-sensitive pixels, and the dimensions of the display and the sensor, in pixels (not in millimetres) are identical.i
On the computer we run a program that shows the image recorded by the camera. The telecommunication program Skype is a well-known such program that can do that, amongst other things. There are also dedicated camera-only programs, which webcam manufacturers typically include on a CD bundled with the webcams they sell. Let’s call our program CamView (not a real program name). We start up CamView on the computer in a non-maximised window, which we’ll call the ‘CamView window’. Then we turn the camera on and point it at the monitor. We aim and focus the camera so precisely that an image of the display area of the monitor fills the image-display area of the CamView window. Ideally this would mean that each pixel on the camera’s sensor is recording an image of the corresponding pixel on the monitor screen. In practice there will be some distortion, but we’ll ignore that for now.
Question 1: what does the monitor show?
Question 2: Next we maximise the CamView window. What does the monitor show now?
Those questions are easy enough to answer, when we remember that the window for any computer program, in default mode, typically has an upper border with tool icons on it, a lower border with status info on it, and sometimes left or right borders as well.
These questions are fairly similar to the question of what one sees when one stands between two parallel, opposing mirrors, as is the case in some lifts (elevators).
Now comes the hard one. In most video-viewing computer programs there is an icon that, upon clicking, maximises the window and removes all borders so that the image-display area occupies the entire display area of the monitor. Call it the ‘full screen icon’ and say that we are in ‘full screen mode’ after it is clicked – until a command is given that terminates that mode and returns to the default mode – ie restores the borders etc. In full screen mode the display area of the monitor corresponds exactly to the images recorded by the camera’s sensor.
Question 3: We now click the full screen icon. Describe what appears on the monitor, and how it changes, from the instant before the icon is clicked, until ten minutes after clicking it – assuming the program remains in full screen mode for that entire time.
That is the difficult one. It took me a while to figure it out, and I was surprised by the answer. It is possible that what I worked out was wrong. If so, I hope that someone will point that out to me.
I have one more question, and it has an even more peculiar answer – one that I found quite charming.
Question 4: Assume the camera is mounted on a very stable tripod. Still in full-screen mode, we pan the camera to the right until it no longer shows any of the monitor. Then we pan the camera back at a constant speed until it again sees only the display area of the monitor, and we stop the panning at that point. What is visible on the monitor after the camera has panned back to the original position? Does that change subsequently? What does it look like ten minutes later? Does the monitor image depend on the panning speed, or on the number of frames per second the camera shoots? If so, how?
In order to avoid spoiling anybody’s fun in trying to work out the answers to these puzzles for themself, I will not post answers now. I will post them a little later on. It will also take me a little while to make some nice pictures to help explain what I am talking about.
Bondi Junction, February 2017
i Although most camera sensors have a 3:2 aspect ratio, which is different from the 16:9 aspect ratio of most modern computer monitors, it is possible on a sophisticated camera to alter the aspect ratio to 16:9, which is achieved by deactivating the sensor pixels in an upper and lower band of the sensor, so that the area used to record an image has the required aspect ratio. We’ll assume that is done and that the number of pixels in the active sensor area equals that on the monitor.