The Nature of Ecological Perceptual Information

The central issue in the perception of the world is how the perceptual system comes into contact with that world. In vision, the assumption for centuries has been that the experienced world must be recovered somehow from the 2D image of the world projected onto the retina by the lens of the eye. Scientists therefore hunted for patterns on the retina that preserved critical structure from the world by copying some part of that critical structure. 

James J Gibson was, for a long time, no exception to this hunt. His early empirical work (grounded in the theory he laid out in The Perception of the Visual World; Gibson, 1950) created and manipulated retinal images that, for example, contained gradients of optical texture that matched gradients of physical texture created as surfaces receded in depth, or changed their shape or orientation relative to a point of observation. But time and again, Gibson found that perceptual experience was not any straight-forward function of retinal stimulation (i.e. sensations). People did not ‘see’ what was on the retina (Reed, 1988). The most powerful demonstration of this fact is Gibson’s analysis of dynamic occlusion (Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969) to which we will return below.

Gibson’s later career was defined by the search for an explanation of how perception could be possible if it wasn’t based on sensations and retinal images. Gibson’s solution was his theory of the ecological information available for visual perception published in The Ecological Approach to Visual Perception (Gibson, 1979). 

What follows is a description of ecological information with reference to some of Gibson’s work and the extensive research literature that has taken place since Gibson’s death in 1979. The focus will be on the information in light for vision, because that has been the focus of the research. However, the principles hold for all the various energy media our perceptual systems interact with, and we will review this briefer evidence as well.

What Kind of Thing is Information?

Most of the environment is ‘over there’, i.e. not in mechanical contact with our bodies. Our behaviour, however, must still be shaped with respect to relevant properties of that environment in order to be functional. Organisms get to interact with patterns in in various energy media (e.g. light for vision) and have no choice but to use these patterns as stand-ins for the environment. This only works out well if the patterns are adequate stand-ins. 

The question that therefore confronts us first is this: what kind of patterns can these energy media provide an organism, and what can those patterns be informative about? The answers, respectively, are kinematic specification of dynamics and an extraordinarily wide variety of behaviourally relevant properties. In order to understand this idea, we need to have a way to characterise both the world and the information it creates.

The World is Dynamic

Objects and events in the world are best described at the level of dynamics. It is only at this level that categories of objects and events can be uniquely identified (Bingham, 1995; Wilson & Bingham, 2001). This simply means that a comprehensive mathematical representation of some object or event will require units of time, length but also mass. 

A dynamical description of an event type is an equation that includes a specific set of variables placed in a particular relation to each other using operators such as addition, multiplication, etc. This equation describes the characteristic form a category of events take as it unfolds over space and time. You then create one particular instance of an event type by setting parameters on those variables. Two throws, for example, are both examples of the dynamic event category "projectile motion" even if they differ in their parameters, e.g. varying release angle, speed or height (Wilson et al, 2016, in press).

This establishes the goal of perception. In order to functionally interact with the world, an organism has to identify (Bingham, 1995) the task at hand, where tasks are the current dynamical state of the world. Perception must therefore provide access to information about those dynamics, so that the organism can set about the business of learning how to complement those dynamics with its own. 

We can now turn to the nature of perceptual information, to see if it is up to the task. I will first discuss this in terms of vision and light, but the principles will apply to other media such as sound. 

Information is Kinematic

The dynamics of the world are typically ‘over there’. Fortunately for us, these dynamics interact with various energy media and project themselves as patterns in that energy. For example, light emitted from a source hits surfaces and bounces off them, changed by the interaction. The interactions are governed by ecological laws (Turvey et al, 1981) and through this interaction radiant light becomes structured ambient light (Gibson, 1979, Chapter 5 (Parts One, Two, Three). Structure simply means that the field of energy is no longer symmetrical; there are variations in density, intensity, etc that make each given view point unique. The field has become an array. 

The first draft hypothesis is therefore this: the form of the array is caused by the lawful (and hence reliable) interactions of the energy with the dynamical properties of the local environment (the task dynamics) and this form might therefore be informative about those dynamics. 

There is an immediate problem, however. The patterns projected into energy media are kinematic, not dynamic. Specifically, a complete description requires only units of time and length, but not mass (Bingham, 1988; Runeson & Frykholm, 1983; Turvey et al, 1981). In order to solve the identification problem (Bingham, 1988, 1995) the dynamics of the task have to get projected in such a way as to preserve the identity of the task. Gibson’s early research demonstrated again and again that this mapping was not achieved by creating a retinal image that was a copy of the dynamics, and the analysis of the world as dynamic and information as kinematic formalises the idea that copying isn’t possible anyway. 

Gibson’s central hypothesis therefore becomes this. 

1. In order for perception to work, arrays must become information. 
2. To be information, arrays must be specific to dynamics (i.e. map 1:1) but this specification must be possible when the two things are not identical. 
3. In addition, if it is possible, then these arrays must also be shown to be used as information by organisms. 

This, in a nutshell, defines the research programme of the ecological approach: identify the behaviourally relevant task dynamics, identify specifying information variables created by those dynamics, and test organisms to see whether that information is being used to perceive the world (see also Wilson & Golonka, 2013). We will next review some of the evidence that the kinematic specification of dynamics (KSD; Runseon & Fryholm, 1983) is not only an option, but indeed the way organisms interact with their worlds. 

Further Reading

Task Dynamics And The Information They Create
What Kind of Thing is an Information Variable? The (Annoying) Case of TauSpecification and Some of Its Consequences