What is the place of language in ecological psychology? Is language a type of direct perception? Is language comprehension direct perception? Does language have affordances?
In trying to answer these questions I discovered that some things we think of as being perceptual have a lot in common with the conventionality of language and that some language-related behaviours look a lot like perception (as typically construed). I end up suggesting that we move away from talking about 'perception' and 'language' as different types of entities and instead focus on information / behavior relations in specific tasks.
In general, ecological language people are happy to talk about language as having affordances. Two papers from the 2010 special issue of ecological psychology on Ecological language even have the word in their titles ("New affordances for language: distributed, dynamical, and dialogical resources" Hodges & Fowler, 2010; "Speech as the perception of affordances" Worgan & Moore, 2010). However, there is a long-standing awareness of the fact that language challenges some important aspects of ecological psychology. As Fowler (1986 p 24) says:
"As to...whether a linguistic message can be said to be perceived...from a direct-realist perspective, direct perception depends on a necessary relation between structure in information media and its distal source. But,...this does not appear to apply to the relation between sign and significance."
What Fowler meant was that because the relation between linguistic structures (e.g., spoken words) and how they structure behavior is based on convention (not natural laws) they cannot support direct perception. It is clear that language does not fit cleanly into an ecological framework as currently defined. I argue that, rather than vaguely using the vocabulary of ecological psychology to describe language, it's crucial to identify precisely where the similarities and differences are between language and perception. I want to preserve the theoretical rigor of ecological psychology.
The goal of the following analysis is to devise an extended framework for ecological information that accommodates language without straining or redefining original notions of affordances or direct-perception.
Comparing language and perception
To begin, let's sum up the basic ecological approach to perception as conceived of by Gibson (1979) and reinforced by Turvey et al (1981).
1) Affordances are dispositional properties of objects and events that provide opportunities for action to complementary organisms
2) Perception is of affordances
3) Affordances uniquely structure energy (specification)
4) Therefore, coordinating behavior wrt information is equivalent to coordinating behavior wrt the affordance property (direct perception)
Illustration of typical perception case:
Illustration of typical language case:
In the perception case, direct perception possible because the task (tree climbing) was about the property of the world that created the information. Behaviour was coordinated WRT the property of the world that caused the information.
In the language example, however, the task is not about the property of the world that created the information. Behaviour is organised according to conventions constrained by culture, linguistic context, etc. If we look closely at the types of behavior that might arise as a result of receiving the verbal instruction "Turn left at the corner" something interesting emerges. The instructions can have a consequence on our actions - we can initiate a left turn at the corner instead of going straight or right. Or, we might respond, "No, I'll go straight because it's faster" or something similar. Or, we might ignore it altogether (although ignoring such a statement in this kind of conversation is very unusual in our culture). The point is, whatever consequence the information has, we definitely won't be using it to control our left turn. The turn itself will be controlled by good-old-fashioned perceptual information. "Turn left at the corner" can influence action selecting, but it can't be used to control the action. I've intentionally used "action selecting" rather than "action selection", even though "action selection" is the accepted phrase in the literature. "Selection" implies that the information is evaluated by the system, which then selects the most appropriate action. This implication should be avoided. "Selecting" implies that it is the information, itself, that selects the action by interacting with an embodied, extended cognitive system. More on this later.
Turvey et al (1981) said:
"Our strategy, as proponents of Gibson's ecological approach, is to argue for a conception of natural law that allows meaningful relations between organism and environment to hold. Further, we constrain our use of the term 'perception' (and thus, of course, 'direct perception') to relations governed by such laws."
The view of language that I propose reserves these terms for cases where these requirements are clearly met. Does this mean that language is outside the remit of ecological psychology? No, because for Gibson, the primary thing is information. And, Gibson (1979 p 255) offers a useful suggestion for how to progress our understanding of things like language.
"Perhaps if [so-called higher mental processes] are reconsidered in relation to ecological perceiving they will begin to sort themselves out in a new and reasonable way that fits with the evidence."
We've identified a key difference between perception and (at least some) language-related behaviours. Bearing this in mind, let's now go back to the beginning - which, for Gibson is that the environment is informative - and build up from there.
Re-analysis of information
1) Objects and events structure energy
2) These structures are lawfully related to the properties in the world that cause them
3) Information is any structure in energy that has a consequence for behaviour
4) Not all behaviours are organised wrt the property of the world that caused the information
5) Nervous systems are for linking information and behaviour
Point 4 simply states the difference identified between the perception and language examples above. The first 4 statements broaden the field of information. The 5th statement claims that linking information and behavior is precisely what we are built to do - variations in nervous system type and complexity can be understood in terms of their consequences for linking information and behavior.
Perception / action is a particular type of information / behavior relation. As I intend to show, calling some types of things 'perception' and other types of things 'language' obscures important characteristics that more clearly define information types. This means that the key objective is to accurately characterise relationships between information and behavior. What factors influence the learning of various information / behavior relations? What type of behaviours is a type of information capable of supporting? What types of information is a particular organism capable of detecting or coordinating its behavior WRT?
As a starting point, I've identified the following factors as important.
Evolutionary factors
- the types of energy an organism is sensitive to
- biases for an organism to pick up certain types of information
- predetermined or strongly constrained relations (e.g., reflexes / instincts)
- sophistication of nervous system - ability to learn new information / behavior relations, to use information of varying degrees of reliability, stability
- must the organism learn to detect the structure?
- must the organism learn to organize its beahviour WRT the structure?
- is the behavior organised WRT the property causing the structure or convention?
- how stable / available / reliable is the structure?
- is the information used in action selecting or in the continuous control of action?
The example of a human infant learning to walk is a paradigmatic perception / action task. All typical perception / action tasks should have similar answers to the ones above.
Not all language-related behaviours are cases of perception / action, though. As we saw earlier, responding to the instructions "Turn left at the corner" involves organising behavior according to convention. This type of information influences action selecting, but it cannot be used in the continuous control of action.
Interestingly, producing the phrase "Turn left at the corner" looks a lot more like a perception / action task. Whether the task involves action selecting or control appears to depend on the level of analysis. At the phoneme level, it seems to be about action-selecting (this is based on a talk at the conference by Brian Gick which seemed to show that, at least, some of these units were produced with only feedforward control). I don't know a whole lot about this area, but it's worth identifying that the level of analysis matters here.
This example focuses on bees who observe the Waggle Dance and then search for food in a particular direction as a result. This is the first example where the answer to the first two questions is 'no' and it shows the work that evolution can do in constraining conventional information. Still, this conventional information is only involved in action selecting, not in online control.
Other behaviours that do not require learning to coordinate behavior wrt the structure include reflexes and (some) instincts. Reflexes and instincts differ in that information for reflexes is for action selecting and information for instincts is for action control.