Why Does Linguisitic Information Mean What It Does?

Sabrina has been working on a series of posts on an ecological analysis of language (here, here and here, plus more on the way). Her focus has been on the nature of the information for language, and the similarities and differences this information has with the information for perception. We're working some of this analysis into a paper, and writing that got me thinking about this in a little more detail.

Our main move on language is to reject the assumption that language is a qualitatively different kind of task than perception & action. The goal is to find ways to talk about these behaviours using the same basic analysis tools. Part of that is to draw the analogy to how perceptual information gets its meaning and use that to describe how linguistic information gets its meaning.

What I want to do here is just map this analogy out a little, because I ended up in an interesting place and I want feedback from people who know more than us on this about whether this is just plain crazy. In particular, if you know anything about the relationship between neural dynamics and the dynamics of speech, we think this is going to be relevant!

How perceptual information gets its meaning
When we talk about meaning, we're asking how an organism can come to learn what an information variable is information about. Perceptual information is about the underlying dynamics of the event that created the information.

What I mean by this is that events in the world can be distinguished and identified only in terms of the dynamics. A dynamical description is one that describes how something changes over time, and includes reference to the underlying forces that caused that change. A fly ball is baseball looks and acts the way it does because it is an example of the projectile motion dynamic. The dynamical equation describing projectile motion events includes terms for the size and mass of the object, the initial speed and angle, gravity, drag and air resistance, and you can use this description to plot out exactly how the position of the ball changes over time.

A dynamical event such as a fly ball creates information by interacting with, say, light; but this information is only kinematic, not dynamic. A kinematic description of an event is one that describes how something changes over time, but without reference to the underlying forces. In practical terms, this means that you can use variables like time, position, velocity and the other temporal derivatives of position but you can use variables that include mass or force. The visual perceptual information for a dynamical event is therefore a pattern in the optic array that can be described in terms of things changing over time.

It turns out that it is possible for a kinematic pattern to specify a dynamic property. What this means is that the dynamical event creates one and only one kinematic pattern as it unfolds over time. If this is the case, detecting the kinematic pattern is equivalent to perceiving the dynamic event, and this is the mechanism for direct perception of the world.

Information is all we have access to, and you never get to peek behind the curtain to check what the dynamics are up to. So in order to learn what a given kinematic pattern means, you have to use that pattern to control some action. If that pattern lets you, say, intercept a fly ball, then that pattern comes to mean the catchable-ness of the ball (the affordance). In other words, perceptual information comes to mean the dynamics of the event that created the kinematic pattern.

How linguistic information gets its meaning (the analogy)

Linguistic information is also created by a dynamic event, but a much more complicated one. Take speech (but the idea works just as well for writing and gesture). The information that is created is kinematic patterns in the acoustic array. These patterns are caused by the underlying dynamics of articulation (how the lips, tongue and vocal cords change over time). However, and this is a big however, linguistic information does not come to mean the dynamics of articulation. When you detect a pattern in the acoustic array, you don't perceive that your conversation partner's throat is up to - you perceive the meaning of the word that was produced.

Remember, the goal is to apply the analysis of how perceptual information gets its meaning to how linguistic information gets its meaning; but we've run into a mismatch. My solution is to remember that the dynamical system producing speech is actually much more than just the articulators. A critical player in speech is the brain, and one of the main reasons the articulators move the way they do is that this is what happens when you couple the neural dynamics of language to an articulation system.

The crazy notion that emerges from this analysis is that linguistic information comes to mean the dynamics of the broader system, the dynamical system formed by the coupling of language related neural dynamics to an articulation system. This means the analogy holds (the kinematic information is about an underlying dynamical event in the world).

Initial problems

For perception, events such as projectile motion have the dynamics they do because of physic (see Turvey, Shaw Reed and Mace, 1981 for the details of this analysis). The dynamics of projectile motion is simply how you describe how an object changes it's state over time when it has been fired off with an initial speed and angle and then left to do it's thing.

This is not true for language. Why do the neural dynamics have the form they do? One crude answer from applying the analogy is that they are like this because that's how you describe what an extensively trained nervous system changes it's state over time when it's producing that sentence rather than another. Obviously this isn't all that satisfactory, but it's all I have just now. 

It's also even more complicated than this, because the dynamics from which linguistic information arises also includes the conversational and social context, etc etc. It's possibly an intractable mess, although people are applying dynamical systems to all kinds of tasks these days.
So this analogy only gets us so far; but it does push the ecological analysis quite a long way into the problem, which I like.

The coupling between neural and articulation dynamics

There is apparently a bit of a literature on this (thanks Tom Hartley and Jon Brock for links). The debate in the literature right now seems to be about whether syllables can be described as oscillators. If they can be, then you can start to talk about things like coupling and entrainment between syllable production and the underlying neural oscillations you can measure in speech production. This recent paper in Frontiers in Language Sciences by Fred Cummins is skeptical but only because he thinks the syllable is the wrong place to look, I think; importantly it has links to all the key papers on this topic.We'll get into that literature at some point, but at this point I'm still trying to come to grips with this analysis and whether this literature fits it and might help us.

If anyone has any bright ideas, questions, comments, papers that might help/hinder this analysis, anything at all, let us know in the comments. This is all a work in progress!

References

Cummins, F. (2012). Oscillators and Syllables: A Cautionary Note Frontiers in Psychology, 3 DOI: 10.3389/fpsyg.2012.00364

Hartley, T. (2002) Syllabic phase: a bottom-up representation of the temporal structure of speech. Progress in Neural Processing 14 ,World Scientific: Singapore, 277-288. Download

Turvey, M. T., Shaw, R. E., Reed, E. S., Mace W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981) Cognition, 9 (3), 237-304 DOI: 10.1016/0010-0277(81)90002-0 Download