This book bridges the gap between cognitive task models of human behavior and neural network models. Traditionally, cognitive modelers have assumed that human memory is a kind of storage place. Symbol networks are assumed to be stored in long-term memory, a kind of separate place in the brain like the stacks in a library. Alternative approaches, based on models of neural activation, assume instead that memory is distributed and changed by the very process of being used. How are these two ideas to be reconciled? The synthesis presented by this book begins not with symbols or neurons or networks, but with sequences of coordinated behavior. The assumption is that 'what the brain remembers' is how to coordinate behavior in time, relating different modalities of perception and movement. Secondarily, these sequences are categorized, such that at a first order we have perceptual categorizations (sounds, images) and at a second order we have conceptual categorizations of perceptions and actions in time. Conceptual categorizations are themselves sequenced and categorized, corresponding to the familiar classification hierarchies in cognitive models.
I do not argue that any of the other approaches are wrong per se, but are incomplete and not properly related to each other. For example, in the traditional symbolic approach of cognitive modeling, there is a tendency to assume that neural memory operates like the constructs in programming languages. By this view, the abilities to perform a sequence of behavior (a procedure), to manipulate concepts abstractly as variables, and to order procedures in time are taken for granted; they are part of the architecture and are not explained. Consequently, processes by which substitution, generalization, and composition occur are not investigated. Experienced phenomenon in which such processes become evident, such as slips and limitations of grammatical comprehension, are either not closely examined or explained like Ptolemy's epicycles on top of a symbol-centric perspective.
On the other hand, many attempts at neural modeling do not address complex cognition, such as problem solving or analogy formation, or do so without relating broader contexts of purpose and personal identity. Some attempts repeat the same flaw as cognitive studies, in attempting to "program" neural networks to behave like programming languages.
I produced this book to respond primarily to the criticism that situated cognition theories ignore how the brain works or reject the representational theory of mind. Inspired by Bartlett's work, I sought to develop a theory of "process memory"a memory for experience in time. Following the methodology of situated cognition, I sought clues in the particulars of human activity, such as typing errors, how a computer interface is used, how a child learns to play in a pool, odd limitations in language comprehension, and so on. Throughout, I examine existing (and often famous) cognitive and neural models with respect to these phenomena. In each case, I show that the experienced behavior can be understood as sequences of perceptual categories being reactivated, substituted, and composed. On this basis, I provide a more parsimonious account of slips (based on just these operations), I relate the generative and symbolic views of analogy formation, and provide a simple explanation of a well-known pattern in language comprehension.
The present book is far from the last word. I do not provide a computer implementation of the memory. I argue instead that the kind of analysis and synthesis I present is required if we are to realize how existing models are bound by assumptions that are constraining computer systems to be less flexible and less creative than people. The surprising claim of this book is that we do not fully appreciate what we have accomplished in our computer systems because we do not understand well enough how the brain works. By having inadvertently identified the brain's memory operations with our computer models, we have failed to appreciate what computer systems can do better than people and what aspects of human learning are yet to be realized as computer systems. In effect, we have both understated what we have accomplished in artificial intelligence and poorly grasped the work remaining to be done.
--Shows that consciousness can be understood as an extended aspect of physical coordination, involving categorization of "what I am doing now," which orders and regulates perceptual motor activity. Consistently treats knowledge as "the mind in motion," as a dynamic process, as an ongoing process of ordering, subsuming, and segmenting our experience in time.
--Analyzes well-known symbolic models to show value, but extends them to fit what is happening at neural level:
--Historical view, tied to recent theories of constructive memory, with explanation of a variety of related theories of the brain (Bartlett, Kauffman, Edelman, Domascio).
--Uses phenomenological analysis to evaluate and critique computer models (Part II and Chapter 10)
--Goes beyond situated cognition analyses to consider neural mechanisms: