You Can Play 20 Questions with Nature and Win

• You can play 20 questions with nature and win: Categorical versus coordinate spatial relations as a case study
• Stephen M. Kosslyn

Intro

• Alan Newell famously asserted that “You can’t play 20 questions with nature and win” (Newell, A. (1973)
• focused on the futility of studying binary distinctions
• However, the distinction between categorical and coordinate spatial relations representations has turned out to be fruitful
• First, from the outset this distinction was cast within the context of a theory of a more general processing system; second, it was formulated from the perspective of multiple levels of analysis within a processing system, and thereby bridges characteristics of information processing with characteristics of the brain.
• In the game of 20 questions, one player thinks of an object or situation, and the others attempt to guess it by asking a series of binary questions: is it living? is it an animal? is it domesticated?
• Each question reduces the search space, and eventually a questioner can pounce on just the right answer
• Newell argued that this game is a bad model for how science should be conducted.
• decried the tendency of psychologists to formulate and test binary distinctions— such as those between episodic versus semantic memory, serial versus parallel search, and gradual versus all-or-none learning
• more often than not such distinctions are illusory, and after an enormous amount of research ultimately all we know is that nature resists clear-cut binary divisions.
• We should consider how to fit the available data together into a single coherent story.
• And in his view, the best way to do this is to attempt to build computer simulation models that mimic human performance.

Drawing Distinctions Within Processing Systems

• fundamental problem with most (if not all) of the binary distinctions that Newell railed against
• distinctions were formulated independently of concerns about how the putative representations or processes would operate within the context of a more general processing system

Divide-and-conquer

• complex tasks never are accomplished by a single process, all in one swoop
• most tasks are treated as if they are combinations of simpler sub-tasks, each of which is grappled with by a separate aspect of the overall processing system.
• brain has clearly divided processing of object properties, such as shape and color, from processing of spatial properties, such as location
• Location is registered by a system that processes spatial properties—the socalled “dorsal system”, which runs from the occipital lobe to posterior parietal cortex.
• Thus, the two problems (recognizing objects in different locations and being able to specify location) have contradictory requirements—and it is rather elegant that the brain deals with each in a separate system.

Weak Modularity

• The brain has numerous specialized systems
• But these systems are not “modules” of the sort proposed by Fodor (1983).
• Fodor’s modules are independent, in the sense that the workings of one cannot affect the inner workings of another
• However, given the nature of the neuroanatomy of the brain, we are better off conceptualizing processing in terms of neural networks— which may share some cortex and some types of processing.
• Moreover, we should expect “leakage” between these systems. Aspects of a theory of high-level vision
• Kosslyn & Koenig, 1992)

Aspects of High Level Vision

Visual Buffer

• visual input during perception is organized in a series of [brain areas](brain areas.md) in the occipital lobe, which I have grouped into a single function structure called the visual buffer
• These areas are topographically organized, such that the pattern of activation over the surface of the cortex (roughly) preserves the pattern of activation on the retina.
• Most of the connections among neurons in these areas are short and inhibitory
• The output from the visual buffer is a representation of edges and regions of an object

Object Properties Processing System

• Output from the visual buffer flows into the ventral system, where it is compared to stored visual memories
• If a match is found, the object (or part of an object) is recognized. Spatial properties processing system
• Output from the visual buffer also flows into the dorsal system, where location and other spatial properties are computed.

Long-term Associative Memory

• The outputs from the object properties processing and spatial properties processing systems converge on long-term associative memories
• Such memories specify the spatial relations among objects or parts of objects. A problem in vision and a possible solution
• The distinction between the ventral and dorsal systems makes sense from the perspective of the two principles briefly outlined earlier, divide-and-conquer and weak modularity
• How can the visual system identify objects when they can project an almost infinite number of images?
• no new parts are added to the image when the object is contorted in its many and varied ways, although some parts may be occluded
• Thus, if a sufficient number of individual parts can be recognized, this is a strong indication that a specific object is present
• the spatial relations between parts remain constant if they are described in a relatively abstract way

Categorical Spatial Relation

• A category is an equivalence class; for instance, if you hold one hand next to the other, the first will remain left or right of the second no matter how high, low, or far away it is from the other hand. Once assigned to the category, the spatial relations are treated as equivalent, with any differences (e.g. between a bent versus outstretched arm) ignored.
• However, the dorsal system cannot compute only categorical spatial relations representations.
• Such representations are useless for another key role of the dorsal system, namely reaching and navigation
• Knowing that a table is “in front of” you (a categorical spatial relation) will not help you walk around it, or pull your chair up to it
• In these cases you need precise metric information, and you need such
• information relative to your body, a part of your body, or relative to another object that serves as an “origin”

Coordinate Spatial Relation

• categorical spatial relations representations typically can be captured by a word or two, and the left cerebral hemisphere is better than the right at such processing
• coordinate spatial relations representations are essential for navigation, and the right cerebral hemisphere is better than the left at such processing
• In short, here is an example of a situation where 20 questions seems to be working
• At the first cut, we divided the entire system into two coarsely defined subsystems, distinguishing between the object-properties-processing ventral system and the spatial-properties-processing dorsal system
• At the second cut, we focused on the dorsal system, and now divided it into two more finely characterized subsystems, which compute categorical versus coordinate spatial relations representations.

Levels of Analysis

• based largely on that of Marr (1982), but adapted in various ways to be more appropriate for cognitive processing rather than vision per se
• a fundamental characteristic of a theory of a processing system is that it begins with an analysis of the task to be accomplished
• The theory of the computation can be conceptualized as specifying a black box, which takes a specific input and produces a specific output; this output in turn is used as input to yet other processes.
• According to Marr, whereas a theory of the computation describes what is computed, a theory of the algorithm specifies how it is computed.
• An algorithm consists of a step-by-step procedure that guarantees that a certain output will be produced on the basis of a certain input.
• Finally, algorithms are implemented in hardware (on a computer) or “wetware” (in a brain)
• The level of the implementation specifies how an algorithm is physically realized
• This observation seems particularly relevant to the encoding and use of spatial relations representations (e.g. Baciu et al., 1999; Kosslyn et al., 1998).
• Interdependence among levels
• Marr sometimes wrote as if a theory at one level of analysis could be formulated with only weak links to theories at the other levels.
• However, computations rely on algorithms, and those algorithms have to operate in
• a brain that does some things well and other things not so well
• In addition, as evolution progressed, older parts of the brain often were relatively preserved—new areas were added, but the old ones rarely were redesigned from scratch.
• Thus, the newer portions had to work with the older ones, which may not have been optimal for the final product (cf. Allman, 1999).
• characteristics at each of the levels of analysis affect theorizing at the other levels— and hence a powerful approach to theorizing about cognition requires that all three levels of analysis be considered at the same time.
• At the level of the algorithm, conceptualizing processing within the context of the larger system played a central role; the fact that object properties and spatial properties are processed separately provided a key constraint on the theory of what is computed and how such computation proceeds
• the idea that the two cerebral hemispheres would differ for the two kinds of processing not only helps to specify the nature of the representations and processes, but also offers one way to test the hypothesis.
• Leveraging multi-level theories

Why is it Important That Scientists Be Able to Play 20 Questions with Nature and Win?

• One reason is simple: cognitive processing is extraordinarily complex, and we must find ways to gain traction in studying it.
• I argue that multi-level theories, which bridge from information processing to the brain, should play a special role in playing the science game of 20 questions.
• First, they lead researchers to collect different sorts of data.
• when theorizing on the basis of such varied types of data, there are more constraints on the theory.
• Moreover, multi-level theories must respect qualitatively different sorts of constraints simultaneously
• Perhaps paradoxically, the more constraints that are available the easier it is to theorize, even though it is more difficult to fit all the constraints together within a common framework
• Newell was troubled not simply by the failure of most binary distinctions to lead to fruitful research, but also by the lack of accumulation of such results.
• He had the sense that research was not accumulating to paint a coherent overall picture, but instead isolated fragments of knowledge were being collected.
• The brain is, after all, a single organ.