版权声明：转载时请以超链接形式标明文章原始出处和作者信息及本声明
http://dushijue.blogbus.com/logs/3259129.html

Seeing is Believing: Depictive Neuromodelling of Visual Awareness

Igor Aleksander, Helen Morton and Barry Dunmall

Intelligent and Interactive Systems Group, Department of Electrical and Electronic Engineering, Imperial College of Science, Technology and Medicine, London SW7, 2 AZ

{I.Aleksander, Helen.Morton, B.Dunmall}@ic.ac.uk

Abstract. The object of seeing is for the brain to create inner states that accurately model the world and recall it for purposeful use.  In this descriptive paper we present virtual neuro-architectures called ‘depictive’ which have been developed to create hypotheses for the mechanisms necessary for such depiction and explain some elements of verbally induced visual working memory. Early work on applications to understanding visual deficits in Parkinsons’ sufferers is included.

1           Introduction

This work discusses the development of hypotheses regarding the generation of visual awareness in the modular architectures found in the living brain. These architectures are modelled using digital neuromodels [1].  The accent is not on the neural modules but the behaviour emerging from the interaction of such modules.  As an example we work with  the perception and recall of both broad shape of a pattern and the detail which makes it up.  This is best illustrated by fig. 1.

Fig. 1.  From left to right these objects are called ‘cross’, ‘tee’ and ‘square’.  The detailed shapes from which the object is composed have both shape and colour.

These are typical figures that are used in visual working memory experiments with human beings [2].  The subject is allowed to observe one or more objects, and is then questioned about the detail in some position within the object (e.g. “What is  the top left shape of the cross?”) .  In this paper we are not concerned with the prowess of individuals in executing the task, we enquire into the possible neurological mechanisms which allow the observer to have such an ability at all.  We develop a hypothesis about this operation that involves eye movement and attention. This is based on what is known about the neuroscience of the system. An implementation of the proposed scheme is tested through neuromodelling and it is shown that the hypothesis is supported. The key questions that this procedure seeks to answer are, how does the system go from a retinotopic image to a coherent sensation of both object and the detailed makeup by its shapes?  How is the process of recall organised at the two levels of object and shape? How does attention operate in visual working memory.

2           Neuroscientific highlights of vision mechanisms

The features that can be extracted from the literature on the neuroscience of vision and that form the elements of this work feature in fig. 2 and are the following. 

-          The prime anatomical area that causes eye movement whether saccadic or smooth is the superior colliculus.  ([3] is a good reference for pursuing this material).

-          The superior colliculus receives direct input from the retina which is capable of causing movement to direct the fovea to areas of high contrast in the in the retinotopic image (including the perifoveal fields).

-          The image pathways diverge into specialisms through primary areasV1 into V2, V3 etc.. 

-          Importantly, in the parietal cortex, the prefrontal and the frontal cortex (broadly dubbed ‘the extrastriate cortex’ following the practice in [5]), there are projections from the motor activity which includes that caused by the superior colliculus. 

-          A most important piece of evidence is the discovery of cells that, on receiving positional signals originating from the muscular output, only fire for specific positions of the eyes.  These are called ‘gaze-locked’ and their presence in areas such as V3 has been known for over ten years [4]. 

-          On the question of memory, it is generally proposed that prefrontal structures which are responsible for perception are also involved in the process of visual memory. Fig. 2 summarises this structure and indicates what needs to be built in a simulation.

Pathway X is an important feature of the hypotheses set out in this paper as it serves several purposes.  The salient among these is that higher visual awareness, postulated to take place in the extrastriate areas [5] (as the main ‘purpose’ of these areas) also influences the superior colliculus. For example, this is thought to be at work in context-dependent tasks such as face exploration or visual planning as may be involved in solving stacking problems “in one’s head” [6]. The further postulated function of this pathway is discussed in part 4 of this paper.

Fig. 2.  A monocular diagram that summarises the neurophysiological data available for neuromodelling 

3.       The Depictive Hypothesis

The material of this section is an elaboration of a previously enunciated hypothesis about the asynchrony of visual awareness [7].

-          Any sensory activity which has the potential of being reported (whether the agent can find the actual verbal expression or not) must be supported by rich neural activity which is at least on a one-one basis with world events that the organism is aware of. We call such neural support a depiction. (See Appendix A for a brief elaboration of this argument)

-          Such depictions can only occur where there are gaze-locked cells as other areas cannot be compensated for eye and other (e.g., head,) movements. We call the vector of all gaze-locking information ‘referent j’.

-           Neurons in areas receiving j need not be physically adjacent in order to achieve binding.  For example a shape sensitive neuron and a colour sensitive neuron will encode information about the same element of the external visual world if they are indexed by the same values of j. We note that this is a a resolution of the extreme postions on binding existing between the binding theory of Crick and Koch [5] which espouses the existence of tuned 40 Hz signals between binding neurons, and the hypothesis by  Zeki and Bartels [8] that there is no need for binding and that the elements of a visual sensation arise asynchronously (as microconsciousnesses).

-          We note that the area of depiction can be defined as the total area of visual awareness available to the observer at any point in time. We further note that, in reality, this involves not only eye and head movement but bodily movement as well. 

In this paper we consider eye movement alone to establish how this contributes both to depiction and visual working memory. This opens the way to the study of other motor activity in a similar way.

4.       The Visual Memory Hypothesis

The literature on visual working memory (e.g.[2])  clearly points to the fact that recall is intimately bound up with attention. It can relate to both broad and detailed reports of previous visual experience.  Attention could be said to be an internalisation of the movement process, and it is a simulation of this which is at the focal centre of this paper.

-          The feedback loop shown in fig.2  involving the superior colliculus, the gaze-locking path to the extrastriate, and the return to the superior colliculus via pathway X forms a learning state machine which allows for the inward exploration of a visual scene  .

-          Recall of both detail and shape of  images such as those shown in fig.1 is achieved by the retention of the gaze locking signals as a subset of the state variables of this state machine.

-          In the absence of visual input the state machine becomes autonomous and generates gaze-locking signals which now become described as inner attentional control patterns.

-          Both foveal and perifoveal information is therefore retained. The state machine when triggered by auditory input (e.g. ‘imagine the tee shape’) is able to imagine the overall perifoveal shape, and highlight the detail of it through inner gaze locking to the extent that the detail can be decoded and output as a voice signal.

5.        The experimental system

Fig. 3.  The NRM virtual vision system during perception (only 14 of the 24 areas are shown for clarity). Every dot is the output of a neuron. Input 1 is the visual world containing the foveal and perifoveal areas, 6 and 7 are parts of the superior colliculus, while 1,5,and 13 are the primary areas. 24 is the crucial extrastriate ‘awareness’ area.

Without going into detail, a simulation comprising 24 neural areas and a total of approximately 360,000 neurons has been set up using the Neural Representation Modeller (www.sonnet.co.uk/nts).  Only 14 of these areas are shown in figure 3. Here the superior colliculus, under the influence of perifoveal inputs is controlling the movement of the ‘eye’ in order to highlight and identify verbally both the overall shape (20) and the foveal detail (16).

 Fig. 4.  This shows the NRM system in visual memory mode.

Fig. 4 shows the behaviour of the system where there is no visual input (blindfold or eyes closed) but it is asked to imagine the ‘square’(at 21).  There is no activity in the primary areas but the superior colliculus activity is sustained in the loop by pathway X.  24 shows that there is a vague awareness of the whole shape, but attention as mediated by the superior colliculus and gaze locking enables the detail (bottom right of 24) to be described in 16 (this appears as ‘Green Dee’ as the experiment is being done in colour.

6.      Application: Visuo-Cognitive Deficits in Parkinson’s Disease

It has been shown through measurements of eye movements that there is a distinct difference between unaffected subjects and PD affected ones in the strategic use of such movements in the solving of visual problems [6].  Unaffected subjects move their eyes to imaginary targets where objects have to be ‘parked’ temporarily in order to find a suitable sequence of moves that solves the problem.  It is well known that PD is the result of dopamine deficit in the substantia nigra, which directly affects the basal ganglia which, in turn, have strong inhibitory synaptic connections to the superior colliculus. It is also known that hallucinations are not uncommon in PD sufferers. This suggests that the feedback loop discussed in this paper may be prone to falling into hallucinatory state-space minima and be less responsive to retinal signals as a result the reduced inhibition from the basal ganglia in the superior colliculus.  We are initiating discussions with PD sufferers to obtain an understanding of their visualization difficulties in solving visual puzzles, and using the model discussed in this paper to attempt to predict (in area 24 of the model) what distortions PD-afflicted people might suffer.

7.      Conclusion and Directions of this Work

Visual awareness has both perceptual and imagining components.  In this paper we have proposed a modelling technique which addresses hypotheses about the sources and of both of these elements.  The work is based on a depictive hypothesis of very rich neural activity that is world-centered, making perception (neural activity in  part of the extrastriate areas) world-representative depending on eye movement and its compensation in creating the depiction.  Central to this operation is the action of the superior colliculus and its action on gaze-locked cells that ensure awareness in extra-striate areas of the cortex.. It has been argued that visual working memory is due to sustained feedback activity which involves the superior colliculus and the extrastriate cortex to the extent that gaze-locking information becomes encoded in the feedback variables.

We note the distinct departure in this work from classical models of visual working memory in the cognitive sciences (e.g. [2]).  These classical descriptions invoke computational terms such as ‘central executive’, ‘buffer memory’, ‘circulatory registers’, ‘attention window’ and so on.  Of course such components do not exist in this computational form in  the brain, they are merely descriptions of functions or algorithms that a computer would need to have visual working memory.  The process of depictive neuromodelling creates models which are based on existing elements of neural processes such as the superior colliculus, and the other areas involved in the specific model discussed here.  These have emergent stability and they contribute to explanations of real functions within the brain. This is particularly important when creating hypotheses that attempt to achieve a better understanding of departures from the normal, as was suggested above in the case of Parkinson’s disease.

Much work remains to be done on visual working memory, in particular when more complex verbal descriptions of events are involved. This is the subject of current work.

8.    Acknowledgements

We wish to thank the Wellcome Trust for granting us a Wellcome Showcase Award for the development of this work in the area of investigating mental deficits.  The invaluable contribution of our collaborators in the Neurosciences division of the Imperial College School of Medicine, namely Chris Kennard and Tim Hodgson, is much appreciated.

Appendix A

Depiction and coherence: To support the hypothesis of depiction, we define an elemental event of visual awareness as the change in the visual world with the smallest geometrical dimensions that the observer can report.  A tiny fly landing on a wall may be an example of such an event. Were the fly smaller it would not be seen.  This change must cause an elemental change in the activity of at least one neuron otherwise, without which, it could not be seen.  The same can be said for a second elemental event in the visual world an so on by induction leading to the conclusion that neural events stand in at least one-one relation to visual events in the external world. The implication of this is the support for many physiological findings that accurate depiction occurs in the extrastriate parts of the brain. Note that depiction does not imply ‘pictures in the head’ that could be discovered by a super-accurate brain scanner.  The elemental events could be arbitrarily distributed in a geometrical sense.  It is also known that the same event is broken down in V1 and partly depicted in deeper areas of the cortex, such as V3, V4 .. etc.. Therefore the ‘at least one-one’ description is stressed. The coherence of the experience is guaranteed by the hypothesized referent j .

References

1.        Aleksander, I., Dunmall, B.,  Del Frate, V., Neurocomputational Models of Visualisation: A preliminary report,  Proc. IWANN 99, Alicante – Spain (1999)

2.        Logie, R. H., Visuo-spatial Processing in Working Memory.  Quarterly Journal of Experimental Psychology, 38A (1986) 229-247

3.        Kandel, E. R., Schwartz, J. H. and Jessel, T. M.,  Principles of Neural Science, Fourth Edition, McGraw Hill, (2000).

4.        Galletti, C. and Battaglini, P. P., Gaze-Dependent Visual Neurons in Area V3A of Monkey Prestriate Cortex.  Journal of Neuroscience, 6, (1989), 1112-1125

5.        Crick, F., and Koch, C., Are we Aware of Neural Activity in Primary Visual Cortex? Nature, 375, (1995)

6.        Hodgson, T.L.,  Dittrich, W. H., Henderson, L and Kennard, C.,  Neuropsychologia, 37: 8, (1999), 927-938

7.        Aleksander, I. and Dunmall, B.:  An extention  to the hypothesis of the asynchrony of visual consciousness. Proc R Soc Lond B 267, (2000)  197-200

8.        Zeki, S., and Bartels, A., The asynchrony of consciousness. Proc R Soc Lond B 265, (2000) 1583-1585.