Within the microcosm of information theory, I explore what it means for a system to be functionally irreducible. This is operationalized as quantifying the extent to which cooperative or \"synergistic\" effects enable random variables X1, ... , Xn to predict (have mutual information about) a single target random variable Y . In Chapter 1, we introduce the problem with some emblematic examples. In Chapter 2, we show how six different measures from the existing literature fail to quantify this notion of synergistic mutual information. In Chapter 3 we take a step towards a measure of synergy which yields the first nontrivial lowerbound on synergistic mutual information. In Chapter 4, we find that synergy is but the weakest notion of a broader concept of irreducibility. In Chapter 5, we apply our results from Chapters 3 and 4 towards grounding Giulio Tononi\u2019s ambitious \u03c6 measure which attempts to quantify the magnitude of consciousness experience.
" }, { "name": "Mo, Chunhui", "degree": "PhD", "year": "2003", "title": "Synaptic Learning Rules for Local Synaptic Interactions: Theory and Application to Direction Selectivity", "advisor": "Koch, Christof", "url": "https://resolver.caltech.edu/CaltechETD:etd-05222003-170638", "creators": [ { "name": { "family": "Mo", "given": "Chunhui" }, "id": "Mo-Chunhui", "display_name": "Mo, Chunhui" } ], "advisors": [ { "name": { "family": "Koch", "given": "Christof" }, "id": "Koch-C", "orcid": "0000-0001-6482-8067", "role": "advisor", "display_name": "Koch, Christof" } ], "committee": [ { "name": { "family": "Fraser", "given": "Scott E." }, "id": "Fraser-S-E", "orcid": "0000-0002-5377-0223", "role": "chair", "display_name": "Fraser, Scott E." }, { "name": { "family": "Koch", "given": "Christof" }, "id": "Koch-C", "orcid": "0000-0001-6482-8067", "role": "member", "display_name": "Koch, Christof" }, { "name": { "family": "Schuman", "given": "Erin Margaret" }, "id": "Schuman-E-M", "orcid": "0000-0002-7053-1005", "role": "member", "display_name": "Schuman, Erin Margaret" }, { "name": { "family": "Laurent", "given": "Gilles J." }, "id": "Laurent-G-J", "orcid": "0000-0002-2296-114X", "role": "member", "display_name": "Laurent, Gilles J." }, { "name": { "family": "Quartz", "given": "Steven R." }, "id": "Quartz-S-R", "role": "member", "display_name": "Quartz, Steven R." } ], "option_major": [ "biochem" ], "doi": "10.7907/EAK1-BQ32", "abstract": "This thesis is organized in two parts, both concerned with local synaptic interactions within the dendritic tree. The first part is focused on how specific synaptic arrangements that can be used to compute direct ion selectivity can be learned in an unsupervised manner. The second part consists of a double synaptic veto model that can account for the observed reverse-phi selectivity of direction-selective cells. We propose an activity-based, local learning model that may account for the direction selectivity in neurons in the visual cortex based on the local veto operation among excitation and inhibition. We implement the learning rule with local calcium concentration changes and a BCM type learning curve (Bienenstock, Cooper and Munro, 1982). Our biophysical simulations suggest that a model cell implementing our learning algorithm develops direction selectivity organically after unsupervised training. The learning rule is also applicable to cells with multiple direction-selective subunits on dendrites and is stable under a number of starting conditions.
\r\n\r\nReverse-phi motion is the illusory reversal of perceived direction of movement when the stimulus contrast is reversed in successive frames. Livingstone (2000) showed that direction-selective cells in striate cortex of the alert macaque monkey showed reversed excitatory and inhibitory regions when two different contrast bars were flashed sequentially during a two-bar interaction analysis. We carry out detailed biophysical simulations of a direction-selective cell model implementing a synaptic shunting scheme. Our results suggest that a simple synaptic-veto mechanism with strong direction selectivity for normal motion cannot account for the observed reverse phi-motion effect. A direct interaction between the ON and OFF pathway, missing in the original shunting-inhibition model, is essential to account for the reversal of response. We propose a double synaptic-veto mechanism in which ON excitatory synapses are gated by both delayed ON inhibition at their null side and by delayed OFF inhibition at their preferred side. The converse applies to OFF excitatory synapses. Mapping this scheme onto the dendrites of a direction-selective neuron permits the model to respond best to normal motion in its preferred direction and to reverse-phi motion in its null direction.
" }, { "name": "Itti, Laurent", "degree": "PhD", "year": "2000", "title": "Models of Bottom- Up and Top-Down Visual Attention", "advisor": "Koch, Christof", "url": "https://resolver.caltech.edu/CaltechETD:etd-12022005-103530", "creators": [ { "name": { "family": "Itti", "given": "Laurent" }, "id": "Itti-Laurent", "orcid": "0000-0002-0168-2977", "display_name": "Itti, Laurent" } ], "advisors": [ { "name": { "family": "Koch", "given": "Christof" }, "id": "Koch-C", "orcid": "0000-0001-6482-8067", "role": "advisor", "display_name": "Koch, Christof" } ], "committee": [ { "name": { "family": "Koch", "given": "Christof" }, "id": "Koch-C", "orcid": "0000-0001-6482-8067", "role": "chair", "display_name": "Koch, Christof" }, { "name": { "family": "Psaltis", "given": "Demetri" }, "id": "Psaltis-D", "orcid": "0000-0003-4684-8800", "role": "member", "display_name": "Psaltis, Demetri" }, { "name": { "family": "Perona", "given": "Pietro" }, "id": "Perona-P", "orcid": "0000-0002-7583-5809", "role": "member", "display_name": "Perona, Pietro" }, { "name": { "family": "Andersen", "given": "Richard A." }, "id": "Andersen-R-A", "orcid": "0000-0002-7947-0472", "role": "member", "display_name": "Andersen, Richard A." }, { "name": { "family": "Shimojo", "given": "Shinsuke" }, "id": "Shimojo-S", "orcid": "0000-0002-1290-5232", "role": "member", "display_name": "Shimojo, Shinsuke" } ], "option_major": [ "cns" ], "doi": "10.7907/MD7V-NE41", "abstract": "When we observe our visual environment, we do not perceive all its components as being equally interesting. Some objects automatically and effortlessly \"pop out\" from their surroundings, that is, they draw our visual attention, in a \"bottom up\" manner, towards them. In a first approximation, focal visual attention acts as a rapidly shiftable \"spotlight,\" which allows only the selected information to reach higher levels of processing and representation. Most models of the bottom-up control of attention are based on the concept of a saliency map, that is, an explicit two-dimensional map that encodes the conspicuity of objects in the visual environment. Competition among neurons in this map gives rise to a single winning location that corresponds to the next attended target. Inhibiting this location automatically allows the system to attend to the next most salient location. A first body of work in this thesis describes a detailed computer implementation of such a scheme, focusing on the problem of combining information across modalities, here orientation, intensity and color information, in a purely stimulus-driven manner. The model is applied to common psychophysical stimuli as well as to very demanding visual search tasks. Its successful performance is used to address the extent to which the primate visual system carries out visual search via one or more such saliency maps and how this can be tested.
\r\n\r\nWe next address the question of what happens once our attention is focused onto a restricted part of our visual field. There is mounting experimental evidence that attention is far more sophisticated than a simple feed-forward spatially-selective filtering process. Indeed, visual processing appears to be significantly different inside the attentional spotlight than outside. That is, in addition to its properties as a feed-forward information processing and transmission bottleneck, focal visual attention feeds back and locally modulates, in a \"top down\" manner, the visual processing and representation of selected objects. The second body of work presented in this thesis is concerned with a detailed computational model of basic pattern vision in humans and its modulation by top-down attention. We start by acquiring a complete dataset of five different simple psychophysical experiments, including discriminations of contrast, orientation and spatial frequency of simple pattern stimuli by human observers. This experimental dataset places strict constraints on our model of early pattern vision. The model, however, is eventually able to reproduce the entire dataset while assuming plausible neurobiological components. The model is further applied to existing psychophysical data which demonstrates how top-down attention alters performance in these simple psychophysical discrimination experiments. Our model is able to quantitatively account for all observations by assuming that attention strengthens the non-linear cortical interactions among visual neurons.
\r\n\r\nTogether, the two aspects of attention studied in this thesis lead us to consider the essential role of non-linear computations in visual processing. We suggest that visual processing, even at its earliest levels, is best characterized not by linear response functions and spatial convolutions, but rather by non-linearly interacting computational devices.
" } ]