Committee Feed
https://feeds.library.caltech.edu/people/Abu-Mostafa-Y-S/committee.rss
A Caltech Library Repository Feedhttp://www.rssboard.org/rss-specificationpython-feedgenenTue, 16 Apr 2024 14:51:49 +0000Bit-Serial Reed-Solomon Decoders in VLSI
https://resolver.caltech.edu/CaltechETD:etd-03252008-090414
Authors: {'items': [{'id': 'Whiting-Douglas-Lee', 'name': {'family': 'Whiting', 'given': 'Douglas Lee'}, 'show_email': 'NO'}]}
Year: 1985
DOI: 10.7907/bjd1-9j44
<p>Reed-Solomon codes are known to provide excellent error-correcting capabilities on many types of communication channels. Although efficient decoding algorithms have been known for over fifteen years, currently available decoder systems are large both in size and in power consumption. Such systems typically use a single, very fast, fully parallel finite-field multiplier in a sequential architecture. Thus, more processing time is required as the code redundancy increases. By using many arithmetic units on a single chip, it is possible to exploit the concurrency inherent in the decoding algorithms to attain performance levels previously possible only with large ECL systems.</p>
<p>An investigation into the structure of binary extension fields reveals that the common arithmetic operations used in decoding can be implemented quite efficiently in a bit-serial fashion, using any of several bases over GF(2). Berlekamp's dual-basis multiplier is generalized to the product of two arbitrary field elements, and a necessary and sufficient condition is then derived for the existence of a self-dual basis. Efficient methods for bit-serial multiplicative inversion are also discussed, greatly reducing the complexity traditionally associated with this operation.</p>
<p>Using these bit-serial techniques, several architectures for implementing each phase of the known Reed-Solomon decoding algorithms are presented and compared. Simple methods are presented to allow power-sum syndrome decoders to handle codes with a variety of block lengths and redundancies. Each approach comes within a factor of log <i>n</i> (where <i>n</i> is the block length of the code) of the recently derived asymptotic lower bounds for both time and area. Results from a student project to lay out a prototype decoder chip using the Berlekamp-Massey algorithm are also discussed. By utilizing the parallelism inherent in the key equation solution, these architectures can decode received words at a speed independent of the redundancy of the code.</p>https://thesis.library.caltech.edu/id/eprint/1117A Hierarchical Timing Simulation Model for Digital Integrated Circuits and Systems
https://resolver.caltech.edu/CaltechETD:etd-04102008-105646
Authors: {'items': [{'id': 'Lin-Tzu-mu', 'name': {'family': 'Lin', 'given': 'Tzu-mu'}, 'show_email': 'NO'}]}
Year: 1985
DOI: 10.7907/41bh-7e43
<p>A hierarchical timing simulation model for digital MOS circuits and systems is presented. This model supports the structured design methodology, and can be applied to both "structure" and "behavior" representations of designs in a uniform manner. A simulator based on this model can run several orders of magnitude faster than any other simulators that offer the same amount of information.</p>
<p>At the structure (transistor) level, the transient behavior of a digital MOS circuit is approximated by that of an RC network for estimating delays. The Penfield-Rubinstein RC tree model is extended to include the effects of parallel paths and initial charge distributions. As far as delay is concerned, a two-port RC network is characterized by three parameters: R: series resistance, C: loading capacitance and D: internal delay. These parameters can be determined hierarchically as networks are composed in various ways. The composition rules are derived directly from the Kirchoff's current and voltage laws, so that the consistency with physics is established.</p>
<p>The (R, C, D) characterization of two-port RC networks is then generalized to describe the behavior of semantic cells at any level of representation. A semantic cell is a functional block which can be abstracted by its steady-state behavior to interface with other cells in the system. As semantic cells are composed, the parameters of the composite cell can be determined from those of the the component cells either analytically or by simulation. A Smalltalk implementation of the hierarchical timing simulation model is also presented.</p>https://thesis.library.caltech.edu/id/eprint/1331I. On a Family of Generalized Colorings. II. Some Contributions to the Theory of Neural Networks. III. Embeddings of Ultrametric Spaces
https://resolver.caltech.edu/CaltechTHESIS:04052019-110135296
Authors: {'items': [{'id': 'Baldi-Pierre', 'name': {'family': 'Baldi', 'given': 'Pierre'}, 'show_email': 'NO'}]}
Year: 1986
DOI: 10.7907/0bwx-nk73
<p>This thesis comprises three apparently very independent parts. However, there is a unity behind I would like to sketch very briefly.</p>
<p>Formally graphs are in the background of most chapters and so is the duality local versus global. The first section is concerned with globally coloring graphs under some local assumptions. Algorithmically it is an intrinsically difficult task and neural networks, the topic of the second part can be used to approach intractable problems. Simple local interactions with emergent collective behavior are one of the essential features of these networks. Their current models are similar to some of those encountered in statistical mechanics, like spin glasses. In the third part, we study ultrametricity, a concept recently rediscovered by theoretical physicists in the analysis of spin-glasses. Ultrametricity can be expressed as a local constraint on the shape of each triangle of the given metric space.</p>
<p>Unless otherwise stated, results in the first and second part are essentially original. Since the third part represents a joint work with Michael Aschbacher, Eric Baum and Richard Wilson, I should perhaps try to outline my contribution though paternity of collective results is somewhat fuzzy. While working on neural networks and spin glasses Eric and I got interested in ultrametricity. Several of us had found an initial polynomial upper bound, but the final results of "n + 1" was first reached independently by Michael and Richard. I think I obtained the theorems: 4.5, 6.1, 6.3 (using an idea of Eric), 6.4, 6.5, 6.6, 6.7 (with Richard and helpful references from Bruce Rothschild and Olga Taussky) and participated in some other results.</p>https://thesis.library.caltech.edu/id/eprint/11440An Information- and Coding-Theoretic Study of Bursty Channels with Applications to Computer Memories
https://resolver.caltech.edu/CaltechETD:etd-03252008-093415
Authors: {'items': [{'id': 'Abdel-Ghaffar-Khaled-Ahmed-Sabry', 'name': {'family': 'Abdel-Ghaffar', 'given': 'Khaled Ahmed Sabry'}, 'show_email': 'NO'}]}
Year: 1986
DOI: 10.7907/gfyh-ss98
<p>This thesis is a study of two-dimensional bursty channels from the information- theoretic as well as the coding-theoretic points of view. An information-theoretic model of bursty channels is defined and analyzed using probabilistic arguments. Two-dimensional burst correcting codes are developed. Their combinatorial and algebraic structures are examined. Two-dimensional bursty channels are used to model computer memories. The results of this thesis give bounds on the storage capacities of computer memories if sophisticated codes are used.</p>
https://thesis.library.caltech.edu/id/eprint/1120A Parallel Execution Model for Logic Programming
https://resolver.caltech.edu/CaltechETD:etd-03192008-143903
Authors: {'items': [{'id': 'Li-Peyyun-Peggy', 'name': {'family': 'Li', 'given': 'Peyyun Peggy'}, 'show_email': 'NO'}]}
Year: 1986
DOI: 10.7907/2ngs-bp80
<p>The Sync Model, a parallel execution method for logic programming, is proposed. The Sync Model is a multiple-solution data-driven model that realizes AND-parallelism and OR-parallelism in a logic program assuming a message-passing multiprocessor system. AND parallelism is implemented by constructing a dynamic data flow graph of the literals in the clause body with an ordering algorithm. OR parallelism is achieved by adding special Synchronization signals to the stream of partial solutions and synchronizing the multiple streams with a merge algorithm.</p>
<p>The Sync Model is proved to be sound and complete. Soundness means it only generates correct solutions and completeness means it generates all the correct solutions. The soundness and completeness of the Sync Model are implied by the correctness of the merge algorithm.</p>
<p>A new class of interconnection networks, the Sneptree, is also presented. The Sneptree is an augmented complete binary tree which can simulate an unbounded complete binary tree optimally. Amongst different connection patterns of the Sneptree, some are regular and extensible so as to be well suited for VLSI implementation. A recursive method is presented to generate the H-structure layout of one type of the Sneptree, called the Cyclic Sneptree. A message routing algorithm between any two leaf nodes of the Cyclic Sneptree is also presented. The routing algorithm, which is of O(n) complexity, gives a good approximation to the shortest path.</p>
<p>The Sneptree is an ideal architecture for the Sync model, in which a dynamic process tree is constructed. With a simple mapping algorithm, the Sync Model can be mapped onto the Sneptree with highly-balanced load and low overhead.</p>
https://thesis.library.caltech.edu/id/eprint/1023Topics in Millimeter-Wave Imaging Arrays
https://resolver.caltech.edu/CaltechETD:etd-03012008-134009
Authors: {'items': [{'id': 'Kasilingam-Dayalan-P', 'name': {'family': 'Kasilingam', 'given': 'Dayalan P.'}, 'show_email': 'NO'}]}
Year: 1987
DOI: 10.7907/rt21-jt48
<p>In this thesis two different types of antenna arrays are investigated as possible configurations for <i>two-dimensional</i> diffraction limited imaging arrays. The first configuration is the "fly's-eye" array of microlenses. It is shown that this configuration may be utilized to achieve diffraction limited imaging with theoretical coupling efficiencies of around 50%. The other configuration is the two-dimensional horn array. It is shown that in this configuration, wide-angled horns etched into silicon achieve theoretical coupling efficiencies of 60%. A design for a <i>two-dimensional</i> imaging array, using horn elements of aperture size 1.5λ<sub>0</sub> was suggested. Also covered in this thesis are the radiation losses and the substrate-mode losses of coplanar transmission lines. It is shown that at millimeter-wave frequencies these losses are prohibitively high. Finally in the appendix a simulation of Schottky diode mixers is described as a possible design tool for analyzing millimeter-wave detector circuits.</p>https://thesis.library.caltech.edu/id/eprint/827Time and Space Integrating Acousto-Optic Signal Processing
https://resolver.caltech.edu/CaltechETD:etd-05052006-155412
Authors: {'items': [{'email': 'kelvin@colorado.edu', 'id': 'Wagner-Kelvin-H', 'name': {'family': 'Wagner', 'given': 'Kelvin H.'}, 'show_email': 'NO'}]}
Year: 1987
DOI: 10.7907/4WH0-H941
<p>One dimensional acousto-optic signal processing techniques are examined from the systems and functional viewpoint, and are then used as building blocks to synthesize multidimensional time and space integrating architectures. Time and space integrating signal processing systems are capable of performing 2-dimensional linear transformations upon images or matrices, by sequentially entering rows of the image with a travelling wave acousto-optic Bragg cell. The travelling rows are frozen by a pulsed laser diode, and the stationary diffracted fields are spatially processed by an optical system. The successively transformed rows are sequentially multiplied by a time varying reference wavefront, and accumulated on a time integrating CCD detector array to complete the two dimensional processing. Long 1-dimensional signals can also be linearly transformed by a time and space integrating system, by using a similar strategy upon a folded, or rastered, version of the high time bandwidth product signal. Small pieces of the long signal are slid into the system with an acousto-optic devices, and are spatially transformed over the device aperture. Then, successively transformed portions of the long signal are multiplied by a reference, and appropriately delayed and accumulated on a 2-D CCD in order to perform multichannel time integrations in the orthogonal dimension. The desired high time bandwidth one dimensional linear transformation is represented in the folded coordinate space of the 2-dimensional output detector.</p>
<p>The operational characteristics of the principal active devices used in these time and space integrating systems are examined from the viewpoint of the system architect. The effects of the devices on the overall system operation are discussed, and device designs intended for application in a time and space integrating system operating environment are proposed.</p>
<p>The final chapter is a detailed theoretical and experimental investigation into the particular operating characteristics of systems designed to perform a folded spectrum analysis of very high time bandwidth signals. This spectrum analysis problem has a shift variant transformation kernel, which can be broken down into a succession of smaller temporal and spatial sub transformations. The 1-dimensional space integrating spectrum analysis operation performed by a lens is used to produce a coarse spectral channelization of the input signal, displayed as a one dimensional spatial profile. Each resolvable spectral channel is fine frequency analyzed by temporal integration, producing a resulting intensity variation of each channel in the orthogonal direction, thereby forming a folded representation of the desired high time bandwidth spectrum analysis. The information which is needed to perform the fine frequency analysis is carried on the optical phase, so interferometric techniques are employed in order to detect the phase and transform it to an optical intensity modulation. Various bias terms are produced on the detector by the interferometric detection operation, and techniques for removing the unwanted bias are investigated. These include spatial carrier encoding of the interferometric terms combined with bandpass filtering, and direct bias subtraction techniques.</p>https://thesis.library.caltech.edu/id/eprint/1638Optical Computing for Adaptive Signal Processing and Associative Memories
https://resolver.caltech.edu/CaltechETD:etd-06142006-094757
Authors: {'items': [{'id': 'Hong-John-Hyunchul', 'name': {'family': 'Hong', 'given': 'John Hyunchul'}, 'show_email': 'NO'}]}
Year: 1987
DOI: 10.7907/3vpt-fn50
<p>Optical techniques for performing two computing tasks are investigated. First, acousto-optical systems that implement adaptive filtering structures are presented for operation in environments that are not well characterized <i>a priori</i> or are time-varying. Theoretical analyses along with experimental confirmations are given to identify the important system parameters that affect the performance. Extensions of the systems to the multidimensional domain of phased array signal processing are discussed as well as novel implementations that use photorefractive crystals as time-integrating elements.</p>
<p>Also investigated are various associative memory models. An acousto-optic implementation of the so-called Hopfield model is presented. The system's storage capacity and attraction radius are characterized experimentally and are shown to agree with computer simulations. Secondly, an upper bound is derived for the storage capacity of holographic associative memories that use planar holograms. It is shown that if the space bandwidth product of the hologram is N<sub>2</sub>, then the holographic memory can store at most N<sub>2</sub>/N<sub>3</sub> associations, where N<sub>3</sub> is the number of pixels in each output item. Finally, associative memories whose performance is invariant with respect to shifts in the input pattern position are considered. It is shown that nonlinear interconnections are required to achieve shift invariant operation, and optical implementations are discussed.</p>
https://thesis.library.caltech.edu/id/eprint/2583Linear Maps with Point Rules: Applications to Pattern Classification and Associative Memory
https://resolver.caltech.edu/CaltechETD:etd-03052008-095021
Authors: {'items': [{'id': 'Venkatesh-Santosh-Subramanyam', 'name': {'family': 'Venkatesh', 'given': 'Santosh Subramanyam'}, 'show_email': 'NO'}]}
Year: 1987
DOI: 10.7907/1YSB-Q028
<p>Generalisations of linear discriminant functions are introduced to tackle problems in pattern classification, and associative memory. The concept of a point rule is defined, and compositions of global linear maps with point rules are incorporated in two distinct structural forms—feedforward and feedback—to increase classification flexibility at low increased complexity. Three performance measures are utilised, and measures of consistency established.</p>
<p>Feedforward pattern classification systems based on multi-channel machines are introduced. The concept of independent channels is defined and used to generate independent features. The statistics of multi-channel classifiers are characterised, and specific applications of these structures are considered. It is demonstrated that image classification invariant to image rotation and shift is possible using multi-channel machines incorporating a square-law point rule. The general form of rotation invariant classifier is obtained. The existence of optimal solutions is demonstrated, and good sub-optimal systems are introduced, and characterised. Threshold point rules are utilised to generate a class of low-cost binary filters which yield excellent classification performance. Performance degradation is characterised as a function of statistical side-lobe fluctuations, finite system space-bandwidth, and noise.</p>
<p>Simplified neural network models are considered as feedback systems utilising a linear map and a threshold point rule. The efficacy of these models is determined for the associative storage and recall of memories. A precise definition of the associative storage capacity of these structures is provided. The capacity of these networks under various algorithms is rigourously derived, and optimal algorithms proposed. The ultimate storage capacity of neural networks is rigourously characterised. Extensions are considered incorporating higher-order networks yielding considerable increases in capacity.</p>https://thesis.library.caltech.edu/id/eprint/883Resource-Bounded Category and Measure in Exponential Complexity Classes
https://resolver.caltech.edu/CaltechTHESIS:03192012-101000956
Authors: {'items': [{'id': 'Lutz-Jack-Harold', 'name': {'family': 'Lutz', 'given': 'Jack Harold'}, 'show_email': 'NO'}]}
Year: 1987
DOI: 10.7907/dccw-3z56
<p>This thesis presents <i>resource-bounded category</i> and <i>resource bounded-measure</i> - two new tools for computational complexity theory - and some applications of these tools to the structure theory of exponential complexity classes.</p>
<p>Resource-bounded category, a complexity-theoretic version of the classical Baire category method, identifies certain subsets of PSPACE, E, ESPACE, and other complexity classes as <i>meager</i>. These meager sets are shown to form a nontrivial ideal of "small" subsets of the complexity class. The meager sets are also (almost) characterized in terms of certain two-person infinite games called <i>resource-bounded Banach-Mazur games</i>.</p>
<p>Similarly, resource-bounded measure, a complexity-theoretic version of Lebesgue measure theory, identifies the <i>measure</i> 0 subsets of E, ESPACE, and other complexity classes, and these too are shown to form nontrivial ideals of "small" subsets. A resource-bounded extension of the classical Kolmogorov zero-one law is also proven. This shows that measurable sets of complexity-theoretic interest either have measure 0 or are the complements of sets of measure 0.</p>
<p>Resource-bounded category and measure are then applied to the investigation of uniform versus nonuniform complexity. In particular, Kannan's theorem that ESPACE ⊊ P/Poly is extended by showing that P/ Poly ∩ ESPACE is only a meager, measure 0 subset of ESPACE. A theorem of Huynh is extended similarly by showing that all but a meager, measure 0 subset of the languages in ESPACE have high space-bounded Kolmogorov complexity.</p>
<p>These tools are also combined with a new hierarchy of exponential time complexity classes to refine known relationships between nonuniform complexity and time complexity.</p>
<p>In the last part of the thesis, known properties of hard languages are extended. In particular, recent results of Schöning and Huynh state that any language L which is ≤<sup>P</sup><sub>m</sub>-hard for E or ≤<sup>P</sup><sub>T</sub>-hard for ESPACE cannot be feasibly approximated (i.e., its symmetric difference with any feasible language has exponential density). It is proven here that this conclusion in fact holds unless only a meager subset of E is ≤<sup>P</sup><sub>m</sub>-reducible to L and only a meager, measure 0 subset of ESPACE is ≤<sup>PSPACE</sup><sub>m</sub>-reducible to L. (It is conjectured, but not proven, that this result is actually stronger than those of Schöning and Huynh.) This suggests a new lower bound method which may be useful in interesting cases.</p>https://thesis.library.caltech.edu/id/eprint/6854Information Theory and Radar: Mutual Information and the Design and Analysis of Radar Waveforms and Systems
https://resolver.caltech.edu/CaltechETD:etd-06272005-152700
Authors: {'items': [{'email': 'mrb@ecn.purdue.edu', 'id': 'Bell-Mark-Robert', 'name': {'family': 'Bell', 'given': 'Mark Robert'}, 'show_email': 'NO'}]}
Year: 1988
DOI: 10.7907/AYD5-RB83
<p>This thesis examines the use of information theory in the analysis and design of radar, with a particular emphasis on the information-theoretic design of radar waveforms. First, a brief review of information theory is presented and then the applicability of mutual information to the measurement of radar performance is examined. The idea of the radar target channel is introduced. The Radar/Information Theory Problem is formulated and solved for a number of radar target channels, providing insight into the problem of designing radar waveforms that maximize the mutual information between the target and the received radar signal. Radar-scattering models are examined in order to obtain usable models for practical waveform design problems. The target impulse response is introduced as a method of characterizing the spatial range distribution of radar targets. The target impulse response is used to formulate a new generalization of the matched filter in radar that matches a transmitted-waveform/receiver-filter pair to a target of known impulse response, providing the maximum signal-to-noise ratio at the receiver under a constraint on transmitted energy and the time duration of the waveform. Next, the problem is formulated and solved of designing radar waveforms that maximize the mutual information between the target and the received radar waveform for a target characterized by an impulse response that is a finite-energy random process. The characteristics of waveforms for optimum detection and for obtaining maximum information about a target are compared. Finally, the information content of radar images is examined. It is concluded that the information-theoretic viewpoint can improve the performance of practical radar systems.</p>
https://thesis.library.caltech.edu/id/eprint/2737The Critical Points of Poynting Vector Fields
https://resolver.caltech.edu/CaltechETD:etd-11082007-131130
Authors: {'items': [{'id': 'Rizvi-Syed-Azhar-Abbas', 'name': {'family': 'Rizvi', 'given': 'Syed Azhar Abbas'}, 'show_email': 'NO'}]}
Year: 1988
DOI: 10.7907/gj18-eq28
<p>In a thought provoking paper Maxwell [The Scientific Papers of James Clerk Maxwell, ed. W. D. Niven, vol. 2, 233-240, Dover Publications, New York (1952)] studied the flow of water on the Earth's surface and how this flow is affected by the local geography. His results linking number of hills and lake bottoms to valleys are simple and the conclusions elegant. Critical points such as summits and lake bottoms play a key role in the overall organization and structuring of the flow lines. This is the spirit in which electromagnetic power flow represented by the Poynting vector field (S) is studied in this thesis. The specialized case of a planar S field which arises due to a single electromagnetic field component <i>E<sup>z</sup></i> or <i>H<sup>z</sup></i> is dealt with here in considerable detail.</p>
<p>In order to analyse the behaviour of the flow lines of a plane Poynting vector field in the neighbourhood of a critical point, the S field is expanded in a Taylor series. Critical points can be classified according to their order, degeneracy or structural stability. The order of a critical point refers to the degree of the leading non zero term in the Taylor series. A critical point is non degenerate if this leading term is sufficient to give a qualitative description of the flow lines in the neighbourhood. A critical point is structurally stable if the flow lines in the neighbourhood do not change drastically when there is a small perturbation of the electromagnetic field. It is found that lowest order critical points, i.e., elementary center point and elementary saddle point, are the only structurally stable critical points. These critical points are always non degenerate. All degenerate and non elementary critical points are found to be structurally unstable. A formula for the index of rotation of the S field at a critical point is derived. The behaviour of the electric or the magnetic field component which lies in the <i>x-y</i> plane is also studied. It is shown that structurally unstable configurations of flow lines change into structurally stable configurations under small perturbations in such a way that the index of rotation is conserved. The statements made above in connection with the behaviour of flow lines and structural stability are illustrated with the help of examples involving linearly polarized system of interfering plane and/or cylindrical waves.</p>
<p>The flow lines of the S field in the vicinity of a perfectly conducting surface are studied. It is found that in structurally stable situations these lines are either parallel to the surface or they form critical points of half saddle type on this surface. Two types of problems involving flow lines and conducting surfaces are identified. The interior problem deals with the situations where all the flow lines are inside a region bounded by a perfect conductor. In the exterior problems all the flow lines are outside a region bounded by a perfectly conducting surface. Conclusions regarding the existence of critical points and the behaviour of flow lines are drawn in the two above mentioned problems. These conclusions are verified by computation of flow lines in a few well known problems of scattering and diffraction.</p>
<p>Finally the critical points of three dimensional Poynting vector fields are considered. A complete classification of these critical points requires further study at this time. In this thesis only structurally stable critical points are classified for these S fields. An example demonstrating the existence of such critical points is given.</p>https://thesis.library.caltech.edu/id/eprint/4465A Model for the Study of Very Noisy Channels, and Applications
https://resolver.caltech.edu/CaltechETD:etd-11082007-085237
Authors: {'items': [{'id': 'Majani-Eric-Etienne', 'name': {'family': 'Majani', 'given': 'Eric Etienne'}, 'show_email': 'NO'}]}
Year: 1988
DOI: 10.7907/9AF1-K251
<p>Very Noisy channels (such as the wideband gaussian channel well-known in deep space communications) have the interesting property that although the maximum number of bits transmitted per symbol is close to zero, the maximum number of bits transmitted per second is not! Furthermore, recent results on the ultimate limits of information density indicate that some channels perform better when pushed to their very noisy limit.</p>
<p>We present a general mathematical model of Very Noisy channels which provides an insight in their behavior, and in some interesting cases, tells us about the limiting behavior of the larger class of noisy channels.</p>
<p>Two classes of Very Noisy Channels are identified and efficient algorithms that compute their capacity are presented. We show that for some Very Noisy broadcast channels, the time-shared coding strategy performs as well as the optimal strategy known as broadcast coding in the limit. Finally, with the help of our model, we derive a tight lower bound on the amount of information lost in a Channel Reduction or Data Compression.</p>
https://thesis.library.caltech.edu/id/eprint/4461The Application of Information Theory to Problems in Decision Tree Design and Rule-Based Expert Systems
https://resolver.caltech.edu/CaltechETD:etd-02022007-103549
Authors: {'items': [{'id': 'Smyth-Padhraic', 'name': {'family': 'Smyth', 'given': 'Padhraic'}, 'show_email': 'NO'}]}
Year: 1988
DOI: 10.7907/cn89-3127
<p>This thesis examines the problems of designing decision trees and expert systems from an information-theoretic viewpoint. A well-known greedy algorithm using mutual information for tree design is analysed. A basic model for tree design is developed leading to a series of bounds relating tree performance parameters. Analogies with prefix-coding and rate-distortion theory lead to interesting interpretations and results. The problem of finding termination rules for such greedy algorithms is discussed in the context of the theoretical models derived earlier, and several experimentally observed phenomena are explained in this manner. In two classification experiments, involving alphanumeric LEDS and local edge detection, the hierarchical approach is seen to offer significant advantages over alternative techniques.</p>
<p>The second part of the thesis begins by analysing the difficulties in designing rule-based expert systems. The inability to model uncertainty in an effective manner is identified as a key limitation of existing approaches. Accordingly, an information-theoretic model for rules and rule-based systems is developed. From a simple definition of rule information content, the ability to specialise and generalise (akin to cognitive processes) in a quantitative manner is demonstrated. The problem of generalised rule induction is posed and the ITRULE algorithm is described which derives optimal rule sets from data. The problem of probabilistic updating in inference nets is discussed and a new maximum-likelihhod rule is proposed based on bounded probabilities. Utility functions and statistical decision theory concepts are used to develop a model of implicit control for rule-based inference. The theory is demonstrated by deriving rules from expert-supplied data and performing backward and forward chaining based on decision-theoretic criteria. The thesis concludes by outlining the many problems which remain to be solved in this area, and by briefly discussing the analogies between rule-based inference nets and neural networks.</p>https://thesis.library.caltech.edu/id/eprint/462Monolithic Millimeter-Wave Two-Dimensional Horn Imaging Arrays
https://resolver.caltech.edu/CaltechETD:etd-11082007-111802
Authors: {'items': [{'id': 'Rebeiz-Gabriel-M', 'name': {'family': 'Rebeiz', 'given': 'Gabriel M.'}, 'show_email': 'NO'}]}
Year: 1988
DOI: 10.7907/B9FG-6H18
<p>A new monolithic millimeter-wave two-dimensional horn imaging array is presented. In this configuration, a dipole is suspended in an etched pyramidal cavity on a 1-µm silicon-oxynitride membrane. This approach allows ample space for low-frequency interconnections, while still maintaining efficient diffraction-limited imaging. The fabrication procedure of the horn array and the deposition parameters of the membrane layer are presented in detail. The array is analysed rigorously, by approximating the horn antenna by a structure of multiple rectangular waveguide sections. Pattern measurements at 93 GHz and 242 GHz agree well with the theory. The results show that horn antennas with an opening between 1.0λ and 1.5λ have high aperture efficiencies and would match well appropriate imaging systems. Also, a new wideband log-periodic antennas is integrated on a thin membrane and tuned by a back plane reflector. The antenna patterns are measured at 167 GHz, 370 GHz and 700 GHz, and the effect of the back-plane reflector is investigated at 370 GHz.</p>https://thesis.library.caltech.edu/id/eprint/4464Error-Correction Coding for Reliable Communication in the Presence of Extreme Noise
https://resolver.caltech.edu/CaltechETD:etd-02012007-093354
Authors: {'items': [{'id': 'Chao-Chi-chao', 'name': {'family': 'Chao', 'given': 'Chi-chao'}, 'show_email': 'NO'}]}
Year: 1989
DOI: 10.7907/h228-d056
<p>This thesis is a study of error-correcting codes for reliable communication in the presence of extreme noise. We consider very noisy channels, which occur in practice by pushing ordinary channels to their physical limits. Both block codes and convolutional codes are examined.</p>
<p>We show that the family of triply orthogonal codes, defined and studied in this thesis, or orthogonal codes can be used to achieve channel capacity for certain classes of very noisy discrete memoryless channels. The performance of binary block codes on the unquantized additive white Gaussian noise channel at very low signal-to-noise ratios is studied. Expressions are derived for the decoder block error as well as bit error probabilities and the asymptotic coding gain near the point where the signal energy is zero.</p>
<p>The average distance spectrum for the ensemble of time-varying convolutional codes is computed, and the result gives a surprisingly accurate prediction of the growth rate of the number of fundamental paths at large distance for fixed codes. A Gilbert-like free distance lower bound is also given. Finally, a Markov chain model is developed to approximate burst error statistics of Viterbi decoding. The model is validated through computer simulations and is compared with the previously proposed geometric model.</p>https://thesis.library.caltech.edu/id/eprint/437Binary Correlators for Optical Computing and Pattern Recognition
https://resolver.caltech.edu/CaltechETD:etd-02082007-130728
Authors: {'items': [{'id': 'Mok-Fai-Ho', 'name': {'family': 'Mok', 'given': 'Fai Ho'}, 'show_email': 'NO'}]}
Year: 1989
DOI: 10.7907/dwpt-gn93
<p>The matrix-vector multiplier is an important building block in optical information processing architectures, examples of which are correlators for pattern recognition, associative memories, and neural networks. Such architectures are most suitable for implementation by optics due to the ease in realizing dense interconnections optically. The success of the implementation partially relies on the quality of the SLM used to record the information for processing. Limited dynamic range for the representation of the data recorded is a common drawback suffered by most commercially available devices. In this thesis, the importance of the dynamic range of the device on the performance of the implementation is investigated. The effect of limited dynamic range on the signal to noise ratio, probability of error, capacity, and training of various forms of matrix-vector multipliers are addressed. Through the use of theoretical analyses, computer simulations, and optical experiments, it will be shown that a large dynamic range is not essential in most applications. Specifically, it is shown that only one bit of dynamic range, i.e. two gray levels, for the representation of each data point, results in acceptable loss in performance.</p>https://thesis.library.caltech.edu/id/eprint/555Coding Beyond the Computational Cutoff Rate
https://resolver.caltech.edu/CaltechETD:etd-02012007-132507
Authors: {'items': [{'id': 'Collins-Oliver-Michael', 'name': {'family': 'Collins', 'given': 'Oliver Michael'}, 'show_email': 'NO'}]}
Year: 1989
DOI: 10.7907/nxwj-1433
<p>This thesis presents a collection of new codes, algorithms, and hardware, which can all be used to reduce the required energy per information bit to noise spectral density ratio on the Gaussian channel. First comes a feedback technique from an outer to an inner code. The basic idea is to perform a second maximum likelihood decoding operation of the inner code that incorporates side information. Next comes a new kind of algebraic outer code which we get from combining Reed Solomon codes with themselves. The most important results, however, deal with the construction of long constraint length Viterbi decoders. One chapter presents a hardware design of a constraint length 15, rate 1/6 decoder. The last chapter gives some results on the partitioning of a deBruijn graph which make the number of interconnections in the design physically realizable.</p>https://thesis.library.caltech.edu/id/eprint/441On Complexity and Efficiency in Encoding and Decoding Error-correcting Codes
https://resolver.caltech.edu/CaltechETD:etd-06102005-114154
Authors: {'items': [{'id': 'Coffey-John-Timothy', 'name': {'family': 'Coffey', 'given': 'John Timothy'}, 'show_email': 'NO'}]}
Year: 1989
DOI: 10.7907/4jy2-w055
<p>A central paradox of coding theory has been noted for many years, and concerns the existence and construction of the best codes. Virtually every linear code is "good" in the sense that it meets the Gilbert-Varshamov bound on distance versus redundancy. Despite the sophisticated constructions for codes derived over the years, however, no one has succeeded in demonstrating a constructive procedure which yields such codes over arbitrary symbol fields. A quarter of a century ago, Wozencraft and Reiffen, in discussing this problem, stated that "we are tempted to infer that any code of which we cannot think is good." Using the theory of Kolmogorov complexity, we show the remarkable fact that this statement holds true in a rigorous mathematical sense: any linear code which is truly random, in the sense that there is no concise way of specifying the code, is good. Furthermore, random selection of a code which does contain some constructive pattern results, with probability bounded away from zero, in a code which does not meet the Gilbert-Varshamov bound regardless of the block length of the code. In contrast to the situation for linear codes, we show that there are effectively random non-linear codes which have no guarantee on distance, and that over all rates, the average non-linear code has much lower distance than the average linear code.</p>
<p>These techniques are used to derive original results on the performance of various classes of codes, including shortened cyclic, generalized Reed-Solomon, and general non-linear codes, under a variety of decoding strategies involving mixed burst- and random-error correction.</p>
<p>The second part of the thesis deals with the problem of finding decoding algorithms for general linear codes. These algorithms are capable of full hard decision decoding or bounded soft decision decoding, and do not rely on any rare structure for their effectiveness.</p>
<p>After a brief discussion of some aspects of the theory of NP-completeness as it relates to coding theory, we propose a simple model of a general decoding algorithm which is sufficiently powerful to be able to describe most of the known approaches to the problem. We provide asymptotic analysis of the complexity of various approaches to the problem under various decoding strategies (full hard decision decoding and bounded hard- and soft-decision decoding) and show that a generalization of information set decoding gives more efficient algorithms than any other approach known.</p>
<p>Finally, we propose a new type of algorithm that synthesizes some of the advantages of information set decoding and other algorithms that exploit the weight structure of the code, such as the zero neighbours algorithm, and discuss its effectiveness.</p>https://thesis.library.caltech.edu/id/eprint/2541Design and Implementation of Linear-Phase and/or Pairwise-Symmetric Perfect-Reconstruction FIR Multirate Filter Banks
https://resolver.caltech.edu/CaltechETD:etd-11102005-091115
Authors: {'items': [{'id': 'Nguyen-Truong-Quang', 'name': {'family': 'Nguyen', 'given': 'Truong Quang'}, 'show_email': 'NO'}]}
Year: 1989
DOI: 10.7907/8z2n-y592
<p>This thesis studies the structures, design procedures and implementations of FIR perfect-reconstruction digital filter banks. The first part of the thesis deals with the structures and the design procedures of the perfect-reconstruction filter banks where the polyphase transfer matrices are lossless. These structures are parameterized by a set of rotation angles [37]. The usual procedure is to blindly optimize these angles to minimize an objective function where the objective function consists of all the stopband energies of the filters which we would like to design. This procedure is very time-consuming because of the nonlinear objective function and the large number of parameters to be optimized. The pairwise-symmetry property is imposed on these perfect reconstruction systems as a means of decreasing the number of parameters (rotation angles). The pairwise-symmetric property together with a method to initialize these rotation angles gives a very efficient design procedure. Design examples and complexity of the pairwise-symmetric, perfect-reconstruction FIR filter banks have compared well with the approximate perfect-reconstruction systems.</p>
<p>The second part of the thesis studies the structures and the design procedures of perfect-reconstruction filter banks which yield linear-phase filters. By confining the problem to a class, we are able to count exactly the number of linear-phase, perfect-reconstruction filter banks in this class. For the two-channel filter banks, we have obtained structures and design procedures for all nontrivial systems. Comparison with the approximated perfect-reconstruction systems in terms of complexity and performance is made. In our subclass of linear-phase, perfect-reconstruction, there are three structures for the case of three-channel filter banks. By limiting the problem to one of these systems, we obtain structures which yield linear-phase, perfect-reconstruct ion filters. The implementation complexity is studied. Design examples for all new methods presented here are included, along with tabulation of lattice and filter coefficients.</p>
<p>[37] Z. Doganata, P. P. Vaidyanathan, and T. Q. Nguyen, "General Synthesis procedures for FIR lossless transfer matrices for perfect-reconstruction multirate filter bank applications," IEEE Trans. on Acoustics, Speech and Signal Processing, Vol. ASSP-36, 1561-1574, Oct. 1988.</p>https://thesis.library.caltech.edu/id/eprint/4488Elementary solutions for the H infinity- general distance problem- equivalence of H2 and H infinity optimization problems
https://resolver.caltech.edu/CaltechETD:etd-05152007-142515
Authors: {'items': [{'id': 'Kavranoglu-D', 'name': {'family': 'Kavranoglu', 'given': 'Davut'}, 'show_email': 'NO'}]}
Year: 1990
DOI: 10.7907/y2q9-nq75
This thesis addresses the H[infinity] optimal control theory. It is shown that SISO H[infinity] optimal control problems are equivalent to weighted Wiener-Hopf optimization in the sense that there exists a weighting function such that the solution of the weighted H2 optimization problem also solves the given H[infinity] problem. The weight is identified as the maximum magnitude Hankel singular vector of a particular function in H[infinity] constructed from the data of the problem at hand, and thus a state-space expression for it is obtained. An interpretation of the weight as the worst-case disturbance in an optimal disturbance rejection problem is discussed.
A simple approach to obtain all solutions for the Nehari extension problem for a given performance level [gamma] is introduced. By a limit taking procedure we give a parameterization of all optimal solutions for the Nehari's problem.
Using an imbedding idea [12], it is proven that four-block general distance problem can be treated as a one-block problem. Using this result an elementary method is introduced to find a parameterization for all solutions to the four-block problem for a performance level [gamma].
The set of optimal solutions for the four-block GDP is obtained by treating the problem as a one-block problem. Several possible kinds of optimality are identified and their solutions are obtained.https://thesis.library.caltech.edu/id/eprint/1824One and two-dimensional digital mutirate systems with applications in sub-sampling and bandlimited signal reconstruction
https://resolver.caltech.edu/CaltechETD:etd-05092007-130540
Authors: {'items': [{'id': 'Liu-V-C', 'name': {'family': 'Liu', 'given': 'Vincent Cheng-Teh'}, 'show_email': 'NO'}]}
Year: 1990
DOI: 10.7907/cvbb-m844
This thesis deals with the two-dimensional (2D) multirate quadrature mirror filter (QMF) bank and new applications of 1D and 2D multirate filter bank concepts to the periodic nonuniform sampling and reconstruction of bandlimited signals. The potential use of multirate filter banks in the statistically optimal estimation of signals in the presence of wide-sense cyclostationary noise is also examined. The two-dimensional QMF bank is free from aliasing if and only if a certain polyphase matrix product related to the filter bank possesses the 2D pseudo-circulant property. A 2D FIR filter bank can be designed with the perfect reconstruction property if the polyphase matrix of its analysis filter bank is constrained to be a 2D lossless matrix. A design example is included. The losslessness constraint is satisfied by imposing a cascaded structure of first-degree lossless sections on the polyphase matrix. A limited factroization theorem is derived for 2D FIR lossless systems where the order in one of the two dimensions is limited to unity. In the area of nonuniform sampling of multiband bandlimited signals, the filter bank approach is utilized to derive a computationally efficient method for reconstructing bandlimited signals. The above scheme can also be viewed as a mean of compressing and reconstructing an oversampled bandlimited signal. It is shown that such a scheme has lower computational complexity than traditional methods of sampling rate alteration. The results can be extended to nonuniform sampling in two-dimensions using integer lattices. A further application of the multirate filter bank is in signal estimation in the presence of cyclostationary noise. The necessary and sufficient condition for the filter bank to preserve the wide-sense stationarity of the input is derived. Several applications where cyclostationary noise is present are indicated, and through the use of simulations the performance of the optimal filter bank can be compared with the conventional scalar optimal filter. The roundoff noise in orthogonal matrix building blocks is analyzed, since these building blocks are commonly present in filter bank implementations.https://thesis.library.caltech.edu/id/eprint/1707General Structural Representations for Multi-Input Multi-Output Discrete-Time FIR and IIR Lossless Systems
https://resolver.caltech.edu/CaltechETD:etd-02222007-083438
Authors: {'items': [{'id': 'Doğanata-Zinnur', 'name': {'family': 'Doğanata', 'given': 'Zinnur'}, 'show_email': 'NO'}]}
Year: 1990
DOI: 10.7907/h011-7b66
<p>Discrete-time lossless systems have been found to be of great importance in many signal processing applications. However, a representation for lossless transfer matrices that spans all such matrices with the smallest possible number of parameters has not been proposed earlier. Existing representations are usually for special cases and therefore not general enough. In this study, two general and minimal representations are presented for multi-input, multi-output FIR and IIR lossless systems. The first representation is in terms of planar rotations and it leads to multi-input, multi-output lattice structures. The second representation is in terms of unit-norm vectors and it enables shorter convergence times in optimization applications. A simple modification of this representation leads to structures that remain lossless under quantization. The structures that follow from these representations share some properties such as the orthogonality of the implementations, and minimality of the number of parameters and scalar delays they are. Since all lossless transfer matrices can be spanned by appropriately adjusting their parameters, these structures can be particularly useful in applications that involve optimization under the constraint of losslessness. Some examples of such applications are included.</p>https://thesis.library.caltech.edu/id/eprint/711Photorefractive Volume Holography in Artificial Neural Networks
https://resolver.caltech.edu/CaltechETD:etd-05022006-155139
Authors: {'items': [{'email': 'dbrady@duke.edu', 'id': 'Brady-David-Jones', 'name': {'family': 'Brady', 'given': 'David Jones'}, 'show_email': 'YES'}]}
Year: 1990
DOI: 10.7907/1YB6-SE42
<p>This thesis describes the use of volume holography to implement large-scale linear transformations on distributed optical fields. Such transformations are useful in the construction of hardware for artificial neural networks. The reconstruction of multiple grating holograms in layers of thin transparencies and in continuous volume media is considered and conditions under which such holograms may be used for linear transformations are derived. The control of the nature of the transformation implemented using fractal sampling grids is reviewed and the impact of such sampling grids on the energy efficiency of the overall system is considered. Information storage in volume holograms is shown to require multiple exposures and the impact of multiple exposures on linear hologram formations in saturable media and photorefractive materials is considered. It is shown for both types of media that the overall diffraction efficiency of a recorded hologram must decrease with the square of the rank of the transformation implemented. A theory for hologram formation in photorefractive materials with multiple trapping species is developed and compared with experimental results. The impact of multiple species and fixing mechanisms on linear hologram formation is evaluated. A method for refreshing the diffraction efficiency of photorefractive holograms in adaptive systems is described and demonstrated. The construction of thick holograms for linear transformations in waveguides is considered. A novel method for controlling such holograms is described and demonstrated. Learning in holographic neural networks is considered and two experimental holographic neural systems are described. The relative strengths of optical and electronic technologies for implementations of neural interconnections are considered.</p>https://thesis.library.caltech.edu/id/eprint/1577Soft-decision decoding of a family of nonlinear codes using a neural network
https://resolver.caltech.edu/CaltechETD:etd-06252007-080630
Authors: {'items': [{'id': 'Erlanson-R-A', 'name': {'family': 'Erlanson', 'given': 'Ruth A.'}, 'show_email': 'NO'}]}
Year: 1991
DOI: 10.7907/c855-aj24
We demonstrate the use of a continuous Hopfield neural network as a K-WinnerTake-All (KWTA) network. We prove that, given an input of N real numbers, such a network will converge to a vector of K positive one components and (N-K) negative one components, with the positive positions indicating the K largest input components. In addition, we show that the [(N K)] such vectors are the only stable states of the system.
One application of the KWTA network is the analog decoding of error-correcting codes. We prove that the KWTA network performs optimal decoding.
We consider decoders that are networks with nodes in overlapping, randomly placed KWTA constraints and discuss characteristics of the resulting codes.
We present two families of decoders constructed by overlapping KWTA constraints in a structured fashion on the nodes of a neural network. We analyze the performance of these decoders in terms of error rate, and discuss code minimum distance and information rate. We observe that these decoders perform near-optimal, soft-decision decoding on a class of nonlinear codes. We present a gain schedule that results in improved decoder performance in terms of error rate.
We present a general algorithm for determining the minimum distance of codes defined by the stable states of neural networks with nodes in overlapping KWTA constraints.
We consider the feasibility of embedding these neural network decoders in VLSI technologies and show that decoders of reasonable size could be implemented on a single integrated circuit. We also analyze the scaling of such implementations with decoder size and complexity.
Finally, we present an algorithm, based on the random coding theorem, to communicate an array of bits over a distributed communication network of simple processors connected by a common noisy bus.https://thesis.library.caltech.edu/id/eprint/2731Design issues in multirate digital filter banks, including transmultiplexers
https://resolver.caltech.edu/CaltechETD:etd-06272007-082624
Authors: {'items': [{'id': 'Koilpillai-R-D', 'name': {'family': 'Koilpillai', 'given': 'Ravinder David'}, 'show_email': 'NO'}]}
Year: 1991
DOI: 10.7907/C6MF-3A43
NOTE: Text or symbols not renderable in plain ASCII are indicated by [...]. Abstract is included in .pdf document.
Several aspects of the theory and design of FIR digital filter banks for analysis/synthesis systems are studied in this thesis. In particular, we focus on filter banks satisfying the perfect reconstruction (PR) property. We present a new approach to design PR filter banks wherein the filter bank is obtained by cosine-modulation of a linear-phase prototype filter of length N = 2mM, m [...] 1 (where M is the number of channels). The PR property is satisfied because the polyphase component matrix of the modulated filter bank is lossless. This is achieved by satisfying the necessary and sufficient condition - a pairwise power complementary property between the 2M polyphase components of the prototype. In this approach, regardless of the number of channels, we still design only the prototype. The design procedure involves the two-channel lossless lattice. This approach compares favorably (in terms of the number of parameters to be optimized and the ease of design) with other design techniques. Design examples and detailed comparisons are presented.
The existing approaches for designing PR filter banks include the lattice based methods, which structurally force the polyphase component matrix to be lossless. New initialization procedures, which can be used to initialize the values of all the lattice parameters (prior to optimization), are presented. The main advantage is that we can get 'good' initializations by using conventional Quadrature Mirror Filter (QMF) banks and pseudo-QMF banks (which can be readily designed, but do not satisfy PR). It is shown that these filter banks have polyphase component matrices that are 'approximately' lossless. The initialization also enables the design of a family of PR filter banks.
In conventional approaches to pseudo-QMF design, the prototype filter is obtained by optimization, wherein lies the main computational effort. We present a new approach in which the prototype of a M-channel filter bank is obtained by spectral factorization (of a 2Mth band filter), thereby eliminating the need for optimization. The overall transfer function T(z) has linear-phase and an approximate 'flat' magnitude response in the region [epsilon][...][omega][...] ([pi] - [epsilon] where [epsilon] depends on the transition bandwidth of the prototype [...]. A new spectral factorization algorithm (non-iterative) which is based on the Inverse Linear Predictive Coding (LPC) technique is presented. Design examples for the above method are obtained by using this algorithm.
Finally, we consider a dual of the QMF circuit - the transmultiplexer (TMUX). Traditional TMUX designs suppress the undesirable crosstalk. The crosstalk-free transmultiplexer (CF-TMUX) focuses on crosstalk cancellation, rather that suppression. It is shown that the filters of a CF-TMUX are the same as the filters of a 1-skewed AF-QMF bank. In addition, if the QMF bank satisfies PR, then the TMUX also achieves PR.https://thesis.library.caltech.edu/id/eprint/2741Learning algorithms for neural networks
https://resolver.caltech.edu/CaltechETD:etd-09232005-083502
Authors: {'items': [{'email': 'amir@alumni.caltech.edu', 'id': 'Atiya-Amir', 'name': {'family': 'Atiya', 'given': 'Amir'}, 'show_email': 'YES'}]}
Year: 1991
DOI: 10.7907/F46C-3V67
This thesis deals mainly with the development of new learning algorithms and the study of the dynamics of neural networks. We develop a method for training feedback neural networks. Appropriate stability conditions are derived, and learning is performed by the gradient descent technique. We develop a new associative memory model using Hopfield's continuous feedback network. We demonstrate some of the storage limitations of the Hopfield network, and develop alternative architectures and an algorithm for designing the associative memory. We propose a new unsupervised learning method for neural networks. The method is based on applying repeatedly the gradient ascent technique on a defined criterion function. We study some of the dynamical aspects of Hopfield networks. New stability results are derived. Oscillations and synchronizations in several architectures are studied, and related to recent findings in biology. The problem of recording the outputs of real neural networks is considered. A new method for the detection and the recognition of the recorded neural signals is proposed.
https://thesis.library.caltech.edu/id/eprint/3725Wiring considerations in analog VLSI systems, with application to field-programmable networks
https://resolver.caltech.edu/CaltechETD:etd-07122007-134330
Authors: {'items': [{'id': 'Sivilotti-M-A', 'name': {'family': 'Sivilotti', 'given': 'Massimo Antonio'}, 'show_email': 'NO'}]}
Year: 1991
DOI: 10.7907/stj4-kh72
This thesis develops a theoretical model for the wiring complexity of wide classes of systems, relating the degree of connectivity of a circuit to the dimensionality of its interconnect technology. This model is used to design an efficient, hierarchical interconnection network capable of accommodating large classes of circuits. Predesigned circuit elements can be incorporated into this hierarchy, permitting semi-customization for particular classes of systems (e.g., photoreceptors included on vision chips). A polynomial-time programming algorithm for embedding the desired circuit graph onto the prefabricated routing resources is presented, and is implemented as part of a general design tool for specifying, manipulating and comparing circuit netlists.
This thesis presents a system intended to facilitate analog circuit design. At its core is a VLSI chip that is electrically configured in the field by selectively connecting predesigned elements to form a desired circuit, which is then tested electrically. The system may be considered a hardware accelerator for simulation, and its large capacity permits testing system ideas, which is impractical using current means. A fast-turnaround simulator permitting rapid conception and evaluation of circuit ideas is an invaluable aid to developing an understanding of system design in a VLSI context.
We have constructed systems using both reconfigurable interconnection switches and laser-programmed interconnect. Prototypes capable of synthesizing circuits consisting of over 1000 transistors have been constructed. The flexibility of the system has been demonstrated, and data from parametric tests have proven the validity of the approach.
Finally, this thesis presents several new circuits that have become key components in many analog VLSI systems. Fast, dense and provably safe one-phase latches and hierarchical arbiters are presented, as are a low-noise analog switch, an isotropic novelty filter, a dense, active high-resistance element, and a subthreshold differential amplifier with a large linear input range.
https://thesis.library.caltech.edu/id/eprint/2863Analog VLSI circuits for sensorimotor feedback
https://resolver.caltech.edu/CaltechETD:etd-06212007-074949
Authors: {'items': [{'id': 'DeWeerth-S-P', 'name': {'family': 'DeWeerth', 'given': 'Stephen P.'}, 'show_email': 'NO'}]}
Year: 1991
DOI: 10.7907/vvye-b883
This thesis presents a design framework and circuit implementations for integrating sensory and motor processing onto very large-scale integrated (VLSI) chips. The designs consist of analog circuits that are composed of bipolar and subthreshold MOS transistors. The primary emphasis in this work is the transformation from the spatially-encoded representation found in sensory images to a scalar representation that is useful for controlling motor systems.
The thesis begins with a discussion of the aggregation of sensory signals and the resulting extraction of high-level features from sensory images. An integrated circuit that computes the centroid of a visual image is presented. A theoretical analysis of the function of this circuit in stimulus localization and a detailed error analysis are also presented. Next, the control of motors using pulse trains is discussed. Pulse-generating circuits for use in bidirectional motor control and the implementation of traditional control schemes are presented. A method for analyzing the operation of these controllers is also discussed. Finally, a framework for the combination of sensory aggregation and pulse-encoded outputs is presented. The need for signal normalization and circuits to perform this task are discussed. Two complete sensorimotor feedback systems are presented.https://thesis.library.caltech.edu/id/eprint/2672Deterministic annealing, clustering, and optimization
https://resolver.caltech.edu/CaltechETD:etd-07122007-085228
Authors: {'items': [{'email': 'rose@ece.ucsb.edu', 'id': 'Rose-K', 'name': {'family': 'Rose', 'given': 'Kenneth'}, 'show_email': 'NO'}]}
Year: 1991
DOI: 10.7907/8N1R-3G60
This work introduces the concept of deterministic annealing (DA) as a useful approach to clustering and other related optimization problems. It is strongly motivated by analogies to statistical physics, but is formally derived within information theory and probability theory. This approach enables escaping local optima that plague traditional techniques, without the extremely slow schedules typically required by stochastic methods. The clustering solutions obtained by DA are totally independent of the choice of initial configuration.
A probabilistic framework is constructed, which is based on the principle of maximum entropy. The association probabilities at a given average distortion are Gibbs distributions parametrized by the corresponding Lagrange multiplier [beta], which is inversely proportional to the temperature in the physical analogy. By computing marginal probabilities within the framework, an effective cost is obtained, which is minimized to find the most probable set of cluster representatives at a given temperature. This effective cost is the free energy in statistical mechanics, which is indeed optimized at isothermal, stochastic equilibrium.
Within the probabilistic framework, annealing is introduced by controlling the Lagrange multiplier [beta]. This annealing is interpreted as gradually reducing the "fuzziness" of the associations. Phase transitions are identified in the process, which are, in fact, cluster splits. A sequence of phase transitions produces a hierarchy of fuzzy-clustering solutions. Critical [beta] are computed exactly for the first phase transition and approximately for the following ones.
Specific algorithms are derivable from the general approach, to address different aspects of clustering in the large variety of application fields. Here, algorithms are derived, and simulation results are presented for the three major classes, namely, hard clustering, fuzzy clustering, and hierarchical clustering. From the experimental results it appears that DA is substantially superior to traditional techniques.
The last part of the work extends the approach to deal with a larger family of optimization problems that can be reformulated as constrained clustering. A probabilistic framework for constrained clustering is derived based on the principle of maximum entropy. It is shown that for our annealing purpose, the constraint can be directly applied to the free energy. Three examples of constrained clustering are discussed. Mass-constrained clustering is formulated and yields an improvement of the clustering procedure. The process is now independent of the number of representatives and their multiplicity in the clusters. Secondly, the travelling salesman problem (TSP) is reformulated as constrained clustering, yielding the elastic net approach. A second Lagrange multiplier is identified, which is used to obtain a more powerful annealing method. Finally, self-organization of neural networks is shown to be closely related to TSP, and a similar annealing method is suggested. A fuzzy solution is sought to obtain the optimal net for a given training data set.
https://thesis.library.caltech.edu/id/eprint/2858Large operand division and an asynchrous approach to fault detection
https://resolver.caltech.edu/CaltechETD:etd-06212007-143920
Authors: {'items': [{'id': 'Kramer-K-A', 'name': {'family': 'Kramer', 'given': 'Kathleen A.'}, 'show_email': 'NO'}]}
Year: 1991
DOI: 10.7907/pt8p-9g83
Larger, faster ICs are creating a rash of new problems for the system designer. Designers faced with building larger and larger systems base their architectures on smaller systems that may scale poorly. As a result of VLSI, many new architectures are coming into favor, either because of the changing importance of design factors or because it is now possible to design bigger chips.
Efficient VLSI methods for implementing the basic arithmetic operations can push back many system-performance limitations. There is continued need for re-evaluation of arithmetic architectures, as the efficiency of implementation is related to both implementation technology and size of the operands. A new binary divider for n-bit integer operands, which produces the quotient and remainder in O(n) time using O(n) area, is presented. For very large operands, such as those required in Public Key Cryptography, the new divider is faster than comparable carry-save dividers and is more area-efficient than implementations using more redundant arithmetic.
A further problem faced by the designer of very large systems is their susceptibility to error. The system must be efficiently designed to function in the presence of errors, which become more likely as the size of the system increases. Qualities inherent in many asynchronous designs can be used to provide fault detection and therefore, fault tolerance. An approach to fault tolerance, one not possible with conventional, clocked, systolic arrays, is presented. This method of fault detection/correction exploits the inherent redundancy of architectures using four-state coding, a data-driven technique for implementing bit-level wave-front arrays.https://thesis.library.caltech.edu/id/eprint/2678Generalization capability of neural networks
https://resolver.caltech.edu/CaltechETD:etd-07202007-143215
Authors: {'items': [{'email': 'jichuanyi@gatech.edu', 'id': 'Ji-Chuanyi', 'name': {'family': 'Ji', 'given': 'Chuanyi'}, 'show_email': 'YES'}]}
Year: 1992
DOI: 10.7907/HR3F-0410
<p>The generalization capability of feedforward multilayer neural networks is investigated from two aspects: the theoretical aspect and the algorithmic aspect.</p>
<p>In the theoretical part, a general relation is derived between the so-called VC-dimension and the statistical lower epsilon-capacity, and then applied to two cases. First, as a general constructive approach, it is used to evaluate a lower bound of the VC-dimension of two layer networks with binary weights and integer thresholds. Second, how the sample complexity may vary with respect to distributions is investigated through analyzing a particular network which separates two binary clusters. Bounds for the capacity of two layer networks with binary weights and integer thresholds are also obtained.</p>
<p>In the algorithmic part, a network reduction algorithm is developed to study generalization in learning analog mappings. It is applied to control a two-link manipulator to draw characters. The network addition-deletion algorithm is described to find an appropriate network structure during learning. It is used to study the effect of sizes of networks on generalization, and applied to various classification problems including hand written digits recognition.</p>https://thesis.library.caltech.edu/id/eprint/2956Invariance hints and the VC dimension
https://resolver.caltech.edu/CaltechETD:etd-07202007-075240
Authors: {'items': [{'id': 'Fyfe-W-J-A', 'name': {'family': 'Fyfe', 'given': 'William John Andrew'}, 'show_email': 'NO'}]}
Year: 1992
DOI: 10.7907/ft2z-te28
We are interested in having a neural network learn an unknown function f. If the function satisfies an invariant of some sort, such as f is an odd function, then we want to be able to take advantage of this information and not have the network deduce the invariant based on an example of f.
The invariant might be defined in terms of an explicit transformation of the input space under which f is constant. In this case it is possible to build a network that necessarily satisfies the invariant.
In general, we define the invariant in terms of a partition of the input space such that if x, x' are in the same partition element then f(x) = f(x'). An example of the invariant would be a a pair (x, x') taken from a single partition element. We can combine examples of the invariant with examples of the function in the learning process. The goal is to substitute examples of the invariant for examples of the function; the extent to which we can actually do this depends on the appropriate VC dimensions. Simulations verify, at least in simple cases, that examples of the invariant do aid the learning process.https://thesis.library.caltech.edu/id/eprint/2950Efficient multichannel methods for high-rate data transmission with application to ISDN (or) pouring water to get more out of copper
https://resolver.caltech.edu/CaltechETD:etd-08082007-081649
Authors: {'items': [{'id': 'Ramesh-R', 'name': {'family': 'Ramesh', 'given': 'Rajaram'}, 'show_email': 'NO'}]}
Year: 1992
DOI: 10.7907/s4r5-wm48
In this thesis, we are concerned with the transmission of data over channels with intersymbol interference. We consider input signals which are multiplexed versions of several parallel input signals, with the aim of splitting the input signal spectrum into disparate frequency bands and shaping the input spectrum by adjusting the power on each of the frequency bands. We introduce a multirate signal processing framework for the representation of the channel under these conditions and derive simple equivalents for the channel and the associated processors.
Using the equivalent circuits, we derive simple equalization schemes for the channel by drawing from the theory of polynomial matrices. We show that vector equalization can be reduced to a combination of prefiltering, postfiltering and scalar equalization of a few of the parallel input signals. We also discuss several interesting properties of this decomposition.
In the case when the channel is corrupted by colored noise, we derive expressions for the optimum prefilters and postfilters with decision feedback equalization that minimize the mean-squared error between the input and the output, given a constraint on the input power. For uncorrelated inputs, the scheme leads to a set of parallel independent scalar channels with the optimum postfilter whitening the noise, which permits the optimal use of trellis codes for data transmission.
We apply the scheme to a special channel, viz., the ISDN digital subscriber loop. The main impairments on this channel are intersymbol interference and crosstalk due to adjacent loops in the same binder group. Crosstalk is an especially interesting case of noise since it depends on the signal being transmitted; we assume that all loops in a binder group transmit using the same scheme. We consider two cases of crosstalk noise: when transmission between different loops in a binder group is synchronized, the crosstalk noise is wide-sense cyclostationary, and with a lack of synchronization between loops, the crosstalk noise is wide-sense stationary. We present methods to determine the optimum filters for data transmission and the optimum input power distributions for both these cases. We demonstrate the possibility of data transmission at the T1 rate, i.e., 1.544 Mb/s over most loops in the local loop plant. We also find that synchronizing transmission between different loops in a binder group does not get us much; the difference in the throughputs for the cases of cyclostationary crosstalk and wide-sense stationary crosstalk does not seem to justify the effort involved in synchronization.https://thesis.library.caltech.edu/id/eprint/3052Testing delay-insensitive circuits
https://resolver.caltech.edu/CaltechETD:etd-07202007-132706
Authors: {'items': [{'id': 'Hazewindus-P-J', 'name': {'family': 'Hazewindus', 'given': 'Pieter Johannes'}, 'show_email': 'NO'}]}
Year: 1992
DOI: 10.7907/0d7v-9d09
A method is developed to test delay-insensitive circuits, using the single stuck-at fault model. These circuits are synthesized from a high-level specification. Since the circuits are hazard-free by construction, there is no test for hazards in the circuit. Most faults cause the circuit to halt during test, since they cause an acknowledgement not to occur when it should. There are stuck-at faults that do not cause the circuit to halt under any condition. These are stimulating faults; they cause a premature firing of a production rule. For such a stimulating fault to be testable, the premature firing has to be propagated to a primary output. If this is not guaranteed to occur, then one or more test points have to be added to the circuit. Any stuck-at fault is testable, with the possible addition of test points. For combinational delay-insensitive circuits, finding test vectors is reduced to the same problem as for synchronous combinational logic. For sequential circuits, the synthesis method is used to find a test for each fault efficiently, to find the location of the test points, and to find a test that detects all faults in a circuit.
The number of test points needed to fully test the circuit is very low, and the size of the additional testing circuitry is small. A test derived with a simple transformation of the handshaking expansion yields high fault coverage. Adding tests for the remaining faults results in a small complete test for the circuit.https://thesis.library.caltech.edu/id/eprint/2955Affinity : a concurrent programming system for multicomputers
https://resolver.caltech.edu/CaltechETD:etd-08152007-074128
Authors: {'items': [{'id': 'Steele-C-S', 'name': {'family': 'Steele', 'given': 'Craig S.'}, 'show_email': 'NO'}]}
Year: 1992
DOI: 10.7907/syrm-sx30
Affinity is an experiment to explore a simple, convenient, and expressive programming model that provides adequate power for complex programming tasks while setting few constraints on potential concurrency. Although the programmer is required to formulate a computational problem explicitly into medium-sized pieces of data and code, most of the additional functions necessary for concurrent execution are implicit. The execution of the light-weight, reactive processes, called actions, implicitly induces atomicity and consistency of data modifications. The programmer accesses shared data structures in a shared-memory fashion, but without the need for explicit locking to manage the problems of concurrent access and mutual exclusion. Program control flow is distributed and implicit.
The name given to the programming model, Affinity, has a definition, "causal connection or relationship," that is fitting to the way programs are structured and scheduled.
Affinity consistency and coherence properties provide a tractable discipline for the dangerous power of a concurrent, shared-memory programming style. Existing programming complexity-management techniques such as object-oriented languages can be used in this multicomputer environment. Affinity programs can compute consistent and correct results despite staleness of data, and asynchrony and nondeterminism in execution of code. Program correctness is invariant under replication, or cloning, of actions. This aspect of the model yields a simple and robust mechanism for fault-tolerance.
The practicality of the Affinity programming model has been demonstrated by an implementation on a second-generation multicomputer, the Ametek S/2010. The implementation is distributed, scalable, and relatively insensitive to network latency. Affinity has demonstrated reasonable efficiency and performance for computations with tens of processing nodes, hundreds of actions, and thousands of shared data structures.https://thesis.library.caltech.edu/id/eprint/3136On the VLSI decompositions for complete graphs, DeBruijn graphs, hypercubes, hyperplanes, meshes, and shuffle-exchange graphs
https://resolver.caltech.edu/CaltechETD:etd-08302007-094049
Authors: {'items': [{'id': 'Ko-T', 'name': {'family': 'Ko', 'given': 'Tsz-Mei'}, 'show_email': 'NO'}]}
Year: 1993
DOI: 10.7907/s7w7-a995
A C-chip VLSI decomposition of a graph G is a collection of C vertex-disjoint subgraphs of G which together contain all of G's vertices and a subset of its edges. If the vertex-disjoint subgraphs are isomorphic to each other, we call one of these isomorphic subgraphs a building block. The efficiency of a VLSI decomposition is defined to be the fraction of edges of G that are in the subgraphs. In this thesis, motivated by the need to construct large Viterbi decoders, we study VLSI decompositions for deBruijn graphs. We obtain some strong necessary conditions for a graph to be a building block for a deBruijn graph, and some slightly more restrictive sufficient conditions which allow us to construct some efficient building blocks for deBruijn graphs. By using the methods described in this thesis, we have found a 64-chip VLSI decomposition of the deBruijn graph B13 with efficiency 0.754. This decomposition is being used by JPL design engineers to build a single-board Viterbi decoder for the K = 15, rate 1/4 convolutional code which will be used on NASA's Galileo mission.
Furthermore, we study VLSI decompositions for the families of complete graphs, hypercubes, hyperplanes, meshes, and shuffle-exchange graphs. In each of these cases, we obtain very efficient or even optimal decompositions. We also prove several general theorems that can be applied to obtain bounds on the efficiencies for VLSI decompositions of other complex graphs. In general, the results presented in this thesis are useful for implementing massively parallel computers.
https://thesis.library.caltech.edu/id/eprint/3284Multidimensional multirate filters and filter banks : theory, design, and implementation
https://resolver.caltech.edu/CaltechETD:etd-08232007-095226
Authors: {'items': [{'email': 'tsuhan@cmu.edu', 'id': 'Chen-Tsuhan', 'name': {'family': 'Chen', 'given': 'Tsuhan'}, 'show_email': 'YES'}]}
Year: 1993
DOI: 10.7907/XHE8-RB96
Multidimensional (MD) multirate systems, which find applications in the coding and compression of image and video data, and in high definition television (HDTV) systems, have recently attracted much attention. Central to these systems is the idea of sampling lattices. The basic building blocks in an MD multirate system are the decimation matrix M, the expansion matrix L, and MD digital filters. When M and L are diagonal, most of the one-dimensional (1D) multirate results can be extended automatically, using separable approaches (i.e., separate operations in each dimension). Separable approaches are commonly used in practice due to their low complexity in implementation. However, nonseparable operations, with respect to nondiagonal decimation and expansion matrices, often provide more flexibility and better performance. Several applications, such as the conversion between progressive and interlaced video signals, actually require the use of nonseparable operations. For the nonseparable case, extensions of 1D results to the MD case are nontrivial. In this thesis, we will introduce some developments in these extensions. The three main results are: the design of nonseparable MD filters and filter banks derived from 1D filters, the commutativity of MD decimators and expanders and its applications to the efficient polyphase implementation of MD rational decimation systems, and the vector space framework for unifying MD filter bank and wavelet theory. In particular, properties of integer matrices like matrix fraction descriptions, coprimeness, the Bezout identity, etc., of which the polynomial versions are known in system theory, are used for the first time in the area of multirate signal processing.
https://thesis.library.caltech.edu/id/eprint/3207A comparison of CDMA and frequency hopping in a cellular environment
https://resolver.caltech.edu/CaltechETD:etd-08302007-154436
Authors: {'items': [{'id': 'Mandell-M-I', 'name': {'family': 'Mandell', 'given': 'Michael I.'}, 'show_email': 'NO'}]}
Year: 1993
DOI: 10.7907/xsbz-qt92
This paper compares the performances of Direct Sequence Code Division Multiple Access (CDMA) and Frequency Hopping (FH) schemes in a cellular multiuser environment. Our multiuser channel model incorporates the effects of propagation, frequency selective fading, and interference among users in the presence of a constrained system bandwidth. This channel model is applicable for cellular mobile communications, as well as other forms of personal communications. The CDMA and FH systems are compared using BPSK modulation. The main point of contrast between these systems is that the orthogonal hopping patterns in a FH system result in a decreased additive interference power, however, the frequency spreading nature of CDMA results in both the ability to combat fading, and the ability to effectively use low rate codes. An information-theoretic analysis is presented, and shows that the system capacity is larger for CDMA than for FH. Hence, with sufficient coding the CDMA system can achieve a higher level of performance than the FH system. However, it is unclear what level of complexity would be required to achieve such performance, and what effect such complexity would have on the practicality of the system. Finally, through the use of simulation, the performances of several simple coding schemes are measured and compared to the theoretical limits. These simple coding schemes perform far below the theoretical limits and also display a tradeoff in performance where the FH system performs better at high levels of traffic, and the CDMA system performs better at low levels of traffic.https://thesis.library.caltech.edu/id/eprint/3288Classification and Approximation with Rule-Based Networks
https://resolver.caltech.edu/CaltechETD:etd-08272007-132407
Authors: {'items': [{'id': 'Higgins-Charles-Marion', 'name': {'family': 'Higgins', 'given': 'Charles Marion'}, 'show_email': 'NO'}]}
Year: 1993
DOI: 10.7907/4r7r-w573
<p>This thesis describes the architecture of learning systems which can explain their decisions through a rule-based knowledge representation. Two problems in learning are addressed: pattern classification and function approximation.</p>
<p>In Part I, a pattern classifier for discrete-valued problems is presented. The system utilizes an information-theoretic algorithm for constructing informative rules from example data. These rules are then used to construct a computational network to perform parallel inference and posterior probability estimation. The network can be extended incrementally; that is, new data can be incorporated without repeating the training on previous data. It is shown that this technique performs comparably with other techniques including the backpropagation network while having unique advantages in incremental learning capability, training efficiency, and knowledge representation. Examples are shown of rule-based classification and explanation.</p>
<p>In Part II, we present a method for the learning of fuzzy logic membership functions and rules to predict a numerical function from examples of the function and its independent variables. This method uses a three-step approach to building a complete function approximation system: first, learning the membership functions and creating a cell-based rule representation; second, simplifying the cell-based rules using an information-theoretic approach for induction of rules from discrete-valued data; and finally, constructing a computational network to compute the function value given its independent variables. Applications of the system to adaptive control are suggested, including a method for learning a complete control system for an unknown plant. Experimental validation of the suggested methods using a ball-and-beam system is shown.</p>https://thesis.library.caltech.edu/id/eprint/3245Subspace Subcodes of Reed-Solomon Codes
https://resolver.caltech.edu/CaltechETD:etd-07172007-105811
Authors: {'items': [{'id': 'Hattori-Masayuki', 'name': {'family': 'Hattori', 'given': 'Masayuki'}, 'show_email': 'NO'}]}
Year: 1995
DOI: 10.7907/m50j-xx64
<p>In this paper we introduce a new class of non-linear cyclic error-correcting codes, which we call subspace subcodes of Reed-Solomon (SSRS) codes. An SSRS code is a subset of a parent Reed-Solomon (RS) code consisting of codewords whose components all lie in a fixed v-dimensional vector subspace S of GF(2ᵐ).</p>
<p>Starting from a (n, k₀, d₀) RS code over GF(2ᵐ), with any positive integer 0 ≤ v ≤ m, there is an SSRS code of length n and designed minimum distance d₀ over the symbol alphabet S, the vector space of binary v-tuples. SSRS codes are constructed using properties of the Galois field GF(2ᵐ). SSRS codes are not linear over GF(2ᵛ) but are Abelian group codes over S. However, they are linear over GF(2), and the symbol-wise cyclic shift of any codeword is also a codeword.</p>
<p>Our first main result is an explicit formula for the dimension of an SSRS code. It is followed by a corollary which gives a simple lower bound, which gives the true value for "most" subspaces. We also prove several important duality properties.</p>
<p>Next, we give a classification of the v-dimensional subspaces of GF(2ᵐ) into categories, such that two subspaces in the same category always produce isometric SSRS codes. Then, we give an efficient means to find the "exceptional" subspaces among the huge number of subspaces. We also present a reasonably simple encoding algorithm that works for systematic shortened linear codes in general.</p>
<p>Finally, we present some numerical examples, which show, among other things, that (1) SSRS codes can have a higher dimension than comparable GBCH codes, so that even if GF(2ᵛ) is a subfield of GF(2ᵐ), it may not be the "best" v-dimensional subspace for constructing SSRS codes; and (2) many high-rate SSRS codes have larger dimension than any previously known code with the same values of n, d and q, including algebraic-geometry codes. These examples suggest that high-rate SSRS codes are likely candidates to replace Reed-Solomon codes in high-performance transmission and storage systems.</p>https://thesis.library.caltech.edu/id/eprint/2912Semantics of VLSI synthesis
https://resolver.caltech.edu/CaltechETD:etd-10162007-093427
Authors: {'items': [{'email': 'vdgoot@earthlink.net', 'id': 'Van-der-Goot-M-R', 'name': {'family': 'Van der Goot', 'given': 'Marcel Rene'}, 'show_email': 'YES'}]}
Year: 1995
DOI: 10.7907/SR5V-KT18
We develop a new form of formal operational semantics, suitable for concurrent programming languages. The semantics directly supports sequential and parallel composition, rendezvous synchronization, shared variables, and non-determinism. Based on an abstract notion of program execution, a refinement relation is defined. We show how the refinement relation can be used to prove that one program implements another.
We use the operational semantics as a semantic framework for a synthesis method for asynchronous VLSI circuits. We define the semantics of the programming notations that are used, and use the refinement relation to prove the correctness of the program transformations that form the basis of the synthesis method. Among other transformations, we proof the correctness of the replacement of atomic synchronization actions by handshake protocols, and the transformation of a sequence of actions into a network of concurrently executing gates.
https://thesis.library.caltech.edu/id/eprint/4110Stochastic Computation
https://resolver.caltech.edu/CaltechETD:etd-02202004-150303
Authors: {'items': [{'email': 'jcort@ll.mit.edu', 'id': 'Cortese-John-Anthony', 'name': {'family': 'Cortese', 'given': 'John Anthony'}, 'show_email': 'NO'}]}
Year: 1995
DOI: 10.7907/W627-YA05
<p>This thesis approaches computation from a communication theory perspective. Data is given to a computer, which is asked to arrive at a binary hypothesis decision. The computation task is viewed as a signal drawn from an ensemble, corrupted by noise, and passed to a receiver which is asked to make a binary signal detection decision.</p>
<p>To illustrate the approach, learning in a neural network is studied. An algorithm based on statistical communication techniques is developed which allows the determination of the neural network size, architecture, and system parameters. The computation, as interpreted in the communication framework, is assigned an equivalent channel capacity which measures the effectiveness with which the computing system extracts information in the Shannon sense from the input data. Numerical simulations of a neural network recognizing handwritten digits are used to illustrate key points.</p>https://thesis.library.caltech.edu/id/eprint/684VLSI systems for analog and Hamming parallel computation
https://resolver.caltech.edu/CaltechETD:etd-10172007-153538
Authors: {'items': [{'email': 'volnei.pedroni@gmail.com', 'id': 'Pedroni-V-A', 'name': {'family': 'Pedroni', 'given': 'Volnei A.'}, 'show_email': 'NO'}]}
Year: 1995
DOI: 10.7907/DC5Z-3Q89
This thesis explores the vast field of physically implementing parallel-computing algorithms. In this research, we introduce a series of new circuit architectures and new technology applications, which implement multi-dimensional functions that are at the heart of many parallel signal processing systems, e.g., neural and Hamming networks, vector quantizers, and median filters. The functions are realized using low-cost, low-power, high-density technologies (CMOS and CCD), fully compatible with current industrial processes. The systems are either analog or hybrid, allowing lower time and/or storage complexities in many types of applications when compared to fully digital systems. Special emphasis is placed on circuit modeling, with the purpose of thoroughly understanding the potentialities - and limitations - of each alternative. The models are verified experimentally on most occasions. As a consequence, the results presented in this dissertation are expected not only to provide new technological alternatives, but also new means of evaluating the technologies themselves.
Chapter 1 presents an introductory discussion on parallel systems. It has three main purposes. One is to describe some of the parallel functions whose implementations we are interested in. Another is to present a graphical discussion on how certain multidimensional systems work, which is probably the best way of describing - and appreciating - systems of this kind. And finally to describe basic guidelines concerning this research.
Chapter 2 discusses a function that is inherent to most analog parallel processors, the winner-take-all function. The reason for it to be developed first is that this function is part of many other function realizations. A global discussion is presented, which provides an overview on the potentialities of most implementations available in CMOS technology, followed by high-resolution alternatives. The use of this function to implement other functions and systems is also illustrated.
Chapter 3 presents a detailed discussion on charge-coupled device (CCD) technology and its applications to parallel signal processing systems. This technology, compatible with conventional double-poly CMOS, is of interest due to its low power consumption and very high integration density, allowing the construction of very efficient vector-matrix multipliers and Hamming networks. To overcome its main limitation (i.e., charge-transfer inefficiency), a locally-controlled architecture is introduced. Several chips and extensive measurements are shown, with the purpose of concretely evaluating the performance of this technology when performing signal processing tasks.
Finally, Chapter 4 describes further research on CMOS cells that compute distance-based functions. These circuits allow the construction of LMS and other distance-based parallel processors, and provide additional valuable means of further examining the use of MOS technology for analog computation. Once again experimental results are presented, and the systems are illustrated through vector quantizers, Hamming networks, vector multipliers, and median filters. This chapter also provides further applications of the winner-take-all function to the construction of more complex functions.
https://thesis.library.caltech.edu/id/eprint/4145Synchronizing processes
https://resolver.caltech.edu/CaltechETD:etd-10112007-083903
Authors: {'items': [{'email': 'hofstee@us.ibm.com', 'id': 'Hofstee-H.-Peter', 'name': {'family': 'Hofstee', 'given': 'H. Peter'}, 'show_email': 'YES'}]}
Year: 1995
DOI: 10.7907/G620-GG65
In this monograph we develop a mathematical theory for a concurrent language based on angelic and demonic nondeterminism. An underlying model is defined with sets of sets of sequences of synchronization actions. A refinement relation is defined for the model, and equivalence classes under this relation are identified with processes. Processes, together with the refinement relation, form a complete distributive lattice.
We define a language with parallel composition, sequential composition, angelic and demonic nondeterminism, and an operator that connects pairs of synchronization actions into synchronization statements and hides these actions from observation. Also, angelic and demonic iteration are defined. All operators are monotonic with respect to the refinement ordering. Many algebraic properties are proven from these definitions. We study duals of processes and prove that they can be related to the most demonic environment in which a process will not deadlock. We give a simple example to illustrate the use of duals.
We study classes of programs for which angelic choice can be implemented by probing the environment for its next action. To this end specifications of processes are extended with simple conditions on the environment. We give a more elaborate example to illustrate the use of these conditions and the compositionality of the method.
Finally we briefly introduce an operational model that describes implementable processes only. This model mentions probes explicitly. Such a model may form a basis for a language that is less restrictive than ours, but that will also have less attractive algebraic properties.
https://thesis.library.caltech.edu/id/eprint/4036A General Approach to Performance Analysis and Optimization of Asynchronous Circuits
https://resolver.caltech.edu/CaltechETD:etd-10172007-090528
Authors: {'items': [{'id': 'Lee-Tak-Kwan', 'name': {'family': 'Lee', 'given': 'Tak Kwan'}, 'show_email': 'NO'}]}
Year: 1995
DOI: 10.7907/ehzs-y537
A systematic approach for evaluating and optimizing the performance of asynchronous VLSI circuits is presented. Index-priority simulation is introduced to efficiently find minimal cycles in the state graph of a given circuit. These minimal cycles are used to determine the causality relationships between all signal transitions in the circuit. Once these relationships are known, the circuit is then modeled as an extended event-rule system, which can be used to describe many circuits, including ones that are inherently disjunctive. An accurate indication of the performance of the circuit is obtained by analytically computing the period of the corresponding extended event-rule system.
https://thesis.library.caltech.edu/id/eprint/4136Switching algorithms and buffer management in asynchronous transfer mode networks
https://resolver.caltech.edu/CaltechETD:etd-12122007-145855
Authors: {'items': [{'id': 'Erimli-B', 'name': {'family': 'Erimli', 'given': 'Bahadir'}, 'show_email': 'NO'}]}
Year: 1996
DOI: 10.7907/agbk-bn50
In this thesis, two different but related concepts in Asynchronous Transfer Mode (ATM) are discussed. Due to its multirate nature, ATM creates new problems in terms of switching and buffering. In the first part, the switching problems are investigated. The situation is rooted upon the multirate connections in a 'circuit-switching-like' environment. The multirate nature of ATM results in the loss of strictly nonblocking three stage space-division switches unless a 'speed-up factor' is provided between the outside ports and the internal links of the switch. To keep this factor to a minimum, call routing algorithms are considered as a possible solution. Several call routing algorithms are compared in terms of their blocking probability under various circumstances. A simple algorithm, named fixed priority routing algorithm, stands out among these, both in terms of simplicity and low blocking rate. Afterwards a bin packing model is used to investigate the reasons behind this.
In the second part, buffer management in ATM nodes is considered. In the traditional sense, the burstier the traffic is, the higher, it was believed, the cell loss will be at a buffer into which a number of these sources are transmitting. It is shown that this is not always the case and under the circumstances defined - the worst-case model - other types of sources that output traffic that is less bursty might create higher cell loss than burstier sources. All sources considered are leaky-bucket controlled and stay within their contract limits with the network at all times. Initially greedy on-off source and the three-state source types are compared. After establishing that the comparison between these two in terms of cell loss rate is highly dependent on the size of the buffer being transmitted onto, other source types that might create even higher cell loss rates are searched for. One such characteristic group of sources is found and is presented.https://thesis.library.caltech.edu/id/eprint/4977Models of visual feature detection and spike coding in the nervous system
https://resolver.caltech.edu/CaltechETD:etd-09212006-152641
Authors: {'items': [{'email': 'tom@tomannau.com', 'id': 'Annau-T-M', 'name': {'family': 'Annau', 'given': 'Thomas Mark'}, 'show_email': 'YES'}]}
Year: 1996
DOI: 10.7907/CN6R-WE94
We propose mathematical models to analyze two nervous system phenomena. The first is a model of the development and function of simple cell receptive fields in mammalian primary visual cortex. The model assumes that images are composed of combinations of a limited set of specific visual features and that the goal of simple cells is to detect the presence or absence of these features. Based on a presumed statistical character of images and their visual features, the model uses a constrained Hebbian learning rule to discover the structure of the features, and thus the appropriate response properties of simple cells, by training on a database of photographs. The response properties of the model simple cells agree qualitatively with neurophysiological observation.
The second is a model of the coding of information in the nervous system by the rate of axonal voltage spikes. Assuming an integrate-and-fire mechanism for spike generation, we develop a quantization-based model of rate coding and use it to derive the mathematical relationship between the amplitude and temporal resolution of a rate encoded signal. We elaborate the model to include integrator leak in the spike generation mechanism and show that it compactly combines coding and the computation of a threshold function.
https://thesis.library.caltech.edu/id/eprint/3681Neurally inspired silicon learning : from synapse transistors to learning arrays
https://resolver.caltech.edu/CaltechETD:etd-01092008-080326
Authors: {'items': [{'id': 'Diorio-Christopher-J', 'name': {'family': 'Diorio', 'given': 'Christopher J.'}, 'show_email': 'NO'}]}
Year: 1997
DOI: 10.7907/vbyq-fy15
A computation is an operation that can be performed by a physical machine. We are familiar with digital computers: Machines based on a simple logic function (the binary NOR) and optimized for manipulating numeric variables with high precision. Other computing machines exist: The neurocomputer, the analog computer, the quantum computer, and the DNA computer all are known. Neurocomputers-defined colloquially as computing machines comprising nervous tissue-exist; that they are computers also is certain. Nervous tissue solves ill-posed problems in real time. The principles underlying neural computation, however, remain for now a mystery.
I believe that there are fundamental principles of computation that we can learn by studying neurobiology. If we can understand how biological information-processing systems operate, then we can learn how to build circuits and systems that deal naturally with real-world data. My goal is to investigate the organizational and adaptive principles on which neural systems operate, and to build silicon integrated circuits that compute using these principles. I call my approach silicon neuroscience: the development of neurally inspired silicon-learning systems.
I have developed, in a standard CMOS process, a family of single-transistor devices that I call synapse transistors. Like neural synapses, synapse transistors provide nonvolatile analog memory, compute the product of this stored memory and the applied input, allow bidirectional memory updates, and simultaneously perform an analog computation and determine locally their own memory updates. I have fabricated a synaptic array that affords a high synapse-transistor density, mimics the low power consumption of nervous tissue, and performs both fast, parallel computation and slow, local adaptation. Like nervous tissue, my array simultaneously and in parallel performs an analog computation and updates the nonvolatile analog memory.
Although I do not believe that a single transistor can model the complex behavior of a neural synapse completely, my synapse transistors do implement a local learning function. I consider their development to be a first step toward achieving my goal of a silicon learning system.
https://thesis.library.caltech.edu/id/eprint/88Model Reduction and Minimality for Uncertain Systems
https://resolver.caltech.edu/CaltechETD:etd-01042008-091550
Authors: {'items': [{'email': 'beck3@illinois.edu', 'id': 'Beck-Carolyn-Louise', 'name': {'family': 'Beck', 'given': 'Carolyn Louise'}, 'orcid': '0000-0003-4880-6380', 'show_email': 'NO'}]}
Year: 1997
DOI: 10.7907/MPV7-2Q79
<p>The emphasis of this thesis is on the development of systematic methods for reducing the size and complexity of uncertain system models. Given a model for a large complex system, the objective of these methods is to find a simplified model which accurately describes the physical system, thus facilitating subsequent control design and analysis.</p>
<p>Model reduction methods and realization theory are presented for uncertain systems represented by Linear Fractional Transformations (LFTs) on a block diagonal uncertainty structure. A complete generalization of balanced realizations, balanced Gramians and balanced truncation model reduction with guaranteed error bounds is given, which is based on computing solutions to a pair of Linear Matrix Inequalities (LMIs). A necessary and sufficient condition for exact reducibility of uncertain systems, the converse of minimality, is also presented. This condition further generalizes the role of controllability and observability Gramians, and is expressed in terms of singular solutions to the same LMIs. These reduction methods provide a systematic means for both uncertainty simplification and state order reduction in the case of uncertain systems, but also may be interpreted as state order reduction for multi-dimensional systems.</p>
<p>LFTs also provide a convenient way of obtaining realizations for systems described by rational functions of several noncommuting indeterminates. Such functions arise naturally in robust control when studying systems with structured uncertainty, but also may be viewed as a particular type of description for a formal power series. This thesis establishes connections between minimal LFT realizations and minimal linear representations of formal power series, which have been studied extensively in a variety of disciplines, including nonlinear system realization theory. The result is a fairly complete development of minimal realization theory for LFT systems.</p>
<p>General LMI problems and solutions are discussed with the aim of providing sufficient background and references for the construction of computational procedures to reduce uncertain systems. A simple algorithm for computing balanced reduced models of uncertain systems is presented, followed by a discussion of the application of this procedure to a pressurized water reactor for a nuclear power plant.</p>
https://thesis.library.caltech.edu/id/eprint/28Recognition of visual object classes
https://resolver.caltech.edu/CaltechETD:etd-01092008-094943
Authors: {'items': [{'email': 'Michael.C.Burl@jpl.nasa.gov', 'id': 'Burl-M-C', 'name': {'family': 'Burl', 'given': 'Michael C.'}, 'show_email': 'YES'}]}
Year: 1997
DOI: 10.7907/96P7-6E62
<p>Humans can look at a scene or a photograph and easily recognize objects. Outside my window I can see cars, people walking a dog on a brick pathway, trees, buildings, etc. This perception is so effortless that it belies the difficulty of the task. Visual perception begins with light that is reflected from the scene into the eye. The light impinges upon the retina and is transduced by a two-dimensional array of photoreceptors into noisy electrical signals. The brain must then accomplish the difficult task of transforming from this low-level representation to a higher-level understanding of the scene in terms of regions, surfaces, textures, and objects.</p>
<p>For computer vision the problem is the same, but the hardware is different. A camera approximates the function of the eye and retina; that is, the camera produces a two-dimensional array of numbers (pixel values) representing the intensity of light reflected from the scene. The fundamental question addressed in this thesis is the following: what mathematical processing should be applied to the pixel values in order for a computer to recognize objects? The methods we propose are not intended as a model of human brain function, although they may provide some insight. We are simply trying to solve the same visual recognition problems as the brain without concern for whether (or how) our algorithms could be realized in neuronal "hardware."</p>
<p>We have developed a new framework for recognizing visual object classes in which the class members consist of characteristic parts in a deformable spatial configuration. Human faces are an object class of this type, since faces consist of eyes, nose, and mouth arranged in a configuration that varies depending on expression and pose and also from one person to another. A second object class is cursive handwriting, which consists of loops, cusps, crossings, etc. arranged in a deformable pattern. In our approach, the allowed object deformations are represented through shape statistics, which are learned from examples. Instances of an object in an image are detected by finding the appropriate features in the correct spatial configuration. Our algorithm is robust with respect to partial occlusion, detector false alarms, and missed features.</p>
<p>Potential applications include intelligent tools for finding objects in image data-bases, human-machine interfaces, user authentication, intelligent data gathering and compression, signature verification, and keyword spotting. Experimental results will be presented for two problems: (1) locating quasi-frontal views of human faces in cluttered scenes and with occlusions and (2) spotting keywords in on-line cursive handwriting data.</p>https://thesis.library.caltech.edu/id/eprint/93One- and Two-Dimensional Cosine Modulated Filter Banks
https://resolver.caltech.edu/CaltechETD:etd-01102008-145701
Authors: {'items': [{'id': 'Lin-Yuan-Pei', 'name': {'family': 'Lin', 'given': 'Yuan-Pei'}, 'show_email': 'NO'}]}
Year: 1997
DOI: 10.7907/n255-7g92
<p>Subband coding as a lossy data compression technique was first introduced for speech coding. It has been demonstrated to be a very competitive coding method for general audio signals as well as images. Subband coding has been incorporated in various popular coding standards. Essential to the implementation of subband coding is an M-channel filter bank that partitions the input signal into M subbands. In the context of 1D (one-dimensional) filter bank design, the CMFB (cosine modulated filter bank) is well-known for design and implementation efficiency. All the filters in the filter bank are cosine modulated versions of a prototype filter. As a result the cost of design as well as complexity is reduced dramatically by a factor of M. In this thesis we study the design of CMFB in 1D case and 2D (two-dimensional) case.</p>
<p>In previous works on 1D CMFB, the filters in the filter bank do not have linear phase, which is considered an important feature in image coding applications. The design of cosine modulated filter banks with linear-phase filters is the first topic to be presented in this thesis. Design examples will be given to show that filter banks with filters having good frequency selectivity can be obtained in spite of the linear phase constraint.</p>
<p>For the design of 2D cosine modulated filter banks, the simplest approach is to cascade 1D filter banks in the form of a tree. This type of 2D filter banks are referred to as separable. The frequency support of the filters in a separable filter bank are restricted to rectangular shapes. Nonseparable filter banks allow more flexible partitions of the frequency plane and achieve better performance. Almost all the existing design techniques for 2D nonseparable filter banks are developed exclusively for the two-channel case. We will consider two types of 2D M-channel nonseparable filter banks, the two-parallelogram type and the four-parallelogram type. These are respectively the classes of filter banks in which the passbands of the filters consist of two and four parallelograms. In these designs additional cosine modulated constraints will be incorporated for design and implementation economy.</p>https://thesis.library.caltech.edu/id/eprint/116Efficient precise computation with noisy components : extrapolating from an electronic cochlea to the brain
https://resolver.caltech.edu/CaltechETD:etd-08092005-104717
Authors: {'items': [{'email': 'rahuls@MIT.EDU', 'id': 'Sarpeshkar-Rahul', 'name': {'family': 'Sarpeshkar', 'given': 'Rahul'}, 'show_email': 'NO'}]}
Year: 1997
DOI: 10.7907/FSBH-QA93
Low-power wide-dynamic-range systems are extremely hard to build. The cochlea is one of the most awesome examples of such a system: It can sense sounds over 12 orders of magnitude in intensity, with an estimated power dissipation of only a few tens of microwatts.
We describe an analog electronic cochlea that processes sounds over 6 orders of magnitude in intensity, while dissipating less than 0.5mW. This 117-stage, 100Hz-10Khz cochlea has the widest dynamic range of any artificial cochlea built to date. This design, using frequency-selective gain adaptation in a low-noise traveling-wave amplifier architecture, yields insight into why the human cochlea uses a traveling-wave mechanism to sense sounds, instead of using bandpass filters.
We propose that, more generally, the computation that is most efficient in its use of resources is an intimate hybrid of analog and digital computation. For maximum efficiency, the information and information-processing resources of the hybrid form of computation must be distributed over many wires, with an optimal signal-to-noise ratio per wire. These results suggest that it is likely that the brain computes in a hybrid fashion, and that an underappreciated and important reason for the efficiency of the human brain, which only consumes 12W, is the hybrid and distributed nature of its architecture.https://thesis.library.caltech.edu/id/eprint/3063Analysis, synthesis, and implementation of networks of multiple-input translinear elements
https://resolver.caltech.edu/CaltechETD:etd-01162008-075623
Authors: {'items': [{'id': 'Minch-B-A', 'name': {'family': 'Minch', 'given': 'Bradley Arthur'}, 'show_email': 'NO'}]}
Year: 1997
DOI: 10.7907/rh58-rz05
At the time of its invention in the seventeenth century, the logarithmic slide rule literally revolutionized the way calculation was done. From then until the advent of the pocket calculator, this analog computational device was widely used to perform multiplications and divisions, to raise numbers to fixed powers and extract fixed roots of numbers. Today, the slide rule may be gone, but it is not forgotten. In this thesis, I present a class of simple translinear network circuits which essentially function as electronic slide rules, accurately computing products, quotients, powers, and roots. I describe two different analysis procedures that allow us to determine the steady-state relationship between input and output currents. I also describe systematic techniques for synthesizing such circuits whereby we can produce a circuit whose steady-state transfer characteristics embody some desired product-of-power-law relationship between input and output currents. These circuits are made from multiple-input translinear elements; such elements produce output currents that are proportional to the exponential of a weighted sum of their input voltages. We can implement the weighted voltage summations with either resistive or capacitive voltage dividers. We can obtain the required exponential voltage-to-current transformations from either bipolar transistors or subthreshold MOS transistors. The subthreshold floating-gate MOS transistor naturally implements the exponential-of-a-weighted-sum operation in a single device. I will present experimental results from several of these translinear network circuits breadboarded from subthreshold floating-gate MOS transistors. I will also describe and present experimental data from a variety of other implementations of the multiple-input translinear element.
https://thesis.library.caltech.edu/id/eprint/199A structured approach to parallel programming
https://resolver.caltech.edu/CaltechETD:etd-01242008-074143
Authors: {'items': [{'id': 'Massingill-B-L', 'name': {'family': 'Massingill', 'given': 'Berna Linda'}, 'show_email': 'NO'}]}
Year: 1998
DOI: 10.7907/5ma9-h225
Parallel programs are more difficult to develop and reason about than sequential programs. There are two broad classes of parallel programs: (1) programs whose specifications describe ongoing behavior and interaction with an environment, and (2) programs whose specifications describe the relation between initial and final states. This thesis presents a simple, structured approach to developing parallel programs of the latter class that allows much of the work of development and reasoning to be done using the same techniques and tools used for sequential programs. In this approach, programs are initially developed in a primary programming model that combines the standard sequential model with a restricted form of parallel composition that is semantically equivalent to sequential composition. Such programs can be reasoned about using sequential techniques and executed sequentially for testing. They are then transformed for execution on typical parallel architectures via a sequence of semantics-preserving transformations, making use of two secondary programming models, both based on parallel composition with barrier synchronization and one incorporating data partitioning. The transformation process for a particular program is typically guided and assisted by a parallel programming archetype, an abstraction that captures the commonality of a class of programs with similar computational features and provides a class-specific strategy for producing efficient parallel programs. Transformations may be applied manually or via a parallelizing compiler. Correctness of transformations within the primary programming model is proved using standard sequential techniques. Correctness of transformations between the programming models and between the models and practical programming languages is proved using a state-transition-based operational model.
This thesis presents: (1) the primary and secondary programming models, (2) an operational model that provides a common framework for reasoning about programs in all three models, (3) a collection of example program transformations with arguments for their correctness, and (4) two groups of experiments in which our overall approach was used to develop example applications. The specific contribution of this work is to present a unified theory/practice framework for this approach to parallel program development, tying together the underlying theory, the program transformations, and the program-development methodology.
https://thesis.library.caltech.edu/id/eprint/321Intelligent holographic databases
https://resolver.caltech.edu/CaltechETD:etd-03172008-142604
Authors: {'items': [{'id': 'Barbastathis-George', 'name': {'family': 'Barbastathis', 'given': 'George'}, 'show_email': 'NO'}]}
Year: 1998
DOI: 10.7907/1R63-9H50
NOTE: Text or symbols not renderable in plain ASCII are indicated by [...]. Abstract is included in .pdf document.
Memory is a key component of intelligence. In the human brain, physical structure and functionality jointly provide diverse memory modalities at multiple time scales. How could we engineer artificial memories with similar faculties? In this thesis, we attack both hardware and algorithmic aspects of this problem.
A good part is devoted to holographic memory architectures, because they meet high capacity and parallelism requirements. We develop and fully characterize shift multiplexing, a novel storage method that simplifies disk head design for holographic disks. We develop and optimize the design of compact refreshable holographic random access memories, showing several ways that 1 Tbit can be stored holographically in volume less than 1 [...], with surface density more than 20 times higher than conventional silicon DRAM integrated circuits. To address the issue of photorefractive volatility, we further develop the two-lambda (dual wavelength) method for shift multiplexing, and combine electrical fixing with angle multiplexing to demonstrate 1,000 multiplexed fixed holograms. Finally, we propose a noise model and an information theoretic metric to optimize the imaging system of a holographic memory, in terms of storage density and error rate.
Motivated by the problem of interfacing sensors and memories to a complex system with limited computational resources, we construct a computer game of Desert Survival, built as a high-dimensional non-stationary virtual environment in a competitive setting. The efficacy of episodic learning, implemented as a reinforced Nearest Neighbor scheme, and the probability of winning against a control opponent improve significantly by concentrating the algorithmic effort to the virtual desert neighborhood that emerges as most significant at any time. The generalized computational model combines the autonomous neural network and von Neumann paradigms through a compact, dynamic central representation, which contains the most salient features of the sensory inputs, fused with relevant recollections, reminiscent of the hypothesized cognitive function of awareness. The Declarative Memory is searched both by content and address, suggesting a holographic implementation. The proposed computer architecture may lead to a novel paradigm that solves "hard" cognitive problems at low cost.
https://thesis.library.caltech.edu/id/eprint/986Monotonicity and connectedness in learning systems
https://resolver.caltech.edu/CaltechETD:etd-09222005-110351
Authors: {'items': [{'email': 'joe_sill@yahoo.com', 'id': 'Sill-J', 'name': {'family': 'Sill', 'given': 'Joseph'}, 'show_email': 'YES'}]}
Year: 1998
DOI: 10.7907/GQWN-1H71
This thesis studies two properties- monotonicity and connectedness- in the context of machine learning. The first part of the thesis examines the role of monotonicity constraints in machine learning from both practical and theoretical perspectives. Two techniques for enforcing monotonicity in machine learning models are proposed. The first method adds to the objective function a penalty term measuring the degree to which the model violates monotonicity. The penalty term can be interpreted as a Bayesian prior favoring functions which obey monotonicity. This method has the potential to enforce monotonicity only approximately, making it appropriate for situations where strict monotonicity may not hold. The second approach consists of a model which is monotonic by virtue of functional form. This model is shown to have universal approximation capabilities with respect to the class M of monotonic functions. A variety of theoretical results are also presented regarding M. The generalization behavior of this class is shown to depend heavily on the probability distribution over the input space. Although the VC dimension of M is [infinity], the VC entropy (i.e., the expected number of dichotomies) is modest for many distributions, allowing us to obtain bounds on the generalization error. Monte Carlo techniques for estimating the capacity and VC entropy of M are presented.
The second part of the thesis considers broader issues in learning theory. Generalization error bounds based on the VC dimension describe a function class by counting the number of dichotomies it induces. In this thesis, a more detailed characterization is presented which takes into account the diversity of a set of dichotomies in addition to its cardinality. Many function classes in common usage are shown to possess a property called connectedness. Models with this property induce dichotomy sets which are highly clustered and have little diversity. We derive an improvement to the VC bound which applies to function classes with the connectedness property.
https://thesis.library.caltech.edu/id/eprint/3689Neural logic : theory and implementation
https://resolver.caltech.edu/CaltechETD:etd-05132005-143440
Authors: {'items': [{'email': 'vincent@paradise.caltech.edu', 'id': 'Bohossian-Vasken-Z', 'name': {'family': 'Bohossian', 'given': 'Vasken Z.'}, 'show_email': 'YES'}]}
Year: 1998
DOI: 10.7907/N0T5-7J92
NOTE: Text or symbols not renderable in plain ASCII are indicated by [...]. Abstract is included in .pdf document.
Human brains are by far superior to computers in solving hard problems like combinatorial optimization and image and speech recognition, although their basic building blocks are several orders of magnitude slower. This observation has boosted interest in the field of artificial networks [20], [37]. The latter are built by interconnecting artificial neurons whose behavior is inspired by that of biological neurons. In this thesis we consider the Boolean version of an artificial neuron, namely, a Linear Threshold (LT) element, which computes a neural-like Boolean function of n binary inputs [32]. An LT element outputs the sign of a weighted sum of its Boolean inputs. The main issues in the study of networks (circuits) consisting of LT elements, called LT circuits, include the estimation of their computational capabilities and limitations and the comparison of their properties with those of traditional Boolean logic circuits based on AND, OR and NOT gates (called AON circuits). For example, there is a strong evidence that LT circuits are more efficient than AON circuits in implementing a number of important functions including the addition, product and division of integers [44], [45].
It is easy to see that an LT element is more powerful than an AON gate, simply because of the freedom one has in selecting the weights. Indeed, different choices of weights produce different Boolean functions. As a matter of fact, the number of n-input Boolean functions that can be implemented by a single LT element is of the order of […], [42], [22]. That additional power comes at the cost of added complexity. Some LT functions require weights that are very different in magnitude, potentially rendering difficult hardware or software implementations of the corresponding LT elements. For that reason, theoretical research in the field of LT circuits has focused on the weights, in particular the power of LT elements with restricted weights. As early as 1971, Muroga, [32], proved that any linear threshold element can be implemented with integer weights. That is, by restricting the magnitudes of the weights to natural numbers, one does not lose any power of the original LT element. We generalize this result to arbitrary subsets of the set of real numbers. For example, we show that one can restrict the weights to be the square of integers, and still be able to realize all LT functions. We ask the following question. What are the conditions on the subset […] which guarantee that all LT functions can be implemented with weights drawn from it?
Another aspect of the complexity of the weights is their growth as the number of inputs increases. It has been shown [17], [33], [38], [43] that there exist linear threshold functions that can be implemented by a single threshold element with exponentially growing weights, but cannot be implemented by a threshold element with smaller polynomialy growing weights. In light of that result the above question was dealt with by defining a class, called […], within the set of linear threshold functions: the class of functions with "small" (i.e. polynomialy growing) weights [43]. We focus on a single LT element. Our contribution consists in two novel methods for constructing threshold functions with minimal weights, which allow us to fill up the gap between polynomial and exponential weight growth by further refining the separation. Namely, we prove that the class of linear threshold functions with polynomial-size weights can be divided into subclasses […], according to the degree, d, of the polynomial. In fact, we prove a more general result-—that there exists a linear threshold function for any arbitrary number of inputs and any weight size.
Even though some LT functions require weights that grow exponentially with the number of input variables, it has been shown recently, in [13], [18], that such functions can be replaced by a two-layer circuit composed of LT gates with polynomially growing, i.e., small weights. We improve the best known bound on the size of that circuit, presented in [18] by focusing on a particular function with large coefficients. We also derive explicit two-layer circuits. Two layer LT circuits are in general composed of different linear threshold elements, but for some useful Boolean functions, such as parity, addition and product, the gates of the first layer are almost identical. To take advantage of this fact we introduce a new Boolean computing element. Instead of the sign function, it computes an arbitrary (with polynomialy many transitions) Boolean function of the weighted sum of its inputs. We call the new computing element an LTM element, which stands for Linear Threshold with Multiple transitions. The advantages of LTM become apparent in the context of VLSI implementation. Indeed, this new model reduces the layout area of the corresponding symmetric function from […] to O(n). We present a VLSI implementations of both LT and LTM elements. Two kinds of elements were fabricated, programmable and hardwired. The programmable elements use the charge on a floating gate in order to store the values of the weights.
For many years, the topic of linear threshold logic, has been approached in two different ways, theory, i.e. computational circuit complexity, [38], [56], and hardware implementation, [48], [40]. Surprisingly, there has been very little interaction between those two approaches. As a whole, the present thesis is one step towards establishing a connection between the theory and implementation of threshold circuits. Its contributions are at three levels. At the theoretical level, new classes of functions such as […] and LTM are defined and their computational power is estimated. At the algorithmic level, we show how to convert real weights to weights drawn from arbitrary subset of the real numbers, e.g., integer weights, we also show how to construct LT functions with minimal weights, and finally we present an algorithm that produces an […] circuit (circuit composed of gates with small weights), that computes the comparison function, COMP. We also present LTM circuits computing useful functions, such as XOR, ADD, PRODUCT. At the implementation level, we show the design, layout and testing of the VLSI implementation of LT and LTM. Establishing a connection between the theoretical and practical aspects of threshold logic will profit both domains by providing solutions for practical problems and by defining new theoretical questions inspired by implementation issues.
https://thesis.library.caltech.edu/id/eprint/1775Statistical optimization of multirate systems and orthonormal filter banks
https://resolver.caltech.edu/CaltechETD:etd-02042008-081232
Authors: {'items': [{'id': 'Tuqan-J', 'name': {'family': 'Tuqan', 'given': 'Jamal'}, 'show_email': 'NO'}]}
Year: 1998
DOI: 10.7907/vx5t-w383
The design of multirate systems and/or filter banks adapted to the input signal statistics is a generic problem that arises naturally in variety of communications and signal processing applications. The two main applications we have in mind are the statistical optimization of subband coders for signal compression and the multirate modeling of WSS random processes. These two applications lead naturally to the important concepts of energy compaction filters and principal component filter banks. In this thesis, we study three problems that are directly related to the above mentioned applications. The first problem is motivated by the observation that in the presence of subband quantizers, it is a loss of generality to assume that the synthesis section in a filter bank is the inverse of the analysis section. We therefore consider the statistical optimization of linear time invariant (LTI) pre- and postfilters surrounding a quantization system. Unlike in previous work, the postfilter is not restricted to be the inverse of the prefilter. Closed form expressions for the optimum filters as well as the resulting minimum mean square error (m.m.s.e.) are derived. The importance of the m.m.s.e. expression is that it clearly quantifies the additional gain obtained by relaxing the perfect reconstruction assumption. In the second problem, we study the quantization of a certain class of non bandlimited signals, modeled as the output of L < M interpolation filters where M is the interpolation factor. Using the fact that these signals are oversampled, we show how to decrease substantially the quantization noise variance using appropriate multirate reconstruction schemes. We also optimize a variety of noise shapers, indicating the corresponding additional reduction in the average mean square error for each case. The results of this chapter extend, using multirate signal processing theory, some well known techniques of efficient A/D converters (e.g. sigma-delta modulators) that usually apply only to bandlimited signals. In the last problem, a novel procedure to design globally optimal FIR energy compaction filters is presented. Energy compaction filters are important due to their close connection to orthonormal filter banks adapted to the input signal statistics. In fact, for the two channel case, the problems are equivalent. A special case of compaction filters arise also in applications such as echo cancelation, time varying systems identification, standard subband filter design and optimal transmitter and receiver design in digital communications. The new proposed approach guarantees theoretical optimality which previous methods could not achieve. Furthermore, the new algorithm is:
i) extremely general in the sense that it can be tailored to cover any of the above applications.
ii) numerically robust.
iii) can be solved efficiently using interior point methods.
The design of a special class of two channel IIR compaction filters is also considered. We show that, in general, this class of optimum IIR compaction filters, parameterized by a single coefficient, are competitive with very high order optimum FIR filters.
https://thesis.library.caltech.edu/id/eprint/492Algorithmic Self-Assembly of DNA
https://resolver.caltech.edu/CaltechETD:etd-05192003-110022
Authors: {'items': [{'email': 'winfree@caltech.edu', 'id': 'Winfree-Erik', 'name': {'family': 'Winfree', 'given': 'Erik'}, 'orcid': '0000-0002-5899-7523', 'show_email': 'YES'}]}
Year: 1998
DOI: 10.7907/HBBV-PF79
<p>How can molecules compute? In his early studies of reversible computation, Bennett imagined an enzymatic Turing Machine which modified a hetero-polymer (such as DNA) to perform computation with asymptotically low energy expenditures. Adleman's recent experimental demonstration of a DNA computation, using an entirely different approach, has led to a wealth of ideas for how to build DNA-based computers in the laboratory, whose energy efficiency, information density, and parallelism may have potential to surpass conventional electronic computers for some purposes. In this thesis, I examine one mechanism used in all designs for DNA-based computer -- the self-assembly of DNA by hybridization and formation of the double helix -- and show that this mechanism alone in theory can perform universal computation. To do so, I borrow an important result in the mathematical theory of tiling: Wang showed how jigsaw-shaped tiles can be designed to simulate the operation of any Turing Machine. I propose constructing molecular Wang tiles using the branched DNA constructions of Seeman, thereby producing self-assembled and algorithmically patterned two-dimensional lattices of DNA. Simulations of plausible self-assembly kinetics suggest that low error rates can be obtained near the melting temperature of the lattice; under these conditions, self-assembly is performing reversible computation with asymptotically low energy expenditures. Thus encouraged, I have begun an experimental investigation of algorithmic self-assembly. A competition experiment suggests that an individual logical step can proceed correctly by self-assembly, while a companion experiment demonstrates that unpatterned two dimensional lattices of DNA will self-assemble and can be visualized. We have reason to hope, therefore, that this experimental system will prove fruitful for investigating issues in the physics of computation by self-assembly. It may also lead to interesting new materials.</p>
https://thesis.library.caltech.edu/id/eprint/1866A method for the specification, composition, and testing of distributed object systems
https://resolver.caltech.edu/CaltechETD:etd-01252008-095244
Authors: {'items': [{'id': 'Sivilotti-P-A-G', 'name': {'family': 'Sivilotti', 'given': 'Paolo A. G.'}, 'show_email': 'NO'}]}
Year: 1998
DOI: 10.7907/z89g-gm27
The formation of a distributed system from a collection of individual components requires the ability for components to exchange syntactically well-formed messages. Several technologies exist that provide this fundamental functionality, as well as the ability to locate components dynamically based on syntactic requirements. The formation of a correct distributed system requires, in addition, that these interactions between components be semantically well-formed. The method presented in this thesis is intended to assist in the development of correct distributed systems.
We present a specification methodology based on three fundamental operators from temporal logic: initially, next, and transient. From these operators we derive a collection of higher-level operators that are used for component specification. The novel aspect of our specification methodology is that we require that these operators be used in the following restricted manner:
•A specification statement can refer only to properties that are local to a single component.
•A single component must be able to guarantee unilaterally the validity of the specification statement for any distributed system of which it is a part. Specification statements that conform to these two restrictions we call certificates.
The first restriction is motivated by our desire for these component specifications to be testable in a relatively efficient manner. In fact, we describe a set of simplified certificates that can be translated into a testing harness by a simple parser with very little programmer intervention. The second restriction is motivated by our desire for a simple theory of composition: If a certificate is a property of a component, that certificate is also a property of any system containing that component.
Another novel aspect of our methodology is the introduction of a new temporal operator that combines both safety and progress properties. The concept underlying this operator has been used implicitly before; but by extracting this concept into a first-class operator, we are able to prove several new theorems about such properties. We demonstrate the utility of this operator and of our theorems by using them to simplify several proofs.
The restrictions imposed on certificates are severe. Although they have pleasing consequences as described above, they can also lead to lengthy proofs of system properties that are not simple conjunctions. To compensate for this difficulty, we introduce collections of certificates that we call services. Services facilitate proof reuse by encapsulating common component interactions used to establish various system properties.
We experiment with our methodology by applying it to several extended examples. These experiments illustrate the utility of our approach and convince us of the practicality of component-based distributed system development. This thesis addresses three parts of the development cycle for distributed object systems: (i) the specification of systems and components, (ii) the compositional reasoning used to verify that a collection of components satisfy a system specification, and (iii) the validation of component implementations.
https://thesis.library.caltech.edu/id/eprint/341Contextual pattern recognition with applications to biomedical image identification
https://resolver.caltech.edu/CaltechETD:etd-09222005-111015
Authors: {'items': [{'email': 'xubosong@csee.ogi.edu', 'id': 'Song-Xubo', 'name': {'family': 'Song', 'given': 'Xubo'}, 'show_email': 'YES'}]}
Year: 1999
DOI: 10.7907/F5YK-HM52
This thesis studies two rather distinct topics: one is the incorporation of contextual information in pattern recognition, with applications to biomedical image identification; and the other is the theoretical modeling of learning and generalization in the regime of machine learning.
In Part I of the thesis, we propose techniques to incorporate contextual information into object classification. In the real world there are cases where the identity of an object is ambiguous due to the noise in the measurements based on which the classification should be made. It is helpful to reduce the ambiguity by utilizing extra information referred to as context, which in our case is the identities of the accompanying objects. We investigate the incorporation of both full and partial context. Their error probabilities, in terms of both set-by-set error and element-by-element error, are established and compared to context-free approach. The computational cost is studied in detail for full context, partial context and context-free cases. The techniques are applied to toy problems as well as real world problems such as white blood cell image classification and microscopic urinalysis. It is demonstrated that superior classification performance is achieved by using context. In our particular application, it reduces overall classification error, as well as false positive and false negative diagnosis rates.
In Part II of the thesis, we propose a novel theoretical framework, called the Bin Model, for learning and generalization. Using the Bin Model, a closed form is derived for generalization that estimates the out-of-sample performance in terms of the in-sample performance. We address the problems of overfitting, and characterize conditions under which it does not appear. The effect of noise on generalization is studied, and the generalization of the Bin Model framework from classification problems to regression problems is discussed.
https://thesis.library.caltech.edu/id/eprint/3690The impact of asynchrony on computer architecture
https://resolver.caltech.edu/CaltechETD:etd-08112005-114144
Authors: {'items': [{'id': 'Manohar-R', 'name': {'family': 'Manohar', 'given': 'Rajit'}, 'show_email': 'NO'}]}
Year: 1999
DOI: 10.7907/xzwa-p598
The performance characteristics of asynchronous circuits are quite different from those of their synchronous counterparts. As a result, the best asynchronous design of a particular system does not necessarily correspond to the best synchronous design, even at the algorithmic level. The goal of this thesis is to examine certain aspects of computer architecture and design in the context of an asynchronous VLSI implementation.
We present necessary and sufficient conditions under which the degree of pipelining of a component can be modified without affecting the correctness of an asynchronous computation.
As an instance of the improvements possible using an asynchronous architecture, we present circuits to solve the prefix problem with average-case behavior better than that possible by any synchronous solution in the case when the prefix operator has a right zero. We show that our circuit implementations are area-optimal given their performance characteristics, and have the best possible average-case latency.
At the level of processor design, we present a mechanism for the implementation of precise exceptions in asynchronous processors. The novel feature of this mechanism is that it permits the presence of a data-dependent number of instructions in the execution pipeline of the processor.
Finally, at the level of processor architecture, we present the architecture of a processor with an independent instruction stream for branches. The instruction set permits loops and function calls to be executed with minimal control-flow overhead.https://thesis.library.caltech.edu/id/eprint/3095Evolution of genetic codes
https://resolver.caltech.edu/CaltechETD:etd-09042007-091804
Authors: {'items': [{'id': 'Ofria-C-A', 'name': {'family': 'Ofria', 'given': 'Charles A.'}, 'orcid': '0000-0003-2924-1732', 'show_email': 'NO'}]}
Year: 1999
DOI: 10.7907/2z40-1m97
In this thesis, I use analytical and computational techniques to study the development of codes in evolutionary systems. We only know of one instance of such a genetic code in the natural world: our own DNA. However, the results from my work are expected to be universally true for all evolving systems. I use mathematical models and conduct experiments with avida, a software-based research platform for the study of evolution in "digital organisms." This allows me to collect statistically powerful data over evolutionary timescales infeasible in a biological system.
In the avida system, Darwinian evolution is implemented on populations of self-replicating computer programs. A typical experiment is seeded with a single ancestor program capable only of reproduction. This ancestor gives rise to an entire population of programs, which adapt to interact with a complex environment, while developing entirely new computational capabilities. I study the process of evolution in this system, taking exact measurements on the underlying genetic codes, and performing tests that would be prohibitively difficult in biological systems.
I have focused on the following areas in studying the evolution of genetic codes:
Information Theory: I treat the process of reproduction as a noisy channel in which codes are transmitted from the parent's genome to the child. Unlike most channels, however, evolution actively selects for codes received with a higher information content, even if this increased information was introduced via noise. A genetic code consists of information about the environment surrounding the organism. As a population adapts, this information increases, and can be approximated through measuring the reduction of per-nucleotide entropy - in effect sites freeze in place as they code for useful functionality. In the avida system, we know the sequence of all genomes in the population, and new computational genes can be identified as they are formed.
The Evolution of Genetic Organization: Organisms incapable of error correction (such as viruses) develop strong code compaction techniques to minimize their target area for mutations, the most prominent of which is overlapping genes. Higher organisms, however, are capable of reducing their mutational load and will explicitly spread out their code, cleanly segregating their genes. I investigate the pressures behind overlapping or segregation of genes, and demonstrate that overlaps have a side effect of drastically reducing the probability of neutral mutations within a gene, and hence hindering continued adaptation. Further, in a changing environment, overlapping genes have a significantly reduced ability to adapt independently. I compare overlapping and singly expressed sections of code in avida, and show a significant (two-fold) difference in the average per-site variation. I also demonstrate the evolutionary pressure for organisms to segregate their genes in a fluctuating environment to improve their adaptive abilities.
Evolving Computer Programs: I explore evolution in digital genetic codes, and isolate some of those features of a programming language that promote continuous adaptation. In the biological world evolution gives rise to complex organisms robust to changing situations in their environment. This increase in complexity and "functionality" of the organisms typically generates more stable systems. On the other hand, as computer programs gain complexity, they only become more fragile. If two programs interact in a way not explicitly designed, the results are neither predictable nor reliable. In fact, computer programs often fail even when put to the use for which they were explicitly intended. Computational organisms, however, have a level of robustness more akin to their biological counterparts, not only performing computations, but often doing so in a manner beyond the efficiency that a human programmer could produce.
Finally, all of this work is tied together, and future directions for its continuation are explored.https://thesis.library.caltech.edu/id/eprint/3323Visual input for pen-based computers
https://resolver.caltech.edu/CaltechETD:etd-03152006-094551
Authors: {'items': [{'email': 'mariomu@vision.caltech.edu', 'id': 'Munich-M-E', 'name': {'family': 'Munich', 'given': 'Mario Enrique'}, 'orcid': '0000-0002-6665-7473', 'show_email': 'YES'}]}
Year: 2000
DOI: 10.7907/1VW0-ZG46
The development of computer technology has had a parallel evolution of the interface between humans and machines, giving rise to interface systems inspired by human communication. Vision has been demonstrated to be the sense of choice for face recognition, gesture recognition, lip reading, etc. This thesis presents the design and implementation of a camera-based, human-computer interface for acquisition of handwriting. The camera focuses on the sheet of paper and images the hand writing; computer analysis of the resulting sequence of images enables the trajectory of the pen to be tracked and the times when the pen is in contact with the paper to be detected. The recovered trajectory is shown to have sufficient spatio-temporal resolution and accuracy to enable handwritten character recognition.
Signatures can be acquired with the camera-based interface with enough resolution to perform verification. This thesis describes the performance of a visual-acquisition signature verification system, emphasizing the importance of the parameterization of the signature to achieving good classification results. The generalization error of the verification algorithm is estimated using a technique that overcomes the small number of example signatures and forgeries provided by the subjects.
The problem of establishing correspondence and measuring the similarity of a pair of planar curves, in our case a pair of signatures, arises in many application in computer vision and pattern recognition. This thesis presents a new method for comparing planar curves and for performing matching at sub-sampling resolution. The analysis of the algorithm as well as its structural properties are described. The performance of the new technique is evaluated for the problem of signature verification and compared to the performance of the well-known Dynamic Programming Matching algorithm.https://thesis.library.caltech.edu/id/eprint/962Object Recognition in Compressed Imagery.
https://resolver.caltech.edu/CaltechTHESIS:10052010-135012445
Authors: {'items': [{'id': 'Daniell-Cynthia-Evors', 'name': {'family': 'Daniell', 'given': 'Cynthia Evors'}, 'show_email': 'NO'}]}
Year: 2000
DOI: 10.7907/6z3z-ar86
<p>It is often necessary to search for objects in large databases of compressed imagery. In the past, object recognition and image compression have generally been treated as separate problems, resulting in inefficient suboptimal performance. Moreover, computational and storage issues make it fundamentally prohibitive to uncompress large images prior to object recognition. We provide two complementary solutions to the problem of object recognition in compressed imagery, each of which integrates subband and correlation filtering in a unique manner.</p>
<p>One key benefit of correlation filters is that, as linear systems, they are highly compatible with the subband filtering process. This enables us to provide a seamless operation in which object recognition and data compression are viewed as continuations of the same process. The public MSTAR data set illustrates our results on a three class problem of 79 Synthetic Aperture Radar images at one foot resolution.</p>
<p>Our general framework, the Pattern Recognition Subband Coder (PRSC), provides simultaneous synthesis and recognition at full resolution in a computationally efficient architecture. Its parallelism enables a result 1.6 times faster, in the limit, than correlation on uncompressed imagery. Furthermore, by jointly optimizing the synthesis and recognition filters, the PRSC achieves 100% recognition accuracy on our compressed data set, improving performance over that produced from the original (uncompressed) data set, by 3.7%. We maintain this success for compression ratios up to 6:1.</p>
<p>Addressing the issue of reduced resolution recognition, our Subband Domain Correlation Filters operate directly on the subband coefficients at multiple resolution levels. For compression ratios of at least 20:1, we achieve recognition performance of at least 90%, 85%, and 75%, respectively, on two, four, and eight foot resolution data.</p>
<p>Thus, through our solutions with compressed imagery, we outperform correlation results on the equivalent original imagery in terms of both speed and accuracy, as well as provide success at reduced resolutions of the data.</p>https://thesis.library.caltech.edu/id/eprint/6098Attentional Control of Complex Systems
https://resolver.caltech.edu/CaltechTHESIS:01032013-152450777
Authors: {'items': [{'email': 'gregbillock@gmail.com', 'id': 'Billock-Joseph-Gregory', 'name': {'family': 'Billock', 'given': 'Joseph Gregory'}, 'show_email': 'NO'}]}
Year: 2001
DOI: 10.7907/132r-jm11
<p>This thesis reports on work done in applying some of the concepts and architectures found in biological computation to computer algorithms. Biology has long inspired computer technology at the level of processing elements. This thesis explores the application of biologically inspired algorithms at a higher level-that of functional structures of the nervous system. The first chapter gives background on the attentional/awareness model of the brain, why it is important to biology and the advantages in real-time performance and in learning facilitation which we expect from applying it in computer algorithms.</p>
<p>The second chapter examines the application of this model to a canonical computer science problem-the bin packing problem. Approaching this NP-complete problem when limited by computational resources and time constraints means that algorithms which throwaway large amounts of the information about the problem perform better than those which attempt to consider everything. The existence of an optimum in the size of a working memory needed to find the best solution under time pressure is shown. The transition between the regime of strict time constraints and more forgiving time constraints is quite sudden. Chapter 3 presents an analytical model for better understanding the performance of various bin packing algorithms.</p>
<p> Chapter 4 examines the application of the attentional model to a real-time computer game testbed. This testbed is explained, and results are shown which illustrate that in a complex, unpredictable environment with tight time and resource constraints conditions, an algorithm which examines only that information which falls into a relatively small part of the playing area can win against player which addresses it all.</p>
<p>Chapter 5 turns to an examination of the role of reduced informational representations upon learning. Solving of various logical-kinetic puzzles by a simulated segmented arm is done by a learning system. A logic supervisory subsystem utilizes attentional/awareness methods to train, and pass control of the different control levels of the articulate arm over to, the neural networks, adaptive resonance theory networks, and declarative computer memory which it trains. Finally, chapter 6 presents an overview and evaluation of the work.</p>https://thesis.library.caltech.edu/id/eprint/7366Kinematic Measurement and Feature Sets for Automatic Speech Recognition
https://resolver.caltech.edu/CaltechTHESIS:11192010-074900476
Authors: {'items': [{'id': 'Fain-D-C', 'name': {'family': 'Fain', 'given': 'Daniel Cark'}, 'show_email': 'NO'}]}
Year: 2001
DOI: 10.7907/9vse-8c78
This thesis examines the use of measured and inferred kinematic information in automatic speech recognition and lipreading, and investigates the relative information content and recognition performance of vowels and consonants. The kinematic information describes the motions of the organs of speech-the articulators. The contributions of this thesis include a new device and set of algorithms
for lipreading (their design, construction, implementation, and testing); incorporation of direct articulator-position measurements into a speech recognizer; and reevaluation of some assumptions regarding vowels and consonants.
The motivation for including articulatory information is to improve modeling of coarticulation
and reconcile multiple input modalities for lipreading. Coarticulation, a ubiquitous phenomenon, is the process by which speech sounds are modified by preceding and following sounds.
To be useful in practice, a recognizer will have to infer articulatory information from sound, video, or both. Previous work made progress towards recovery of articulation from sound. The present project assumes that such recovery is possible; it examines the advantage of joint acousticarticulatory representations over acoustic-only. Also reported is an approach to recovery from video in which camera placement (side view, head-mounted) and lighting are chosen to robustly obtain
lip-motion information.
Joint acoustic-articulatory recognition experiments were performed using the University of Wisconsin X-ray Microbeam Speech Production Database. Speaker-dependent monophone recognizers, based on hidden Markov models, were tested on paragraphs each lasting about 20 seconds. Results
were evaluated at the phone level and tabulated by several classes (vowel, stop, and fricative). Measured articulator coordinates were transformed by principal components analysis, and velocity and acceleration were appended. Concatenating the transformed articulatory information to a standard acoustic (cepstral) representation reduced the error rate by 7.4 %), demonstrating across-speaker
statistical significance (p = 0.018). Articulation improved recognition of male speakers more than female, and recognition of vowels more than fricatives or stops.
The analysis of vowels, stops, and fricatives included both the articulatory recognizer of chapter 3 and other recognizers for comparison. The information content of the different classes was also estimated. Previous assumptions about recognition performance are false, and findings of information content require consonants to be defined to include vowel-like sounds.https://thesis.library.caltech.edu/id/eprint/6182Rate Loss of Network Source Codes
https://resolver.caltech.edu/CaltechETD:etd-05232002-173821
Authors: {'items': [{'id': 'Feng-Hanying', 'name': {'family': 'Feng', 'given': 'Hanying'}, 'show_email': 'NO'}]}
Year: 2002
DOI: 10.7907/GVDP-7248
In this thesis, I present bounds on the performance of a variety of network source codes. These <em>rate loss</em> bounds compare the rates achievable by each network code to the rate-distortion bound <em>R(D)</em> at the corresponding distortions. The result is a collection of optimal performance bounds that are easy to calculate.
I first present new bounds for the rate loss of multi-resolution source codes (MRSCs). Considering an <em>M</em>-resolution code with <em>M</em>>=2, the rate loss at the <em>i</em>th resolution with distortion <em>D_i</em> is defined as <em>L_i=R_i-R(D_i)</em>, where <em>R_i</em> is the rate achievable by the MRSC at stage <em>i</em>. For 2-resolution codes, there are three scenarios of particular interest: (i) when both resolutions are equally important; (ii) when the rate loss at the first resolution is 0 (<em>L_1=0</em>); (iii) when the rate loss at the second resolution is 0 (<em>L_2=0</em>). The work of Lastras and Berger gives constant upper bounds for the rate loss in scenarios (i) and (ii) and an asymptotic bound for scenario (iii). In this thesis, I show a constant bound for scenario (iii), tighten the bounds for scenario (i) and (ii), and generalize the bound for scenario (ii) to <em>M</em>-resolution greedy codes.
I also present upper bounds for the rate losses of additive MRSCs (AMRSCs), a special MRSC. I obtain two bounds on the rate loss of AMRSCs: one primarily good for low rate coding and another which depends on the source entropy.
I then generalize the rate loss definition and present upper bounds for the rate losses of multiple description source codes. I divide the distortion region into three sub-regions and bound the rate losses by small constants in two sub-regions and by the joint rate losses of a normal source with the same variance in the other sub-region.
Finally, I present bounds for the rate loss of multiple access source codes (MASCs). I show that lossy MASCs can be almost as good as codes based on joint source encoding.https://thesis.library.caltech.edu/id/eprint/1965Generalization Error Estimates and Training Data Valuation
https://resolver.caltech.edu/CaltechETD:etd-09062005-083717
Authors: {'items': [{'email': 'zander@fantastivision.com', 'id': 'Nicholson-Alexander-Marshall', 'name': {'family': 'Nicholson', 'given': 'Alexander Marshall'}, 'show_email': 'YES'}]}
Year: 2002
DOI: 10.7907/1H16-VX81
This thesis addresses several problems related to generalization in machine learning systems. We introduce a theoretical framework for studying learning and generalization. Within this framework, a closed form is derived for the expected generalization error that estimates the out-of-sample performance in terms of the in-sample performance. We consider the problem of overfitting and show that, using a simple exhaustive learning algorithm, overfitting does not occur. These results do not assume a particular form of the target function, input distribution or learning model, and hold even with noisy data sets. We apply our analysis to practical learning systems, illustrate how it may be used to estimate out-of-sample errors in practice, and demonstrate that the resulting estimates improve upon errors estimated with a validation set for real world problems.
Based on this study of generalization, we develop a technique for quantitative valuation of training data. We demonstrate that this valuation may be used to select training sets that improve generalization performance. With a reasonable prior over target functions, it further allows us to estimate the level of noise in a data set and provides for detection and correction of noise in individual examples. Finally, this data valuation can be used to classify new examples, yielding a new learning algorithm that is shown to be relatively robust to noise.
https://thesis.library.caltech.edu/id/eprint/3347Eye Position Modulation of Visual Cortex and the Sensory Set Hypothesis
https://resolver.caltech.edu/CaltechTHESIS:01312012-161153127
Authors: {'items': [{'id': 'Rosenbluth-David', 'name': {'family': 'Rosenbluth', 'given': 'David'}, 'show_email': 'NO'}]}
Year: 2002
DOI: 10.7907/jnhb-e433
<p>What we see depends on where we look. This is obvious as a statement about the nonuniformity of our external visual environment. But it is also true, in a much less obvious sense, as a statement about the internal neurophysiology of the visual system. What we see depends on where we look in the neurophysiological sense that eye position signals have a dramatic effect on the responsiveness of visual cortical neurons. This thesis empirically studies the way in which point of regard (what point in space the eyes are fixating) influences neurons in visual cortical areas V1 and V4 and then presents a theoretical exploration of how these
two different ways in which "What we see depends on where we look" might be functionally intertwined.</p>
<p>The empirical data presented here adds to the growing body of evidence that eye position signals are ubiquitous in visual cortex, an observation which reopens speculation about the functional role that these signals might play in different visual cortical areas. The presence of eye position signals in visual areas of the ventral visual processing stream raises the possibility that these signals might facilitate object identity. Eye position signals might be exploited by visual cortex as a conditioned stimulus, which can become functionally linked to the responses of visual cortical neurons (unconditional response) through repeated pairing with the unconditioned stimulus, the retinal stimulus, in a classical conditioning paradigm. In this way the visual system would be capable of learning systematic relationships between point of regard and statistical characteristics of the visual environment. The learned response to the conditioned stimulus could then be exploited as a preparatory signal, to speed or otherwise alter visual processing to suit the current context. In exploring this theoretical viewpoint, we discuss the circumstances under which context dependent coding provides
advantages and how a code switching strategy might be implemented through physiological parcellation mediated by gain control. Eye position signals are here considered to be one among many different types of extra-retinal signals, present in visual cortical areas, whose presence might be similarly exploited. As such, the data and theory presented here should be considered as contributing to the broader literature on the influence of signals from outside the classical receptive field.</p>
https://thesis.library.caltech.edu/id/eprint/6799Network Source Coding: Theory and Code Design for Broadcast and Multiple Access Networks
https://resolver.caltech.edu/CaltechETD:etd-05302003-125004
Authors: {'items': [{'id': 'Zhao-Qian', 'name': {'family': 'Zhao', 'given': 'Qian'}, 'show_email': 'NO'}]}
Year: 2003
DOI: 10.7907/61XN-MV62
<p>In the information age, network systems and applications have been growing rapidly to provide us with more versatile and high bit rate services. However, the limited bandwidth restricts the amount of information that can be sent through the networks. Thus efficient data representation or source coding is imperative for future network development. Distinct from the traditional source coding strategy, network source codes take advantage of the network topology and are able to maximally compress data before transmission.</p>
<p>In this thesis, I present a variety of source coding techniques for use in network environments and demonstrate the benefits of network source codes over traditional source codes from both theoretical and practical perspectives.</p>
<p>First, I address source coding for broadcast systems. The results I obtain include derivation of the theoretical limits of broadcast system source codes, algorithm design for optimal broadcast system vector quantizers, implementation of the optimal code, and experimental results.</p>
<p>Then, I focus on multiple access systems which are the dual systems of broadcast systems. I present the properties of multiple access source codes and generalize traditional entropy code design algorithms to attain the corresponding optimal multiple access source codes for arbitrary joint source statistics. I further introduce a family of polynomial complexity code design algorithms that approximates the optimal solutions. Application to universal coding for multiple access networks when the joint source statistics are unknown a priori is briefly discussed. Finally, I demonstrate algorithmic performance by showing experimental results on a variety of data sets.</p>
<p>inally, in seeking a simple lossy source coding method for general networks, I apply entropy constrained dithered quantization in network source code design and present the coding results for multi-resolution source codes and multiple access source codes. Multi-resolution and multiple access dithered quantizers are low complexity codes that achieve performance very close to the theoretical rate-distortion bound.</p>https://thesis.library.caltech.edu/id/eprint/2289A Probabilistic Approach to Human Motion Detection and Labeling
https://resolver.caltech.edu/CaltechETD:etd-12102002-113833
Authors: {'items': [{'id': 'Song-Yang', 'name': {'family': 'Song', 'given': 'Yang'}, 'show_email': 'NO'}]}
Year: 2003
DOI: 10.7907/945J-QX86
<p>Human motion analysis is a very important task for computer vision with many potential applications. There are several problems in human motion analysis: detection, tracking, and activity interpretation. Detection is the most fundamental problem of the three, but remains untackled due to its inherent difficulty. This thesis develops a solution to the problem. It is based on a learned probabilistic model of the joint positions and velocities of the body parts, where detection and labeling are performed by hypothesis testing on the maximum a posterior estimate of the pose and motion of the body. To achieve efficiency in learning and testing, a graphical model is used to approximate the conditional independence of human motion. This model is also shown to provide a natural way to deal with clutter and occlusion.</p>
<p>One key factor in the proposed method is the probabilistic model of human motion. In this thesis, an unsupervised learning algorithm that can obtain the probabilistic model automatically from unlabeled training data is presented. The training data include useful foreground features as well as features that arise from irrelevant background clutter. The correspondence between parts and detected features is also unknown in the training data. To learn the best model structure as well as model parameters, a variant of the EM algorithm is developed where the labeling of the data (part assignments) is treated as hidden variables. We explore two classes of graphical models: trees and decomposable triangulated graphs and find that the later are superior for our application. To better model human motion, we also consider the case when the model consists of mixtures of decomposable triangulated graphs.</p>
<p>The efficiency and effectiveness of the algorithm have been demonstrated by applying it to generate models of human motion automatically from unlabeled image sequences, and testing the learned models on a variety of sequences. We find detection rates of over 95% on pairs of frames. This is very promising for building a real-life system, for example, a pedestrian detector.</p>https://thesis.library.caltech.edu/id/eprint/4917Holographic Information Systems
https://resolver.caltech.edu/CaltechETD:etd-04282003-142947
Authors: {'items': [{'id': 'Panotopoulos-Georgios', 'name': {'family': 'Panotopoulos', 'given': 'Georgios'}, 'show_email': 'NO'}]}
Year: 2003
DOI: 10.7907/P6B7-VR22
<p>The goal of this work is to investigate the use of holographic techniques for information processing and transmission systems. Until recently information has been processed and transmitted mainly electronically. With the advent of optical fiber communications the monopoly of electronics has receded in the telecommunications field, but the domain of information processing is still dominated by electronic processors.</p>
<p>This thesis follows a top-down approach to the design of processors that integrate both electronic and optical components. It begins with the design considerations of a compact, rapidly reconfigurable opto-electronic processor, which possesses an optical bus in addition to the traditional electronic bus. The optical bus takes advantage of the massive parallelism that is afforded by optics and can be coupled to a holographic digital memory, allowing rapid reconfiguration of the device. The capability of rapid reconfiguration gives rise to a new computational paradigm, where the reprogramming of the device can become part of the computation. We suggest additional applications of this processor, namely as a smart reading head for large scale holographic disk memories. Finally we present novel algorithms that were developed specifically to take advantage of the additional capabilities of our processor.</p>
<p>The next section is concerned with the wavelength and angular tuning of strong volume holograms, both in the reflection and 90-degree geometries. Since photons have no charge, we need to rely on their wave properties to manipulate them, both for long-range transmission, such as telecommunications, and short-range transmission, such as on chip interconnects. In this section we investigate how volume holograms can be used to selectively redirect information bearing light beams.</p>
<p>The final part of this thesis is concerned with material issues. Holographic recording of strong volume gratings is one of the most commonly used approaches, and photorefractive materials have a strong bearing on the overall performance of the final system. Two properties of iron doped lithium niobate are investigated, namely the dependence of absorption on temperature and the quadratic electro-optic coefficient. The former is crucial for the commonly used technique of thermal fixing, and the latter can become significant should we choose to use applied continuous fields to tune our gratings.</p>https://thesis.library.caltech.edu/id/eprint/1527Wireless Networks, from Collective Behavior to the Physics of Propagation
https://resolver.caltech.edu/CaltechETD:etd-05202003-154451
Authors: {'items': [{'email': 'massimo@ece.ucsd.edu', 'id': 'Franceschetti-Massimo', 'name': {'family': 'Franceschetti', 'given': 'Massimo'}, 'orcid': '0000-0002-4057-8152', 'show_email': 'YES'}]}
Year: 2003
DOI: 10.7907/SCTG-FN57
This thesis addresses some of the key challenges in the emerging wireless scenario. It focuses on the problems of connectivity, coverage, and wave propagation, following a mathematically rigorous approach. The questions addressed are very basic and extremely easy to state. Their solution, however, can be difficult and leads to the development of a new kind of percolation theory, to a new theorem in geometry, and to a new model of wave propagation in urban environments. The problems are connected together to provide guidelines in the design of wireless networks.
https://thesis.library.caltech.edu/id/eprint/1887Modeling Artificial, Mobile Swarm Systems
https://resolver.caltech.edu/CaltechETD:etd-05282003-205506
Authors: {'items': [{'email': 'agassw@yahoo.com', 'id': 'Agassounon-William-B-G', 'name': {'family': 'Agassounon', 'given': 'William B. G.'}, 'show_email': 'NO'}]}
Year: 2003
DOI: 10.7907/EJYZ-3Y55
<p>Swarm intelligence is a new research paradigm that offers novel approaches for studying and solving distributed problems using solutions inspired by social insects and other natural behaviors of vertebrates. In this thesis, we present methodologies for modeling artificial, mobile systems within the swarm intelligence framework. The proposed methodologies provide guidelines in the study and design of artificial swarm systems for the following two classes of experiments: distributed sensing and distributed manipulation.</p>
<p>Event discovery and information dissemination through local communication in artificial swarm systems present similar characteristics as natural phenomena such as foraging and food discovery in insect colonies and the spread of infectious diseases in animal populations, respectively. We show that the artificial systems can be described in similar mathematical terms as those used to describe the natural systems. The proposed models can be classified in two main categories: non-embodied and embodied models. In the first category agents are modeled as mobile bodiless points, whereas the other models take into account the physical interference between agents resulting from embodiment. Furthermore, within each category, we distinguish two subcategories: spatial and nonspatial models. In the spatial models we keep track of the trajectory of each agent, the correlation between the positions occupied by the agents over consecutive time steps, or make use of the spatial distribution resulting from the movement pattern of the agents. In the nonspatial models we assume that agents hop around randomly and occupy independent positions over consecutive time steps.</p>
<p>In our description of distributed manipulation in swarm robotic systems we present two case studies of non-collaborative and collaborative manipulations, respectively. The general approach proposed here consists of first representing the group behavior of the active agents with a Finite State Machine (FSM) then describing mathematically the dynamics of the group. The first case study is the aggregation experiment that consists of collecting and gathering objects scattered around an enclosed arena. We present a macroscopic model that accurately captures the dynamics of the experiment and a suite of threshold-based, scalable, and fully distributed algorithms for allocating the workers to the task optimally. The second case study is that of the stick-pulling experiment in which a group of robots is used to pull sticks from the ground. This task requires the collaborative effort of two robots to be successful. Here, we present a discrete-time macroscopic model that helps us uncover counter-intuitive behaviors that result from collaboration between the agents.</p>
<p>We complete each proposed modeling methodology by showing how the parameters of the models can be calculated using solely the characteristics of the environment and those of the agents and by analyzing the constraints and limitations of the different models. Finally, we use different tools (simulations and real robots) to validate the proposed models.</p>
https://thesis.library.caltech.edu/id/eprint/2183Cyclic Combinational Circuits
https://resolver.caltech.edu/CaltechETD:etd-05032004-153842
Authors: {'items': [{'email': 'mriedel@umn.edu', 'id': 'Riedel-Marcus-D', 'name': {'family': 'Riedel', 'given': 'Marcus D.'}, 'orcid': '0000-0002-3318-346X', 'show_email': 'YES'}]}
Year: 2004
DOI: 10.7907/410B-XR25
<p>A collection of logic gates forms a combinational circuit if the outputs can be described as Boolean functions of the current input values only. Optimizing combinational circuitry, for instance, by reducing the number of gates (the area) or by reducing the length of the signal paths (the delay), is an overriding concern in the design of digital integrated circuits.</p>
<p>The accepted wisdom is that combinational circuits must have acyclic (i.e., loop-free or feed-forward) topologies. In fact, the idea that "combinational" and "acyclic" are synonymous terms is so thoroughly ingrained that many textbooks provide the latter as a definition of the former. And yet simple examples suggest that this is incorrect. In this dissertation, we advocate the design of cyclic combinational circuits (i.e., circuits with loops or feedback paths). We demonstrate that circuits can be optimized effectively for area and for delay by introducing cycles.</p>
<p>On the theoretical front, we discuss lower bounds and we show that certain cyclic circuits are one-half the size of the best possible equivalent acyclic implementations. On the practical front, we describe an efficient approach for analyzing cyclic circuits, and we provide a general framework for synthesizing such circuits. On trials with industry-accepted benchmark circuits, we obtained significant improvements in area and delay in nearly all cases. Based on these results, we suggest that it is time to re-write the definition: combinational might well mean cyclic.</p>https://thesis.library.caltech.edu/id/eprint/1591Visual Recognition: Computational Models and Human Psychophysics
https://resolver.caltech.edu/CaltechETD:etd-06022005-150332
Authors: {'items': [{'email': 'feifeili@stanford.edu', 'id': 'Li-Fei-Fei', 'name': {'family': 'Li', 'given': 'Fei-Fei'}, 'orcid': '0000-0002-7481-0810', 'show_email': 'NO'}]}
Year: 2005
DOI: 10.7907/G5NP-KH67
<p>Object and scene recognition is one of the most essential functionalities of human vision. It is also of fundamental importance for machines to be able to learn and recognize meaningful objects and scenes. In this thesis, we explore the following four aspects of object and scene recognition.</p>
<p>It is well known that humans can be "blind" even to major aspects of natural scenes when we attend elsewhere. The only tasks that do not need attention appear to be carried out in the early stages of the visual system. Contrary to this common belief, we show that subjects can rapidly detect animals or vehicles in briefly presented novel natural scenes while simultaneously performing another attentionally demanding task. By comparison, they are unable to discriminate large T’s from L’s, or bisected two-color disks from their mirror images under the same conditions. We explore this phenonmenon further by removing color from the natural scenes, or increasing the number of images peripherally. We find evidence that suggests that familiarity and meaningfulness might be among the factors that determine attentional requirements for both natural and synthetic stimuli.</p>
<p>So what exactly do we see when we glance at a natural scene? And does what we see change as the glance becomes longer? We asked naive subjects to report what they saw in nearly a hundred briefly presented photographs. After each presentation subjects reported what they had just seen as completely as possible. Afterward, another group of sophisticated individuals who were not aware of the goals of the experiment were instructed to score each of the descriptions produced by the subjects in the first stage. Individual scores were assigned to more than a hundred different attributes. Given the evaluation of the responses, we show that within a single glance, much object and scene level information is perceived by human subjects. But the richness of our perception seems asymmetrical. Subjects tend to have a bias to natural scenes being perceived as outdoor rather than indoor.</p>
<p>In computer vision, it is commonly known that learning visual models of object categories notoriously requires thousands of training examples. We show that it is possible to learn much information about a category from just one image, or a handful of images. The key insight is that, rather than learning from scratch, one can take advantage of knowledge coming from previously learnt categories, no matter how different these categories might be. We explore a Bayesian implementation of this idea. Object categories are represented by probabilistic models. Prior knowledge is represented as a probability density function on the parameters of these models. The posterior model for an object category is obtained by updating the prior in the light of one or more observations. We test a simple implementation of our algorithm on a database of 101 diverse object categories. We compare category models learnt by a simple implementation of our Bayesian approach to models learnt from maximum likelihood (ML) and maximum a posteriori (MAP) methods. We find that in a database of more than 100 categories the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully.</p>
<p>We also propose a novel approach to learn and recognize natural scene categories. Unlike previous work, it does not require experts to annotate the training set. We represent the image of a scene by a collection of local regions, denoted as codewords obtained by unsupervised learning. Each region is represented as part of a "theme." In previous work, such themes were learnt from hand-annotations of experts, while our method learns the theme distributions as well as the codewords distribution over the themes without supervision. We report satisfactory categorization performances on a large set of 13 categories of complex scenes.</p>https://thesis.library.caltech.edu/id/eprint/2390Networks of Relations
https://resolver.caltech.edu/CaltechETD:etd-06032005-140944
Authors: {'items': [{'email': 'cook@ini.uzh.ch', 'id': 'Cook-Matthew-M', 'name': {'family': 'Cook', 'given': 'Matthew M.'}, 'show_email': 'NO'}]}
Year: 2005
DOI: 10.7907/CVKM-D684
<p>Relations are everywhere. In particular, we think and reason in terms of mathematical and English sentences that state relations. However, we teach our students much more about how to manipulate functions than about how to manipulate relations. Consider functions. We know how to combine functions to make new functions, how to evaluate functions efficiently, and how to think about compositions of functions. Especially in the area of boolean functions, we have become experts in the theory and art of designing combinations of functions to yield what we want, and this expertise has led to techniques that enable us to implement mind-bogglingly large yet efficient networks of such functions in hardware to help us with calculations. If we are to make progress in getting machines to be able to reason as well as they can calculate, we need to similarly develop our understanding of relations, especially their composition, so we can develop techniques to help us bridge between the large and small scales. There has been some important work in this area, ranging from practical applications such as relational databases to extremely theoretical work in universal algebra, and sometimes theory and practice manage to meet, such as in the programming language Prolog, or in the probabilistic reasoning methods of artificial intelligence. However, the real adventure is yet to come, as we learn to develop a better understanding of how relations can efficiently and reliably be composed to get from a low level representation to a high level representation, as this understanding will then allow the development of automated techniques to do this on a grand scale, finally enabling us to build machines that can reason as amazingly as our contemporary machines can calculate.</p>
<p>This thesis explores new ground regarding the composition of relations into larger relational structures. First of all a foundation is laid by examining how networks of relations might be used for automated reasoning. We define exclusion networks, which have close connections with the areas of constraint satisfaction problems, belief propagation, and even boolean circuits. The foundation is laid somewhat deeper than usual, taking us inside the relations and inside the variables to see what is the simplest underlying structure that can satisfactorily represent the relationships contained in a relational network. This leads us to define zipper networks, an extremely low-level view in which the names of variables or even their values are no longer necessary, and relations and variables share a common substrate that does not distinguish between the two. A set of simple equivalence operations is found that allows one to transform a zipper network while retaining its solution structure, enabling a relation-variable duality as well as a canonical form on linear segments. Similarly simple operations allow automated deduction to take place, and these operations are simple and uniform enough that they are easy to imagine being implemented by biological neural structures.</p>
<p>The canonical form for linear segments can be represented as a matrix, leading us to matrix networks. We study the question of how we can perform a change of basis in matrix networks, which brings us to a new understanding of Valiant's recent holographic algorithms, a new source of polynomial time algorithms for counting problems on graphs that would otherwise appear to take exponential time. We show how the holographic transformation can be understood as a collection of changes of basis on individual edges of the graph, thus providing a new level of freedom to the method, as each edge may now independently choose a basis so as to transform the matrices into the required form.</p>
<p>Consideration of zipper networks makes it clear that "fan-out," i.e., the ability to duplicate information (for example allowing a variable to be used in many places), is most naturally itself represented as a relation along with everything else. This is a notable departure from the traditional lack of representation for this ability. This deconstruction of fan-out provides a more general model for combining relations than was provided by previous models, since we can examine both the traditional case where fan-out (the equality relation on three variables) is available and the more interesting case where its availability is sub ject to the same limitations as the availability of other relations. As we investigate the composition of relations in this model where fan-out is explicit, what we find is very different from what has been found in the past.</p>
<p>First of all we examine the relative expressive power of small relations: For each relation on three boolean variables, we examine which others can be implemented by networks built solely from that relation. (We also find, in each of these cases, the complexity of deciding whether such a network has a solution. We find that solutions can be found in polynomial time for all but one case, which is NP-complete.) For the question of which relations are able to implement which others, we provide an extensive and complete answer in the form of a hierarchy of relative expressive power for these relations. The hierarchy for relations is more complex than Post's well-known comparable hierarchy for functions, and parts of it are particularly difficult to prove. We find an explanation for this phenomenon by showing that in fact, the question of whether one relation can implement another (and thus should be located above it in the hierarchy) is undecidable. We show this by means of a complicated reduction from the halting problem for register machines. The hierarchy itself has a lot of structure, as it is rarely the case that two ternary boolean relations are equivalent. Often they are comparable, and often they are incomparable—the hierarchy has quite a bit of width as well as depth. Notably, the fan-out relation is particularly difficult to implement; only a very few relations are capable of implementing it. This provides an additional ex post facto justification for considering the case where fan-out is absent: If you are not explicitly provided with fan-out, you are unlikely to be able to implement it.</p>
<p>The undecidability of the hierarchy contrasts strongly with the traditional case, where the ubiquitous availability of fan-out causes all implementability questions to collapse into a finite decidable form. Thus we see that for implementability among relations, fan-out leads to undecidability. We then go on to examine whether this result might be taken back to the world of functions to find a similar difference there. As we study the implementability question among functions without fan-out, we are led directly to questions that are independently compelling, as our functional implementability question turns out to be equivalent to asking what can be computed by sets of chemical reactions acting on a finite number of species. In addition to these chemical reaction networks, several other nondeterministic systems are also found to be equivalent in this way to the implementability question, namely, Petri nets, unordered Fractran, vector addition systems, and "broken" register machines (whose decrement instruction may fail even on positive registers). We prove equivalences between these systems.</p>
<p>We find several interesting results in particular for chemical reaction networks, where the standard model has reaction rates that depend on concentration. In this setting, we analyze questions of possibility as well as questions of probability. The question of the possibility of reaching a target state turns out to be equivalent to the reachability question for Petri nets and vector addition systems, which has been well studied. We provide a new proof that a form of this reachability question can be decided by primitive recursive functions. Ours is the first direct proof of this relationship, avoiding the traditional excursion to Diophantine equations, and thus providing a crisper picture of the relationship between Karp's coverability tree and primitive recursive functions.</p>
<p>In contrast, the question of finding the probability (according to standard chemical kinetics) of reaching a given target state turns out to be undecidable. Another way of saying this is that if we wish to distinguish states with zero probability of occurring from states with positive probability of occurring, we can do so, but if we wish to distinguish low probability states from high probability states, there is no general way to do so. Thus, if we wish to use a chemical reaction network to perform a computation, then if we insist that the network must always get the right answer, we will only be able to use networks with limited computational power, but if we allow just the slightest probability of error, then we can use networks with Turing-universal computational ability. This power of probability is quite surprising, especially when contrasted with the conventional computational complexity belief that BPP = P.</p>
<p>Exploring the source of this probabilistic power, we find that the probabilities guiding the network need to depend on the concentrations (or perhaps on time)—fixed probabilities aren’t enough on their own to achieve this power. In the language of Petri nets, if one first picks a transition at random, and then fires it if it is enabled, then the probability of reaching a particular target state can be calculated to arbitrary precision, but if one first picks a token at random, and then fires an enabled transition that will absorb that token, then the probability of reaching a particular target state cannot in general be calculated to any precision whatsoever.</p>
<p>In short, what started as a simple thorough exploration of the power of composition of relations has led to many decidability and complexity questions that at first appear completely unrelated, but turn out to combine to paint a coherent picture of the relationship between relations and functions, implementability and reachability, possibility and probability, and decidability and undecidability.</p>https://thesis.library.caltech.edu/id/eprint/2424Ultra-High-Q Planar Microcavities and Applications
https://resolver.caltech.edu/CaltechETD:etd-05272005-113247
Authors: {'items': [{'email': 'darmani@gmail.com', 'id': 'Armani-Deniz-Karapetian', 'name': {'family': 'Armani', 'given': 'Deniz Karapetian'}, 'show_email': 'YES'}]}
Year: 2005
DOI: 10.7907/EZHA-VY23
Ultra-high-Q (UHQ) silica microspheres have found research applications in diverse fields ranging from telecommunications to nonlinear optics to biological and chemical sensing. However, despite having quality factors greater than 108, the silica microsphere has not moved to an industrial setting because of several major drawbacks. The most hindering is the manual fabrication technique used that makes tight process control difficult and integration with other optical or electrical components impossible. Despite the strong desire to fabricate an integrated UHQ microresonator on a planar substrate, the highest quality factor achieved for any micro-fabricated planar micro-cavity (at the time of my first publication) was over 4 orders of magnitude lower than for silica microspheres. In this thesis, a process for creating planar micro-cavities with Q factors in excess of 400 million on silicon wafers is demonstrated. The advantage of these planar ultra-high-Q (UHQ) microtoroid resonators is that they successfully overcome the previously mentioned drawbacks of silica microsphere resonators while maintaining nearly identical, if not better, performance characteristics. Additionally, due to the planar nature of these new devices, functionality has been integrated in-situ while maintaining UHQ for the first time, such as active resonant frequency tuning, coupling control, and low-threshold lasing.https://thesis.library.caltech.edu/id/eprint/2146Data Complexity in Machine Learning and Novel Classification Algorithms
https://resolver.caltech.edu/CaltechETD:etd-04122006-114210
Authors: {'items': [{'id': 'Li-Ling', 'name': {'family': 'Li', 'given': 'Ling'}, 'show_email': 'NO'}]}
Year: 2006
DOI: 10.7907/EW2G-9986
<p>This thesis summarizes four of my research projects in machine learning. One of them is on a theoretical challenge of defining and exploring complexity measures for data sets; the others are about new and improved classification algorithms.</p>
<p>We first investigate the role of data complexity in the context of binary classification problems. The universal data complexity is defined for a data set as the Kolmogorov complexity of the mapping enforced by that data set. It is closely related to several existing principles used in machine learning such as Occam's razor, the minimum description length, and the Bayesian approach. We demonstrate the application of the data complexity in two learning problems, data decomposition and data pruning. In data decomposition, we illustrate that a data set is best approximated by its principal subsets which are Pareto optimal with respect to the complexity and the set size. In data pruning, we show that outliers usually have high complexity contributions, and propose methods for estimating the complexity contribution. Experiments were carried out with a practical complexity measure on several toy problems.</p>
<p>We then propose a family of novel learning algorithms to directly minimize the 0/1 loss for perceptrons. A perceptron is a linear threshold classifier that separates examples with a hyperplane. Unlike most perceptron learning algorithms, which require smooth cost functions, our algorithms directly minimize the 0/1 loss, and usually achieve the lowest training error compared with other algorithms. The algorithms are also computationally efficient. Such advantages make them favorable for both standalone use and ensemble learning, on problems that are not linearly separable. Experiments show that our algorithms work very well with AdaBoost.</p>
<p>We also study ensemble methods that aggregate many base hypotheses in order to achieve better performance. AdaBoost is one such method for binary classification problems. The superior out-of-sample performance of AdaBoost has been attributed to the fact that it minimizes a cost function based on the margin, in that it can be viewed as a special case of AnyBoost, an abstract gradient descent algorithm. We provide a more sophisticated abstract boosting algorithm, CGBoost, based on conjugate gradient in function space. When the AdaBoost exponential cost function is optimized, CGBoost generally yields much lower cost and training error but higher test error, which implies that the exponential cost is vulnerable to overfitting. With the optimization power of CGBoost, we can adopt more "regularized" cost functions that have better out-of-sample performance but are difficult to optimize. Our experiments demonstrate that CGBoost generally outperforms AnyBoost in cost reduction. With suitable cost functions, CGBoost can have better out-of-sample performance.</p>
<p>A multiclass classification problem can be reduced to a collection of binary problems with the aid of a coding matrix. The quality of the final solution, which is an ensemble of base classifiers learned on the binary problems, is affected by both the performance of the base learner and the error-correcting ability of the coding matrix. A coding matrix with strong error-correcting ability may not be overall optimal if the binary problems are too hard for the base learner. Thus a trade-off between error-correcting and base learning should be sought. In this paper, we propose a new multiclass boosting algorithm that modifies the coding matrix according to the learning ability of the base learner. We show experimentally that our algorithm is very efficient in optimizing the multiclass margin cost, and outperforms existing multiclass algorithms such as AdaBoost.ECC and one-vs-one. The improvement is especially significant when the base learner is not very powerful.</p>https://thesis.library.caltech.edu/id/eprint/1361Multi-robot Systems: Modeling Swarm Dynamics and Designing Inspection Planning Algorithms
https://resolver.caltech.edu/CaltechETD:etd-05192006-063455
Authors: {'items': [{'email': 'kjerstinwilliams@gmail.com', 'id': 'Williams-Kjerstin-Irja', 'name': {'family': 'Williams', 'given': 'Kjerstin Irja'}, 'show_email': 'NO'}]}
Year: 2006
DOI: 10.7907/G1T2-FB74
<p>For a variety of applications, the capability of simultaneous sensing and action in multiple locations that is inherent to multi-robot approaches offers potential advantages over single robot systems in robustness, efficiency, and application feasibility.</p>
<p>At the fully distributed and reactive end of the multi-robot system spectrum, I present mathematical modeling methodologies developed to predict and optimize a self-organized robotic swarm’s performance for several tasks. These models allow us to better understand the relationship between agent and group behavior by capturing the dynamics of these highly stochastic, nonlinear, asynchronous systems at various levels of abstraction, in some cases even achieving mathematical tractability. The models deliver qualitatively and quantitatively correct predictions several orders of magnitude more quickly than an embodied simulator can. Swarm modeling lays the foundation for more generalized SI system design methodology by saving time, enabling generalization to different robotic platforms, and estimating optimal design and control parameters.</p>
<p>In considering more complex target tasks and behaviors, efficiency and completeness of execution may be of concern, and a swarm approach may not be appropriate. In such cases a more deliberative approach may be warranted. In that context, I introduce the multi-robot boundary coverage problem, in which a group of robots is required to completely inspect the boundary of all two-dimensional objects in a specified environment. To make such a guarantee, I present a centralized planning approach that constructs a two-component abstraction of the problem: a graph representing the particular instance of the inspection task and a graph problem whose solution represents a complete plan for inspection. Using the building blocks of this approach, related inspection tasks that require the robotic system to adapt to a changes in team size and task assignment are also explored. The application of these planning methods to the case of long-term deployment for surveillance applications that require repetitive coverage is also discussed.</p>
<p>The recurring theme of this thesis is that we must look beyond implementation and validation of a particular system and ask how its design can contribute to the development of a more general design methodology.</p>https://thesis.library.caltech.edu/id/eprint/1876The Self-Replication and Evolution of DNA Crystals
https://resolver.caltech.edu/CaltechETD:etd-04302007-164103
Authors: {'items': [{'email': 'rebecca.b.schulman@gmail.com', 'id': 'Schulman-Rebecca-Beth', 'name': {'family': 'Schulman', 'given': 'Rebecca Beth'}, 'orcid': '0000-0003-4555-3162', 'show_email': 'NO'}]}
Year: 2007
DOI: 10.7907/3F8C-9D50
<p>How life began is still a mystery. While various theories suggest that life began in deep sea volcanic vents or that a world where life consisted predominantly of RNA molecules preceded us, there is no hard evidence to give shape to the chain of events that led to cellular life.</p>
<p>Perhaps the fundamental enigma of our origins is how life began to self-replicate in such a way that evolution could produce Earth's "endless forms most beautiful." With the exception of biological organisms, we have no examples of self-replicating, evolving chemical systems, despite an extensive research program with the goal of identifying them.</p>
<p>In this thesis, I construct a chemical system that is capable of the most basic self-replication and evolution. The system uses no enzymes or biological sequences, can support and replicate a combinatorial genome, and is completely autonomous. There are no fundamental obstacles to the replication by this system of much more complex sequences or to open-ended evolution.</p>
<p>The design of the system is inspired by the work of Graham Cairns-Smith, who has proposed that life began with clay. Clays are tiny layered crystals; some clay crystals can contain one of several different patterns of atoms or molecules in each layer. The choice of patterns for the layers could be viewed as a sort of genome: it would be copied as the clay grew, and if the crystal broke, each new piece would inherit its pattern from the old piece and could replicate it in the same manner. If some patterns of layers could grow and reproduce faster than other patterns, crystals with faster-growing patterns would be selected for.</p>
<p>Instead of the atoms or small molecules of which clay consists, I use molecules consisting of 4-6 interwoven, synthetic DNA strands called DNA tiles to construct crystals that replicate and evolve as Cairns-Smith imagined. While the choice of construction material was influenced by ease of use -- in contrast to clay crystals, DNA tile crystals have been previously characterized and are easy to crystallize and image in the laboratory -- the choice was fundamentally made because DNA tile monomers are programmable, allowing us to create novel crystal morphologies rationally.</p>
<p>The crystals I construct, termed "zig-zag ribbons", contain a sequence of information ("a genome") in each row. Growth of the ribbon adds rows, one at time, each of which contain an arrangement of DNA tiles that encode the same information sequence as the previous row. Altering the set of "tiles" used to assemble ribbons allows us to alter the alphabets for and the permitted lengths of sequences that can be copied.</p>
<p>I describe how to design tile sets that can replicate genomes with different alphabets and the kind of sequence evolution that is in theory possible with some simple tile sets. Altering the tile set can not only change the kinds of sequences that may be replicated, it can also make growth and splitting more robust. I show how to make changes to the crystals' design to prevent errors during growth and splitting and to reduce the rate of spontaneous generation of new crystals.</p>
<p>It has been previously shown that DNA tile crystallization can be used to perform universal computation; I show that in theory crystals that can compute can undergo open-ended evolution as they try to produce more and more complex programs to take advantage of available growth resources. This mechanism is simple enough to potentially observe in the laboratory in the near future. In experiments, I demonstrate a much more basic kind of replication and evolution, in which zig-zag ribbons maintain a preference for a certain width into a second generation.</p>
<p>This work suggests that the concept of a self-replicating chemistry is closely related to the concept of a chemistry that can store information and compute. It is only by clearly understanding how chemistry can perform these latter tasks that we can hope to understand how self-replication and evolution can occur, and by implication, understand how life might have begun.</p>https://thesis.library.caltech.edu/id/eprint/1554Signal Processing Methods for Genomic Sequence Analysis
https://resolver.caltech.edu/CaltechETD:etd-04092007-162353
Authors: {'items': [{'email': 'bjyoon@tamu.edu', 'id': 'Yoon-Byung-Jun', 'name': {'family': 'Yoon', 'given': 'Byung-Jun'}, 'show_email': 'NO'}]}
Year: 2007
DOI: 10.7907/48J3-G286
<p>Signal processing is the art of representing, transforming, analyzing, and manipulating signals. It deals with a wide range of signals, from speech and audio signals to images and video signals, and many others. Signal processing techniques have been found very useful in diverse applications. Traditional applications include signal enhancement, denoising, speech recognition, audio and image compression, radar signal processing, and digital communications, just to name a few. In recent years, signal processing techniques have been also applied to the analysis of biological data with considerable success. For example, they have been used for predicting protein-coding genes, analyzing ECG signals and MRI data, enhancing and normalizing DNA microarray images, modeling gene regulatory networks, and so forth.</p>
<p>In this thesis, we consider the application of signal processing methods to the analysis of biological sequences, especially, DNA and RNA molecules. We demonstrate how conventional signal processing techniques--such as digital filters and filter banks--can contribute to this end, and also show how we can extend the traditional models--such as the hidden Markov models (HMMs)--to better serve this purpose.</p>
<p>The first part of the thesis focuses on signal processing methods that can be utilized for analyzing RNA sequences. The primary purposes of this part are to develop a statistical model that is suitable for representing RNA sequence profiles and to propose an effective framework that can be used for finding new homologues (i.e., similar RNAs that are biologically related) of known RNAs. Many functional RNAs have secondary structures that are well conserved among different species. The RNA secondary structure gives rise to long-range correlations between distant bases, which cannot be represented using traditional HMMs. In order to overcome this problem, we propose a new statistical model called the context-sensitive HMM (csHMM). The csHMM is an extension of the traditional HMM, where certain states have variable emission and transition probabilities that depend on the context. The context-sensitive property increases the descriptive power of the model significantly, making csHMMs capable of representing long-range correlations between distant symbols. Based on the proposed model, we present efficient algorithms that can be used for finding the optimal state sequence and computing the probability of an observed symbol string. We also present a training algorithm that can be used for optimizing the parameters of a csHMM. We give several examples that illustrate how csHMMs can be used for modeling various RNA secondary structures and recognizing them.</p>
<p>Based on the concept of csHMM, we introduce profile-csHMMs, which are specifically constructed csHMMs that have linear repetitive structures (i.e., state-transition diagrams). Profile-csHMMs are especially useful for building probabilistic representations of RNA sequence families, including pseudoknots. We also propose a dynamic programming algorithm called the sequential component adjoining (SCA) algorithm that can systematically find the optimal state sequence of an observed symbol string based on a profile-csHMM. In order to demonstrate the effectiveness of profile-csHMMs, we build a structural alignment tool for RNA sequences and show that the profile-csHMM approach can yield highly accurate predictions at a relatively low computational cost. At the end, we describe how the profile-csHMM can be used for finding homologous RNAs, and we propose a practical scheme for making the search significantly faster without affecting the prediction accuracy.</p>
<p>In the second part of the thesis, we focus on the application of digital filters and filter banks in DNA sequence analysis. Firstly, we demonstrate how we can use digital filters for predicting protein-coding genes. Many coding regions in DNA molecules are known to display a period-3 behavior, which can be effectively detected using digital filters. Efficient schemes are proposed that can be used for designing such filters. Experimental results will show that the digital filtering approach can clearly identify the coding regions at a very low computational cost. Secondly, we propose a method based on a bank of IIR lowpass filters that can be used for predicting CpG islands, which are specific regions in DNA molecules that are abundant in the dinucleotide CpG. This filter bank is used to process the sequence of log-likelihood ratios obtained from two Markov chains, where the respective Markov chains model the base transition probabilities inside and outside the CpG islands. The locations of the CpG islands are predicted by analyzing the output signals of the filter bank. It will be shown that the filter bank approach can yield reliable prediction results without sacrificing the resolution of the predicted start/end positions of the CpG islands.</p>
https://thesis.library.caltech.edu/id/eprint/5182A Treatise on Econometric Forecasting
https://resolver.caltech.edu/CaltechETD:etd-05222007-101946
Authors: {'items': [{'id': 'Martinez-Estrada-Alfredo', 'name': {'family': 'Martinez Estrada', 'given': 'Alfredo'}, 'show_email': 'NO'}]}
Year: 2007
DOI: 10.7907/WXN5-9A47
We investigate the effects of model misspecification and stochastic dynamics in the problem of forecasting. In economics and many fields of engineering, many researchers are guilty of the dangerous practice of treating their mathematical models as the true data generating mechanisms responsible for the observed phenomena and downplaying or omitting all together the important step of model verification. In recent years, econometricians have acknowledged the need to account for model misspecification in the problems of estimation and forecasting. In particular, a large body of work has emerged to address properties of estimators under model misspecification, along with a plethora of misspecification testing methodologies. In this work, we investigate the combined effects of model misspecification and various types of stochastic dynamics on forecasts based on linear regression models. The data generating process (DGP) is assumed unknown to the forecaster except for the nature of process dependencies, i.e., independent identically distributed, covariance stationary, or nonstationary. Estimation is carried out by means of ordinary least squares, and forecasts are evaluated with the mean squared forecast error (MSFE) or mean square error of prediction. We investigate the sample size dependence of the MSFE. For this purpose, we develop an algorithm to approximate the MSFE by an expression depending only on the sample size n and moments of the processes. The approximation is constructed by Taylor series expansions of the squared forecast error which do not require knowledge of the functional form of the DGP. The approximation can be used to determine the existence of optimal observation windows which result in the minimum MSFE. We assess the accuracy of the approximating algorithm with Monte Carlo experiments.
https://thesis.library.caltech.edu/id/eprint/1951Robotics Training Algorithms for Optimizing Motor Learning in Spinal Cord Injured Subjects
https://resolver.caltech.edu/CaltechETD:etd-08142006-165844
Authors: {'items': [{'id': 'Cai-Lance-Lin-Lan', 'name': {'family': 'Cai', 'given': 'Lance Lin-Lan'}, 'show_email': 'YES'}]}
Year: 2007
DOI: 10.7907/EH12-WD80
<p>The circuitries within the spinal cord are remarkably robust and plastic. Even in the absence of supraspinal control, such circuitries are capable of generating functional movements and changing their level of excitability based on a specific combination of properceptive inputs going into the spinal cord. This has led to an increase in locomotor training, such as Body Weight Support Treadmill training (BWST) for spinal cord injured (SCI) patients. However, today, little is known about the underlying physiological mechanisms responsible for the locomotor recovery achieved with this type of rehabilitative training, and the optimal rehabilitative strategy is still unknown.</p>
<p>This thesis describes a mouse model to study the effect of rehabilitative training on SCI. Using this model, the effects of locomotor recovery on adult spinal mice following complete spinal cord transaction is examined. Results that indicate adult spinal mice can be robotically trained to step, and when combined with the administration of quipazine (a broad serotonin agonist), there is an interaction and retention effect. Results also demonstrate that the training paradigm can be optimized in using “Assisted-as-Needed” (AAN) training. To find the optimal AAN training parameters, a learning model is developed to test the effect of various parameters of the AAN training algorithm. Simulation results from our model show that learning is training-dependent. In addition, the model predicts that improved motor learning can improve post-SCI by making the AAN training more adaptable.</p>
<p>The primary contributions of this thesis are twofold, in biology and engineering. We develop a mouse model using novel robotic devices and controls that can be used to study SCI and other locomotor disorders in the future by taking advantage of the many different strains of transgenic mice that are commercially available. We also further confirm that sensory integration responsible for motor control is distributed throughout the hierarchy of the neuromuscular system and can be achieved within the isolated spinal cord. Lastly, by developing a learning model, we can start looking into how variability plays a role in motor learning, the understanding of which will have profound implications in neurophysiology, machine learning and adaptive optimal controls research.</p>https://thesis.library.caltech.edu/id/eprint/3117Discriminative vs. Generative Object Recognition: Objects, Faces, and the Web
https://resolver.caltech.edu/CaltechETD:etd-05312007-204007
Authors: {'items': [{'id': 'Holub-Alex-David', 'name': {'family': 'Holub', 'given': 'Alex David'}, 'show_email': 'NO'}]}
Year: 2007
DOI: 10.7907/2HC2-K923
The ability to automatically identify and recognize objects in images remains one of the most challenging and potentially useful problems in computer vision. Despite significant progress over the past decade computers are not yet close to matching human performance. This thesis develops various machine learning approaches for improving the ability of computers to recognize object categories. In particular, it focuses on approaches which are able to distinguish between object categories which are visually similar to one another. Examples of similar visual object categories are motorcycles and bicycles, and lions and cougars. Distinguishing between similar object categories may require different algorithms than distinguishing between different categories. We explore two common machine learning paradigms, generative and discriminative learning, and analyze their respective abilities to distinguish between different sets of object categories. One set of object categories which we are exposed to on a daily basis are face images, and a significant portion of this thesis is spent analyzing different methods for accurately representing and discriminating between faces. We also address a key issue related to the discriminative learning paradigms, namely how to collect the large training set of images necessary to accurately learn discriminative models. In particular, we suggest a novel active learning which intelligently chooses the most informative image to label and thus drastically reduces (up to 10x) the time required to collect a training set. We validate and analyze our algorithms on large data-sets collected from the web and show how using hybrid generative-discriminative techniques can drastically outperform previous algorithms. In addition, we show how to use our techniques in practical applications such as finding similar-looking individuals within large data-sets of faces, discriminating between large sets of visual categories, and increasing the efficiency and speed of web-image searchihttps://thesis.library.caltech.edu/id/eprint/2344Towards Automatic Discovery of Human Movemes
https://resolver.caltech.edu/CaltechETD:etd-02262008-172531
Authors: {'items': [{'email': 'fanti@vision.caltech.edu', 'id': 'Fanti-Claudio', 'name': {'family': 'Fanti', 'given': 'Claudio'}, 'show_email': 'NO'}]}
Year: 2008
DOI: 10.7907/M4PS-B302
<p>Consider a number of moving points, each attached to a joint of the human body and projected onto an image. Johannson showed that humans can effortlessly detect and recognize the presence of other humans from such displays. This is true even when some of the body parts are missing (e.g., because of occlusion) and unrelated clutter points are added to the display. Furthermore, subtle aspects like age range and gender, as well as the ongoing activity, can be inferred with a surprising degree of accuracy from such a seemingly scarce amount of information. We are interested in replicating some of these abilities in a machine.</p>
<p>We start by introducing a labeling and detection scheme in a Johannson-like display. Our method is based on a probabilistic representation of the positions and motion of body parts, which we use to calculate a likely interpretation of the scene by means of belief propagation techniques. We show how learning and inference can be done efficiently, and we provide an experimental validation of the method.</p>
<p>In the second part of our work, we present our position on the analysis of human behaviors. We hypothesize a hierarchical description of motion, which provides a natural interpretation of actions and activities as stochastic sequences of "atomic motions" or movemes. We take an initial step in that direction by illustrating how to learn a dictionary of movemes from the trajectories of body parts, which can be used to concisely represent the video for further analysis.</p>
https://thesis.library.caltech.edu/id/eprint/5178Nonparametric Detection and Estimation of Highly Oscillatory Signals
https://resolver.caltech.edu/CaltechETD:etd-05112008-152328
Authors: {'items': [{'email': 'hannes.helgason@gmail.com', 'id': 'Helgason-Hannes', 'name': {'family': 'Helgason', 'given': 'Hannes'}, 'show_email': 'YES'}]}
Year: 2008
DOI: 10.7907/SAEK-MV13
<p>This thesis considers the problem of detecting and estimating highly oscillatory signals from noisy measurements. These signals are often referred to as chirps in the literature; they are found everywhere in nature, and frequently arise in scientific and engineering problems. Mathematically, they can be written in the general form A(t) exp(ilambda varphi(t)), where lambda is a large constant base frequency, the phase varphi(t) is time-varying, and the envelope A(t) is slowly varying. Given a sequence of noisy measurements, we study two problems seperately: 1) the problem of testing whether or not there is a chirp hidden in the noisy data, and 2) the problem of estimating this chirp from the data.</p>
<p>This thesis introduces novel, flexible and practical strategies for addressing these important nonparametric statistical problems. The main idea is to calculate correlations of the data with a rich family of local templates in a first step, the multiscale chirplets, and in a second step, search for meaningful aggregations or chains of chirplets which provide a good global fit to the data. From a physical viewpoint, these chains correspond to realistic signals since they model arbitrary chirps. From an algorithmic viewpoint, these chains are identified as paths in a convenient graph. The key point is that this important underlying graph structure allows to unleash very effective algorithms such as network flow algorithms for finding those chains which optimize a near optimal trade-off between goodness of fit and complexity.</p>
<p>Our estimation procedures provide provably near optimal performance over a wide range of chirps and numerical experiments show that both our detection and estimation procedures perform exceptionally well over a broad class of chirps. This thesis also introduces general strategies for extracting signals of unknown duration in long streams of data when we have no idea where these signals may be. The approach is leveraging testing methods designed to detect the presence of signals with known time support.</p>
<p>Underlying our methods is a general abstraction which postulates an abstract statistical problem of detecting paths in graphs which have random variables attached to their vertices. The formulation of this problem was inspired by our chirp detection methods and is of great independent interest.</p>https://thesis.library.caltech.edu/id/eprint/1726Molecules Computing: Self-Assembled Nanostructures, Molecular Automata,and Chemical Reaction Networks
https://resolver.caltech.edu/CaltechETD:etd-05292008-142339
Authors: {'items': [{'email': 'david.soloveichik@utexas.edu', 'id': 'Soloveichik-David', 'name': {'family': 'Soloveichik', 'given': 'David'}, 'orcid': '0000-0002-2585-4120', 'show_email': 'NO'}]}
Year: 2008
DOI: 10.7907/ZGE0-AF38
Many endeavors of molecular-level engineering either rely on biological material such as nucleic acids and restriction enzymes, or are inspired by biological processes such as self-assembly or cellular regulatory networks. This thesis develops theories on three such topics: self-assembled nanostructures, molecular automata, and chemical reaction networks. The abstractions and underlying methods of the theories presented herein are based on computer science and include Turing machines and circuits. Toward engineering self-assembled nanostructures, we create a theory of scale-free shapes in which the complexity of their self-assembly is connected to the shapes' descriptional complexity. Further, we study patterns in terms of whether they can be self-assembled robustly without an increase in scale to accommodate redundancy. We also describe a new method of ensuring resilience to more types of error simultaneously. Toward creating molecular automata we study the computational power of a restriction enzyme-based automaton. Toward designing chemical reaction networks, we develop a technique of storing and processing information in molecular counts, which is capable of achieving Turing universal computation. We also study the computational complexity of simulating stochastic chemical reaction networks and formally connect robustness and simulation efficiency. Lastly, we describe nucleic acid implementations of Boolean logic circuits and arbitrary mass-action kinetics. The three areas of this thesis are promising realizations of molecular-level engineering, and the theories presented here inform the range of possibility or delineate inherent difficulties in these areas. https://thesis.library.caltech.edu/id/eprint/2261Adaptive Learning Algorithms and Data Cloning
https://resolver.caltech.edu/CaltechETD:etd-05292008-231048
Authors: {'items': [{'id': 'Pratap-Amrit', 'name': {'family': 'Pratap', 'given': 'Amrit'}, 'show_email': 'NO'}]}
Year: 2008
DOI: 10.7907/GV3D-AB69
<p>This thesis is in the field of machine learning: the use of data to automatically learn a hypothesis to predict the future behavior of a system. It summarizes three of my research projects.</p>
<p>We first investigate the role of margins in the phenomenal success of the Boosting Algorithms. AdaBoost (Adaptive Boosting) is an algorithm for generating an ensemble of hypotheses for classification. The superior out-of-sample performance of AdaBoost has been attributed to the fact that it can generate a classifier which classifies the points with a large margin of confidence. This led to the development of many new algorithms focusing on optimizing the margin of confidence. It was observed that directly optimizing the margins leads to a poor performance. This apparent contradiction has been the topic of a long unresolved debate in the machine-learning community. We introduce new algorithms which are expressly designed to test the margin hypothesis and provide concrete evidence which refutes the margin argument.</p>
<p>We then propose a novel algorithm for Adaptive sampling under Monotonicity constraint. The typical learning problem takes examples of the target function as input information and produces a hypothesis that approximates the target as an output. We consider a generalization of this paradigm by taking different types of information as input, and producing only specific properties of the target as output. This is a very common setup which occurs in many different real-life settings where the samples are expensive to obtain. We show experimentally that our algorithm achieves better performance than the existing methods, such as Staircase procedure and PEST.</p>
<p>One of the major pitfalls in machine learning research is that of selection bias. This is mostly introduced unconsciously due to the choices made during the learning process, which often lead to over-optimistic estimates of the performance. In the third project, we introduce a new methodology for systematically reducing selection bias. Experiments show that using cloned datasets for model selection can lead to better performance and reduce the selection bias.</p>https://thesis.library.caltech.edu/id/eprint/2267Adaptive Feature Selection in Pattern Recognition and Ultra-Wideband Radar Signal Analysis
https://resolver.caltech.edu/CaltechETD:etd-05302008-134607
Authors: {'items': [{'email': 'jianghao@caltech.edu', 'id': 'Jiang-Hao', 'name': {'family': 'Jiang', 'given': 'Hao'}, 'show_email': 'NO'}]}
Year: 2008
DOI: 10.7907/7NR6-AR24
<p>Feature selection from measured data aims to extract informative features to reveal the statistic or stochastic mechanism underlying the complicated or high dimensional original data. In this thesis, the feature selection problem is probed under two situations, one is pattern recognition and the other is ultra-wideband radar signal analysis.</p>
<p>Classical pattern recognition methods select features by their ability to separate the multiple classes with certain gauge measure. The deficiency in this general strategy is its lack of adaptation in specific situations. This deficiency may be overcome by viewing the selected features as a function of not only the training samples but also the unlabeled test data. From this perspective, this thesis proposes an adaptive sequential feature selection algorithm which utilizes an information-theoretic measure to reduce the classification task complexity sequentially, and finally outputs the probabilistic classification result and its variation level. To verify the potential advantage of this algorithm, this thesis applies it to one important problem of neural prosthesis, which concerns decoding a finite number of classes, intended reach directions, from recordings of neural activities in the Parietal Reach Region of one rhesus monkey. Experimental results show that the classification scheme of combining the adaptive sequential feature selection algorithm and the information fusion method outperforms some classical pattern recognition rules, such as the nearest neighbor rule and support vector machine, in decoding performance.</p>
<p>The second scenario in this thesis targets developing a human presence and motion pattern detector through ultra-wideband radar signal analysis. To augment the detection robustness, both static and dynamic features should be utilized. The static features reflect the information of target geometry and its variability, while the dynamic features extract the temporal structure among radar scans. The problem of static feature selection is explored in this thesis, which utilizes the Procrustes shape analysis to generate the representative template for the target images, and makes statistical inference in the tangent space through the Hotelling one sample test. After that, the waveform shape variation structure is decomposed in the tangent space through the principal component analysis. The selected principal components not only accentuate the prominent dynamics of the target motion, but also generate another informative classification feature.</p>
https://thesis.library.caltech.edu/id/eprint/2318From Ordinal Ranking to Binary Classification
https://resolver.caltech.edu/CaltechETD:etd-05302008-143505
Authors: {'items': [{'email': 'htlin@csie.ntu.edu.tw', 'id': 'Lin-Hsuan-Tien', 'name': {'family': 'Lin', 'given': 'Hsuan-Tien'}, 'orcid': '0000-0003-2968-0671', 'show_email': 'YES'}]}
Year: 2008
DOI: 10.7907/7B0F-E145
<p>We study the ordinal ranking problem in machine learning. The problem can be viewed as a classification problem with additional ordinal information or as a regression problem without actual numerical information. From the classification perspective, we formalize the concept of ordinal information by a cost-sensitive setup, and propose some novel cost-sensitive classification algorithms. The algorithms are derived from a systematic cost-transformation technique, which carries a strong theoretical guarantee. Experimental results show that the novel algorithms perform well both in a general cost-sensitive setup and in the specific ordinal ranking setup.</p>
<p>From the regression perspective, we propose the threshold ensemble model for ordinal ranking, which allows the machines to estimate a real-valued score (like regression) before quantizing it to an ordinal rank. We study the generalization ability of threshold ensembles and derive novel large-margin bounds on its expected test performance. In addition, we improve an existing algorithm and propose a novel algorithm for constructing large-margin threshold ensembles. Our proposed algorithms are efficient in training and achieve decent out-of-sample performance when compared with the state-of-the-art algorithm on benchmark data sets.</p>
<p>We then study how ordinal ranking can be reduced to weighted binary classification. The reduction framework is simpler than the cost-sensitive classification approach and includes the threshold ensemble model as a special case. The framework allows us to derive strong theoretical results that tightly connect ordinal ranking with binary classification. We demonstrate the algorithmic and theoretical use of the reduction framework by extending SVM and AdaBoost, two of the most popular binary classification algorithms, to the area of ordinal ranking. Coupling SVM with the reduction framework results in a novel and faster algorithm for ordinal ranking with superior performance on real-world data sets, as well as a new bound on the expected test performance for generalized linear ordinal rankers. Coupling AdaBoost with the reduction framework leads to a novel algorithm that boosts the training accuracy of any cost-sensitive ordinal ranking algorithms theoretically, and in turn improves their test performance empirically.</p>
<p>From the studies above, the key to improve ordinal ranking is to improve binary classification. In the final part of the thesis, we include two projects that aim at understanding binary classification better in the context of ensemble learning. First, we discuss how AdaBoost is restricted to combining only a finite number of hypotheses and remove the restriction by formulating a framework of infinite ensemble learning based on SVM. The framework can output an infinite ensemble through embedding infinitely many hypotheses into an~SVM kernel. Using the framework, we show that binary classification (and hence ordinal ranking) can be improved by going from a finite ensemble to an infinite one. Second, we discuss how AdaBoost carries the property of being resistant to overfitting. Then, we propose the SeedBoost algorithm, which uses the property as a machinery to prevent other learning algorithms from overfitting. Empirical results demonstrate that SeedBoost can indeed improve an overfitting algorithm on some data sets.</p>https://thesis.library.caltech.edu/id/eprint/2321Visual Prediction of Rover Slip: Learning Algorithms and Field Experiments
https://resolver.caltech.edu/CaltechETD:etd-10032007-121619
Authors: {'items': [{'email': 'anelia.angelova@gmail.com', 'id': 'Angelova-Anelia-Nedelcheva', 'name': {'family': 'Angelova', 'given': 'Anelia Nedelcheva'}, 'orcid': '0000-0003-1822-7943', 'show_email': 'YES'}]}
Year: 2008
DOI: 10.7907/F7FY-5T13
<p>Perception of the surrounding environment is an essential tool for intelligent navigation in any autonomous vehicle. In the context of Mars exploration, there is a strong motivation to enhance the perception of the rovers beyond geometry-based obstacle avoidance, so as to be able to predict potential interactions with the terrain. In this thesis we propose to remotely predict the amount of slip, which reflects the mobility of the vehicle on future terrain. The method is based on learning from experience and uses visual information from stereo imagery as input. We test the algorithm on several robot platforms and in different terrains. We also demonstrate its usefulness in an integrated system, onboard a Mars prototype rover in the JPL Mars Yard.</p>
<p>Another desirable capability for an autonomous robot is to be able to learn about its interactions with the environment in a fully automatic fashion. We propose an algorithm which uses the robot's sensors as supervision for vision-based learning of different terrain types. This algorithm can work with noisy and ambiguous signals provided from onboard sensors. To be able to cope with rich, high-dimensional visual representations we propose a novel, nonlinear dimensionality reduction technique which exploits automatic supervision. The method is the first to consider supervised nonlinear dimensionality reduction in a probabilistic framework using supervision which can be noisy or ambiguous.</p>
<p>Finally, we consider the problem of learning to recognize different terrains, which addresses the time constraints of an onboard autonomous system. We propose a method which automatically learns a variable-length feature representation depending on the complexity of the classification task. The proposed approach achieves a good trade-off between decrease in computational time and recognition performance.</p>
https://thesis.library.caltech.edu/id/eprint/3886Encoding of Financial Signals in the Human Brain
https://resolver.caltech.edu/CaltechETD:etd-10262007-140735
Authors: {'items': [{'email': 'bruguier@alumni.caltech.edu', 'id': 'Bruguier-Antoine-Jean', 'name': {'family': 'Bruguier', 'given': 'Antoine Jean'}, 'orcid': '0000-0003-3668-7927', 'show_email': 'YES'}]}
Year: 2008
DOI: 10.7907/76FW-PD72
<p>Neuroeconomists investigate how the human brain analyzes and makes decisions about financial situations. They use functional magnetic resonance imaging (fMRI) of subjects who participate in economic games. Here we present three such experiments.</p>
<p>In the first experiment, we investigate how the brain recombines expected reward (ER) and risk. Recent fMRI results show that the brain decomposes a gamble in terms of these two metrics. However, economic theory predicts that the brain must recombine them in order to obtain an effective evaluation of the gamble. It was not clear what biological mechanism directs such recombination. Here we show that the brain uses the correlation of noise to recombine signals. We implement a new technique based on canonical correlation analysis and we show that ER is added to risk to form a metric that activates the medial prefrontal cortex.</p>
<p>In the second experiment, we investigate how the brain encodes two gambles instead of one. The brain is likely to encode the utility of each gamble in a common area but in separate groups of neurons. However, it is unknown how the brain indexes the gambles. Indeed, which group of neuron encodes which gamble can be decided in many ways. We hypothesized that the brain would use either the physical position of the gambles or an idiosyncratic parameter, such as ER or risk. Here we introduce a new analysis technique based on Hotelling T-squared statistics and we show that the brain uses risk as an index.</p>
<p>In the third experiment, we investigate a much more complex situation: a stock market. Contrary to what standard finance theory predicts, we hypothesize that the brain does not use mathematical models but instead heuristically uses a social cognition approach. Specifically, we posit that humans understand stock markets by using Theory of Mind (ToM), the ability to attribute to others mental states different from one's own. Here we show that humans engage brain structures related to ToM (paracingulate cortex, anterior cingulate cortex, insula, and amygdala). Subsequent behavioral tests show that ToM, rather than mathematical, abilities are better predictors of success in forecasting stock markets.</p>https://thesis.library.caltech.edu/id/eprint/4273Probabilistic, Features-Based Object Recognition
https://resolver.caltech.edu/CaltechETD:etd-11232007-213140
Authors: {'items': [{'email': 'pierre.moreels@gmail.com', 'id': 'Moreels-Pierre', 'name': {'family': 'Moreels', 'given': 'Pierre'}, 'show_email': 'YES'}]}
Year: 2008
DOI: 10.7907/YGYX-XX55
<p>Object recognition is of fundamental importance in computer vision. In a few years, pedestrian detection, car detection, and more generally scene recognition will likely be reliable enough to allow fully-automated car navigation, and the human driver will be relegated to the back seat to sip his coffee.</p>
<p>In this thesis we are interested in recognizing individual objects and categories. In order to reduce the volume of information one has to process, images are characterized by sets of features. These features, also called interest points, are targeted at image locations with high local information content. Various systems for detecting interest points and for describing the local image appearance near these points, have been proposed in the last two decades. We investigate which combinations from this plethora of detectors and descriptors, are most suited for object recognition tasks.</p>
<p>On to the problem of object recognition, we are first interested in recognizing individual objects. In a few years, one can imagine that customers in shops, will take with their cell phone a picture of a product that looks interesting, send it to a remote server with a huge database of individual objects, and get back information about that specific product. We propose a system for individual object recognition, inspired from previous work on coarse-to-fine recognition. All steps of the recognition process are translated into principled probabilistic terms, which allows us to outperform a state-of-the-art commercial system for individual recognition.</p>
<p>Regarding categories, faces are probably the category that has received the most attention in computer vision literature. Here we propose a system to recognize images of the same individual in large databases of images. This can be of high interest when looking for images of a given person over the internet. Our method's advantage is that it works on real-world images, as opposed to the face databases from the literature, collected in laboratories with controlled lighting, pose and background conditions.</p>
<p>Finally, we are interested in recognition of object categories in general. Using support vector machines for the classification task, we propose a features-based kernel that improves recognition performance on object categories.</p>https://thesis.library.caltech.edu/id/eprint/4644Blind Channel Estimation Using Redundant Precoding: New Algorithms, Analysis, and Theory
https://resolver.caltech.edu/CaltechETD:etd-03102008-010821
Authors: {'items': [{'email': 'borching@gmail.com', 'id': 'Su-Borching', 'name': {'family': 'Su', 'given': 'Borching'}, 'orcid': '0000-0001-8617-2601', 'show_email': 'NO'}]}
Year: 2008
DOI: 10.7907/R7MS-KQ06
<p>Digital signal processing (DSP) techniques have played an important role in channel equalization and estimation in communication systems. While channel equalization and estimation are usually done by pilot-assisted methods in most systems, algorithms for blind channel estimation have also been largely studied due to high bandwidth efficiency. However, up to date, most blind methods possess disadvantages such as slow convergence speed, high complexity, poor performance, etc., compared to pilot-assisted methods. These drawbacks have made many consider blind methods as inapplicable in modern communication systems which feature fast-varying channels.</p>
<p>In this thesis, we consider the blind channel estimation problem in block transmission systems with linear redundant precoding (LRP) which have been widely adopted in modern communication systems in recent years. The main contribution of this thesis is to considerably reduce the amount of received data required for blind estimation and suggest blind methods which are applicable even in fast-varying environments (e.g., in wireless channels). New algorithms are proposed, performance analysis derived, and theoretical issues studied.</p>
<p>The first part of the thesis focuses on new algorithms for blind channel estimation and blind block synchronization in LRP systems. Two major types of linear redundant precoding, namely zero-padding (ZP) and cyclic prefixing (CP), are considered in this thesis. We first propose a generalized, subspace-based algorithm for blind channel estimation in ZP systems of which two previously reported algorithms are special cases. The generalization uses an integer parameter called {it repetition index} which represents the number of repeated uses of each received block. The number of received blocks required for subspace-based blind estimation is roughly inversely proportional to the repetition index. By choosing a larger repetition index, the amount of received data can be significantly reduced.</p>
<p>The concept of repetition index is also applied in blind channel estimation in CP systems, which are more widely used than ZP systems in many current communication standards such as orthogonal frequency division multiplexing (OFDM) systems. The use of repetition index in CP systems is much less obvious and conceptually more complicated than in ZP systems. By choosing a repetition index larger than unity, the number of received blocks needed for blind estimation is significantly reduced compared to all previously reported methods. Theoretically, the proposed method can perform blind estimation using only three received blocks in absence of noise. In practice, the number of received blocks needed to yield a satisfactory bit error rate performance is usually on the order of half the block size. The proposed algorithm can be directly applied in OFDM systems without any modification of transmitter structure. A semiblind algorithm for channel estimation in OFDM systems is also proposed based on the extension of the blind algorithm.</p>
<p>Another important problem, namely the blind block synchronization, is also studied. Most existing blind estimation methods in LRP systems assume the block boundaries of the received streams are perfectly known to the receiver, but this assumption is usually not true in practice since no extra known samples are transmitted. Two algorithms for blind block synchronization are proposed for ZP and CP systems, respectively. In particular, the block synchronization problem in CP systems is a broader version of the timing synchronization problem in the OFDM systems. The proposed algorithms exploit the concept of repetition index and both theoretical and simulation results suggest their advantages over all previously reported algorithms, especially when the amount of received data is limited.</p>
<p>The second part of the thesis deals with theoretical issues related to blind channel estimation. Performance analysis of the generalized blind channel estimation algorithm in ZP systems is first given and shows that the system performance in terms of channel estimation mean square error (MSE) is very close to the Cramer-Rao bound (CRB), even when only two received blocks are available. Another important theoretical problem, namely the signal richness preservation problem, is also studied. Signal richness is an essential property for input signals in subspace-based blind channel estimation algorithms studied in this thesis. This property, however, may be altered by a linear precoder. Necessary and sufficient conditions for a linear precoder to preserve signal richness are explored. Several relevant interesting mathematical problems are also studied.</p>
https://thesis.library.caltech.edu/id/eprint/913Soft-Error Tolerant Quasi Delay-insensitive Circuits
https://resolver.caltech.edu/CaltechETD:etd-11092007-180524
Authors: {'items': [{'email': 'wonjin@caltech.edu', 'id': 'Jang-Wonjin', 'name': {'family': 'Jang', 'given': 'Wonjin'}, 'show_email': 'NO'}]}
Year: 2008
DOI: 10.7907/ZVFF-WE07
<p>A hard error is an error that damages a circuit irrevocably; a soft error flips the logic states without causing any physical damage to the circuit, resulting in transient corruption of data. They result in transient, inconsistent corruption of data.</p>
<p>The soft-error tolerance of logic circuits is recently getting more attention, since the soft- error rate of advanced CMOS devices is higher than before. As a response to the concern on soft errors, we propose a new method for making asynchronous circuits tolerant to soft errors. Since it relies on a property unique to asynchronous circuits, the method is different from what is done in synchronous circuits with triple modular redundancy. Asynchronous circuits have been attractive to the designers of reliable systems, because of their clock-less design, which makes them more robust to variations on computation time of modules. The quasi delay-insensitive (QDI) design style is one of the most robust asynchronous design styles for general computation; it makes one minimal assumption on delays in gates and wires. QDI circuits are easy to verify, simple, and modular, because the correct operation of a QDI circuit is independent of delays in gates and wires.</p>
<p>Here, we shall overview how to design a QDI circuit, and what will happen if a soft error occurs on a QDI circuit. Then the crucial components of the method are shown: (1) a special kind of duplication for random logic (when each bit has to be corrected individually), (2) special protection circuitry for arbiter and synchronizer (as needed for example for external interrupts), (3) reconfigurable circuits using a special configuration unit, and (4) error correcting for memory arrays and other structures in which the data bits can be self- corrected. The solution of protecting random logic is compared with alternatives, which use other types of error correcting codes (e.g., parity code) in a QDI circuit. It turns out that the duplication generates efficient circuits more commonly than other possible constructions. Finally, the design of a soft-error tolerant asynchronous microprocessor is detailed and testing results of the soft-error tolerance of the microprocessor are shown.</p>
https://thesis.library.caltech.edu/id/eprint/5260Foundational Aspects of Nonlocality
https://resolver.caltech.edu/CaltechETD:etd-05282009-115941
Authors: {'items': [{'email': 'gversteeg@gmail.com', 'id': 'VerSteeg-Gregory-Lee', 'name': {'family': 'Ver Steeg', 'given': 'Gregory Lee'}, 'show_email': 'YES'}]}
Year: 2009
DOI: 10.7907/R7FG-AC54
<p>Nonlocality refers to correlations between spatially separated parties that are stronger than those explained by the existence of local hidden variables. Quantum mechanics is known to allow some nonlocal correlations between particles in a phenomena known as entanglement. We explore several aspects of nonlocality in general and how they relate to quantum mechanics.</p>
<p>First, we construct a hierarchy of theories with nonlocal correlations stronger than those allowed in quantum mechanics and derive several results about these theories. We show that these theories include codes that can store an amount of information exponential in the number of physical bits used. We use this result to demonstrate an unphysical consequence of theories with stronger-than-quantum correlations: learning even an approximate description of states in such theories would be practically impossible.</p>
<p>Next, we consider the difficult problem of determining whether specific correlations are nonlocal. We present a novel learning algorithm and show that it provides an outer bound on the set of local states, and can therefore be used to identify some nonlocal states.</p>
<p>Finally, we put nonlocal correlations to work by showing that the entanglement present in the vacuum of a quantum field can be used to detect spacetime curvature. We quantify how the entangling power of the quantum field varies as a function of spacetime curvature.</p>https://thesis.library.caltech.edu/id/eprint/2221Credit Risk and Nonlinear Filtering: Computational Aspects and Empirical Evidence
https://resolver.caltech.edu/CaltechETD:etd-05272009-141742
Authors: {'items': [{'email': 'ac3827@columbia.edu', 'id': 'Capponi-Agostino', 'name': {'family': 'Capponi', 'given': 'Agostino'}, 'orcid': '0000-0001-9735-7935', 'show_email': 'NO'}]}
Year: 2009
DOI: 10.7907/7XV3-9Q45
<p>This thesis proposes a novel credit risk model which deals with incomplete information on the firm's asset value. Such incompleteness is due to reporting bias deliberately introduced by insider managers and executives of the firm and unobserved by outsiders.</p>
<p>The pricing of corporate securities and the evaluation of default measures in our credit risk framework requires the solution of a computationally unfeasible nonlinear filtering problem. The model introduces computational issues arising from the fact that the optimal probability density on the firm's asset value is the solution of a nonlinear filtering problem, which is computationally unfeasible. We propose a polynomial time-sequential Bayesian approximation scheme which employs convex optimization methods to iteratively approximate the optimal conditional density of the state on the basis of received market observations. We also provide an upper bound on the total variation distance between the actual filter density and our approximate estimator. We use the filter estimator to derive analytical expressions for the price of corporate securities (bond and equity) as well as for default measures (default probabilities, recovery rates, and credit spreads) under our credit risk framework. We propose a novel statistical calibration method to recover the parameters of our credit risk model from market price of equity and balance sheet indicators. We apply the method to the Parmalat case, a real case of misreporting and show that the model is able to successfully isolate the misreporting component. We also provide empirical evidence that the term structure of credit default swaps quotes exhibits special patterns in cases of misreporting by using three well known cases of accounting irregularities in US history: Tyco, Enron, and WorldCom.</p>
<p>We conclude the thesis with a study of bilateral credit risk, which accommodates the case in which both parties of the financial contract may default on their payments. We introduce the general arbitrage-free valuation framework for counterparty risk adjustments in presence of bilateral default risk. We illustrate the symmetry in the valuation and show that the adjustment involves a long position in a put option plus a short position in a call option, both with zero strike and written on the residual net value of the contract at the relevant default times. We allow for correlation between the default times of each party of the contract and the underlying portfolio risk factors. We introduce stochastic intensity models and a trivariate copula function on the default times exponential variables to model default dependence. We provide evidence that both default correlation and credit spread volatilities have a relevant and structured impact on the adjustment. We also study a case involving British Airways, Lehman Brothers, and Royal Dutch Shell, illustrating the bilateral adjustments in concrete crisis situations.</p>
https://thesis.library.caltech.edu/id/eprint/2178Dynamic Simulation and Control of Articulated Limbs
https://resolver.caltech.edu/CaltechETD:etd-02162009-054828
Authors: {'items': [{'email': 'marcus@post.harvard.edu', 'id': 'Mitchell-Marcus-Quintana', 'name': {'family': 'Mitchell', 'given': 'Marcus Quintana'}, 'show_email': 'NO'}]}
Year: 2009
DOI: 10.7907/2FAM-6A26
Many useful mechanisms can be modelled as articulated systems: collections of rigid bodies linked together with joints that constrain relative movement. The two parts of this thesis study the complementary problems of simulation and control for such systems. In the first part, we describe an implementation and extension of a physically based modelling framework known as "dynamic constraints" in which forces of constraint linking bodies in an articulated system are explicitly calculated. In addition to identifying some important robustness and stability issues for these calculations, we extend the framework to systems whose internal degrees of freedom can be directly parameterized. This permits significant efficiency gains for mechanisms which model limbs. The second part of the thesis centers on the adaptive control of limb configuration through simulated actuators. In this problem, the nonlinear structure and parametric details of a limb are assumed to be unknown. We present and illustrate the performance of an adaptive scheme which performs considerably better than conventional nonadaptive techniques, and which is competitive with adaptive methods which use more a priori knowledge of limb dynamics. https://thesis.library.caltech.edu/id/eprint/647Signal Processing Algorithms for MIMO Radar
https://resolver.caltech.edu/CaltechETD:etd-06082009-131045
Authors: {'items': [{'email': 'cy.scott.chen@gmail.com', 'id': 'Chen-Scott-Chun-Yang', 'name': {'family': 'Chen', 'given': 'Scott Chun-Yang'}, 'show_email': 'NO'}]}
Year: 2009
DOI: 10.7907/TPT1-9V58
<p>Radar is a system that uses electromagnetic waves to detect, locate and measure the speed of reflecting objects such as aircraft, ships, spacecraft, vehicles, people, weather formations, and terrain. It transmits the electromagnetic waves into space and receives the echo signal reflected from objects. By applying signal processing algorithms on the reflected waveform, the reflecting objects can be detected. Furthermore, the location and the speed of the objects can also be estimated. Radar was originally an acronym for "RAdio Detection And Ranging". Today radar has become a standard English noun. Early radar development was mostly driven by military and military is still the dominant user and developer of radar technology. Military applications include surveillance, navigation, and weapon guidance. However, radar now has a broader range of applications including meteorological detection of precipitation, measuring ocean surface waves, air traffic control, police detection of speeding traffic, sports radar speed guns, and preventing car or ship collisions.</p>
<p>Recently, the concept of MIMO radar has been proposed. The MIMO radar is a multiple antenna radar system which is capable of transmitting arbitrary waveform from each antenna element. In the traditional phased array radar, the transmitting antennas are limited to transmit scaled versions of the same waveform. However the MIMO radar allows the multiple antennas to transmit arbitrary waveforms. Like MIMO communications, MIMO radar offers a new paradigm for signal processing research. MIMO radar possesses significant potentials for fading mitigation, resolution enhancement, and interference and jamming suppression. Fully exploiting these potentials can result in significantly improved target detection, parameter estimation, target tracking and recognition performance. The MIMO radar technology has rapidly drawn considerable attention from many researchers. Several advantages of MIMO radar have been discovered by many different researchers such as increased diversity of the target information, excellent interference rejection capability, improved parameter identifiability, and enhanced flexibility for transmit beampattern design. The degrees of freedom introduced by MIMO radar improves the performance of the radar systems in many different aspects. However, it also generates some issues. It increases the number of dimensions of the received signals. Consequently, this increases the complexity of the receiver. Furthermore, the MIMO radar transmits an incoherent waveform on each of the transmitting antennas. This in general reduces the processing gain compared to the phased array radar. The multiple arbitrary waveforms also affects the range and Doppler resolution of the radar system.</p>
<p>The main contribution of this thesis is to study the signal processing issues in MIMO radar and propose novel algorithms for improving the MIMO radar system. In the first part of this thesis, we focus on the MIMO radar receiver algorithms. We first study the robustness of the beamformer used in MIMO radar receiver. It is known that the adaptive beamformer is very sensitive to the DOA (direction-of-arrival) mismatch. In MIMO radar, the aperture of the virtual array can be much larger than the physical receiving array in the SIMO radar. This makes the performance of the beamformer more sensitive to the DOA errors in the MIMO radar case. In this thesis, we propose an adaptive beamformer that is robust against the DOA mismatch. This method imposes constraints such that the magnitude responses of two angles exceed unity. Then a diagonal loading method is used to force the magnitude responses at the arrival angles between these two angles to exceed unity. Therefore the proposed method can always force the gains at a desired interval of angles to exceed a constant level while suppressing the interferences and noise. A closed form solution to the proposed minimization problem is introduced, and the diagonal loading factor can be computed systematically by a proposed algorithm. Numerical examples show that this method has an excellent SINR (signal to noise-plus-interference ratio) performance and a complexity comparable to the standard adaptive beamformer. We also study the space-time adaptive processing (STAP) for MIMO radar systems. With a slight modification, STAP methods developed originally for the single-input multiple-output (SIMO) radar (phased array radar) can also be used in MIMO radar. However, in the MIMO radar, the rank of the jammer-and-clutter subspace becomes very large, especially the jammer subspace. It affects both the complexity and the convergence of the STAP algorithm. In this thesis, we explore the clutter space and its rank in the MIMO radar. By using the geometry of the problem rather than data, the clutter subspace can be represented using prolate spheroidal wave functions (PSWF). Using this representation, a new STAP algorithm is developed. It computes the clutter space using the PSWF and utilizes the block diagonal property of the jammer covariance matrix. Because of fully utilizing the geometry and the structure of the covariance matrix, the method has very good SINR performance and low computational complexity.</p>
<p>The second half of the thesis focuses on the transmitted waveform design for MIMO radar systems. We first study the ambiguity function of the MIMO radar and the corresponding waveform design methods. In traditional (SIMO) radars, the ambiguity function of the transmitted pulse characterizes the compromise between range and Doppler resolutions. It is a major tool for studying and analyzing radar signals. The idea of ambiguity function has recently been extended to the case of MIMO radar. In this thesis, we derive several mathematical properties of the MIMO radar ambiguity function. These properties provide some insights into the MIMO radar waveform design. We also propose a new algorithm for designing the orthogonal frequency-hopping waveforms. This algorithm reduces the sidelobes in the corresponding MIMO radar ambiguity function and makes the energy of the ambiguity function spread evenly in the range and angular dimensions. Therefore the resolution of the MIMO radar system can be improved. In addition to designing the waveform for increasing the system resolution, we also consider the joint optimization of waveforms and receiving filters in the MIMO radar for the case of extended target in clutter. An extended target can be viewed as a collection of infinite number of point targets. The reflected waveform from a point target is just a delayed and scaled version of the transmitted waveform. However, the reflected waveform from an extended target is a convolved version of the transmitted waveform with a target spreading function. A novel iterative algorithm is proposed to optimize the waveforms and receiving filters such that the detection performance can be maximized. The corresponding iterative algorithms are also developed for the case where only the statistics or the uncertainty set of the target impulse response is available. These algorithms guarantee that the SINR performance improves in each iteration step. The numerical results show that the proposed iterative algorithms converge faster and also have significant better SINR performances than previously reported algorithms.</p>
https://thesis.library.caltech.edu/id/eprint/2521Neuro-Evolution Using Recombinational Algorithms and Embryogenesis for Robotic Control
https://resolver.caltech.edu/CaltechTHESIS:06092010-140839602
Authors: {'items': [{'email': 'tonyroy46@yahoo.com', 'id': 'Roy-Anthony-Mathew', 'name': {'family': 'Roy', 'given': 'Anthony Mathew'}, 'show_email': 'NO'}]}
Year: 2010
DOI: 10.7907/YNED-VN66
Control tasks involving dramatic nonlinearities, such as decision making, can be challenging for classical design methods. However, autonomous, stochastic design methods such as evolutionary computation have proved effective. In particular, genetic algorithms that create designs via the application of recombinational rules are robust and highly scalable. Neuro-Evolution Using Recombinational Algorithms and Embryogenesis (NEURAE) is a genetic algorithm that creates C++ programs that in turn create neural networks which can function as logic gates. The neural networks created are scalable and robust enough to feature redundancies that allow the network to function despite internal failures. An analysis of NEURAE evinces how biologically inspired phenomena apply to simulated evolution. This allows for an optimization of NEURAE that enables it to create controllers for a simulated swarm of Khepera-inspired robots.https://thesis.library.caltech.edu/id/eprint/5944Modeling and Predicting Object Attention in Natural Scenes
https://resolver.caltech.edu/CaltechTHESIS:05262011-172742472
Authors: {'items': [{'email': 'spain@vision.caltech.edu', 'id': 'Spain-Merrielle-Therese', 'name': {'family': 'Spain', 'given': 'Merrielle Therese'}, 'show_email': 'NO'}]}
Year: 2011
DOI: 10.7907/JTEE-7367
<p>Humans automatically attend to certain objects in a scene. Better understanding this process could improve a computer's ability to parse scene images and convey information about them to humans. This thesis is arranged in three parts. The first part explores how important a particular object is in a photograph of a complex scene. We propose a definition of importance and present two methods for measuring object importance from human observers. Using this ground truth, we fit a function for predicting the importance of each object directly from a segmented image; our function combines many object-related and image-related features. We validate our importance predictions on a large set of objects and find that the most important objects may be identified automatically. We find that object position and size are particularly informative, while a popular measure of saliency is not.</p>
<p>The second part explores the relationship between object naming, eye movements, and saliency maps. Eye movements correlate with shifts in attention and are thought to be a consequence of optimal resource allocation for high-level tasks such as visual recognition. Saliency maps, are often built on the assumption that "early" features (e.g., color, contrast, orientation, and motion) as opposed to objects themselves drive attention. We measure the eye position of humans viewing scenes and then ask them to recall objects that they saw in each scene. Weighted with recall frequency or maximum saliency, these objects predict fixations in individual images better than early saliency, suggesting that early saliency may have an indirect effect on attention, acting through detected objects.</p>
<p>The third part explores the problem of locating objects in a scene irrespective of category. We introduce the first benchmark for category-independent object detection. It is composed of a large public dataset of annotated high-resolution scene images and suitable metrics for performance evaluation. We demonstrate our benchmark by comparing three methods for generalized object detection against a baseline and an upper bound.</p>https://thesis.library.caltech.edu/id/eprint/6459The Roles of Majorization and Generalized Triangular Decomposition in Communication and Signal Processing
https://resolver.caltech.edu/CaltechTHESIS:06032011-113200456
Authors: {'items': [{'email': 'ccweng@ntuee.tw', 'id': 'Weng-Ching-Chih', 'name': {'family': 'Weng', 'given': 'Ching-Chih'}, 'show_email': 'NO'}]}
Year: 2011
DOI: 10.7907/2R1B-QE65
<p>Signal processing is an art that deals with the representation, transformation, and manipulation of the signals and the information they contain based on their specific features. The field of signal processing has always benefited from the interaction between theory, applications, and technologies for implementing the systems. The development of signal processing theory, in particular, relies heavily on mathematical tools including analysis, probability theory, matrix theory, and many others.</p>
<p>Recently, the theory of majorization, which is an extremely useful tool for deriving inequalities, was introduced to the signal processing society in the context of MIMO communication system design. This also led many researchers to develop a fundamental matrix decomposition called generalized triangular decomposition (GTD), which was general enough to include many existing matrix orthogonal decompositions as special cases.</p>
<p>The main contribution of this thesis is toward the use of majorization and GTD to the theory and many applications of signal processing. In particular, the focus is on developing new signal processing methods based on these mathematical tools for digital communication, data compression, and filter bank design. We revisit some classical problems and show that the theories of majorization and GTD provide a general framework for solving these problems. For many important new problems not solved earlier, they also provide elegant solutions.</p>
<p>The first part of the thesis focuses on transceiver design for multiple-input multiple-output (MIMO) communications. The first problem considered is the joint optimization of transceivers with linear precoders, decision feedback equalizers (DFEs), and bit allocation schemes for frequency flat MIMO channels. We show that the generalized triangular decomposition offers an optimal family of solutions to this problem. This general framework incorporates many existing designs, such as the optimal linear transceiver, the ZF-VBLAST system, and the geometric mean decomposition (GMD) transceiver, as special cases. It also predicts many novel optimal solutions that have not been observed before. We also discuss the use of each of these theoretical solutions under practical considerations. In addition to total power constraints, we also consider the transceiver optimization under individual power constraints and other linear constraints on the transmitting covariance matrix, which includes a more realistic individual power constraint on each antenna. We show the use of semi-definite programming (SDP), and the theory of majorization again provides a general framework for optimizing the linear transceivers as well as the DFE transceivers. The transceiver design for frequency selective MIMO channels is then considered. Block diagonal GMD (BD-GMD), which is a special instance of GTD with block diagonal structure in one of the semi-unitary matrices, is used to design transceivers that have many desirable properties in both performance and computation.</p>
<p>The second part of the thesis focuses on signal processing algorithms for data compressions and filter bank designs. We revisit the classical transform coding problem (for optimizing the theoretical coding gain in the high bit rate regime) from the view point of GTD and majorization theory. A general family of optimal transform coders is introduced based on GTD. This family includes the Karhunen-Lo\'{e}ve transform (KLT), and the prediction-based lower triangular transform (PLT) as special cases. The coding gain of the entire family, with optimal bit allocation, is maximized and equal to those of the KLT and the PLT. Other special cases of the GTD-TC are the GMD (geometric mean decomposition) and the BID (bidiagonal transform). The GMD in particular has the property that the optimum bit allocation is a uniform allocation. We also propose using dither quantization in the GMD transform coder. Under the uniform bit loading scheme, it is shown that the proposed dithered GMD transform coders perform significantly better than the original GMD coder in the low rate regime.</p>
<p>Another important signal processing problem, namely the filter bank optimization based on the knowledge of input signal statistics, is then considered. GTD and the theory of majorization are again used to give a new look to this classical problem. We propose GTD filter banks as subband coders for optimizing the theoretical coding gain. The orthonormal GTD filter bank and the biorthogonal GTD filter bank are discussed in detail. We show that in both cases there are two fundamental properties in the optimal solutions, namely, {\it total decorrelation} and {\it spectrum equalization}. The optimal solutions can be obtained by performing the frequency dependent GTD on the Cholesky factor of the input power spectrum density matrices. We also show that in both theory and numerical simulations, the optimal GTD subband coders have superior performance than optimal traditional subband coders. In addition, the uniform bit loading scheme can be used in the optimal biorthogonal GTD coders with no loss of optimality. This solves the granularity problem in the conventional optimum bit loading formula. The use of the GTD filter banks in frequency selective MIMO communication systems is also discussed. Finally, the connection between the GTD filter bank and the traditional filter bank is clearly indicated.</p>
https://thesis.library.caltech.edu/id/eprint/6496Searching Large-Scale Image Collections
https://resolver.caltech.edu/CaltechTHESIS:04252011-145432540
Authors: {'items': [{'email': 'mohamedadaly@gmail.com', 'id': 'Aly-Mohamed-Alaa-El-Dien-Mahmoud-Hussein', 'name': {'family': 'Aly', 'given': 'Mohamed Alaa El-Dien Mahmoud Hussein'}, 'show_email': 'NO'}]}
Year: 2011
DOI: 10.7907/VRGJ-4J54
Searching quickly and accurately in a large collection of images has become an increasingly important problem. The ultimate goal is to make visual search possible: allow users to search using images in addition to typing text. The typical approach is to index all the images of interest (e.g., images of landmarks, books, or DVDs) in a database and let users question the system with query images. Such a database can reach billions of images, and this poses challenges in terms of memory and computational requirements and recognition performance. In this work we provide an in depth study of systems used for searching large-scale image collections.
Specifically, we provide a thorough comparison of the two leading image search approaches: Full Representation (FR) vs. Bag of Words (BoW). We derive theoretical estimates of how the memory and computational cost scale with the number of images in the database, and empirically evaluate the performance and run time on four real-world datasets. Our experiments suggest that FR provides better recognition performance than BoW, though it requires more memory. Therefore, we address these shortcomings by presenting novel methods that increase the recognition performance of BoW and decrease the memory requirements of FR. Finally, we present a novel way to parallelize FR on multiple machines and scale up database sizes to 100 million images with interactive run time.https://thesis.library.caltech.edu/id/eprint/6353Towards Open Ended Learning: Budgets, Model Selection, and Representation
https://resolver.caltech.edu/CaltechTHESIS:02092011-171146758
Authors: {'items': [{'email': 'gomes@vision.caltech.edu', 'id': 'Gomes-Ryan-Geoffrey', 'name': {'family': 'Gomes', 'given': 'Ryan Geoffrey'}, 'show_email': 'NO'}]}
Year: 2011
DOI: 10.7907/T92X-DQ05
<p>Biological organisms learn to recognize visual categories continuously over the course of their lifetimes. This impressive capability allows them to adapt to new circumstances as they arise, and to flexibly incorporate new object categories as they are discovered. Inspired by this capability, we seek to create artificial recognition systems that can learn in a similar fashion.</p>
<p>We identify a number of characteristics that define this Open Ended learning capability. Open Ended learning is unsupervised: object instances need not be explicitly labeled with a category indicator during training. Learning occurs incrementally as experience ensues; there is no training period that is distinct from operation and the categorization system must operate and update itself in a timely fashion with limited computational resources. Open Ended learning systems must flexibly adapt the number of categories as new evidence is uncovered.</p>
<p>Having identified these requirements, we develop Open Ended categorization systems based on probabilistic graphical models and study their properties. From the perspective of building practical systems, the most challenging requirement of Open Ended learning is that it must be carried out in an unsupervised fashion. We then study the question of how best to represent data items and categories in unsupervised learning algorithms in order to extend their domain of application.</p>
<p>Finally, we conclude that continuously learning categorization systems are likely to require human intervention and supervision for some time to come, which suggests research in how best to structure machine-human interactions. We end this thesis by studying a system that reverses the typical role of human and machine in most learning systems. In Crowd Clustering, humans perform the fundamental image categorization tasks, and the machine learning system evaluates and aggregates the results of human workers.</p> https://thesis.library.caltech.edu/id/eprint/6241Unsupervised Learning of Categorical Segments in Image Collections
https://resolver.caltech.edu/CaltechTHESIS:04262011-213152111
Authors: {'items': [{'email': 'marco@vision.caltech.edu', 'id': 'Andreetto-Marco', 'name': {'family': 'Andreetto', 'given': 'Marco'}, 'show_email': 'NO'}]}
Year: 2011
DOI: 10.7907/ZH04-VT55
Which one comes first: segmentation or recognition? We propose a unified framework for carrying out the two simultaneously and without supervision. The framework combines a flexible probabilistic model for representing the shape and appearance of each segment, with the popular "bag of visual words" model for recognition. If applied to a collection of images, our framework can simultaneously discover the segments of each image, and the correspondence between such segments, without supervision. Such recurring segments may be thought of as the "parts" of corresponding objects that appear multiple times in the image collection. Thus, the model may be used for learning new categories, detecting/classifying objects, and segmenting images, without using expensive human annotation.https://thesis.library.caltech.edu/id/eprint/6355Network Structure Optimization with Applications to Minimizing Variance and Crosstalk
https://resolver.caltech.edu/CaltechTHESIS:12102011-161913831
Authors: {'items': [{'email': 'dionysios.barmpoutis@gmail.com', 'id': 'Barmpoutis-Dionysios', 'name': {'family': 'Barmpoutis', 'given': 'Dionysios'}, 'show_email': 'NO'}]}
Year: 2012
DOI: 10.7907/ER8Y-ZK49
This thesis provides a unified methodology for analyzing structural properties of graphs, along with their applications. In the last several years, the field of complex networks has been extensively studied, and it is now well understood that the way a large network is built is closely intertwined with its function. Structural properties have an impact on the function of the network, and the form of many systems has been evolved in order to optimize for given functions. Despite the great progress, particularly in how structural attributes affect the various network functions, there is a significant gap in the quantitative study of how much these properties can change in a network without a significant impact on the functionality of the system, or what the bounds of these structural attributes are. Here, we find and analytically prove tight bounds of global graph properties, as well as the form of the graphs that achieve these bounds. The attributes studied include the network efficiency, radius, diameter, average distance, betweenness centrality, resistance distance, and average clustering. All of these qualities have a direct impact on the function of the network, and finding the graph that optimizes one or more of them is of interest when designing a large system. In addition, we measure how sensitive these properties are with respect to random rewirings or addition of new edges, since designing a network with a given set of constraints may include a lot of trade-offs. This thesis also studies properties that are of interest in both natural and engineered networks, such as maximum immunity to crosstalk interactions and random noise. We are primarily focused on networks where information is transmitted through a means that is accessible by all the individual units of the network and the interactions among the different entities that comprise it do not necessarily have a dedicated mechanism that facilitates information transmission, or isolates them from other parts of the network. Two examples of this class are biological and chemical reaction networks. Such networks suffer from unwanted crosstalk interactions when two or more units spuriously interact with each other. In addition, they are subject to random fluctuations in their output, both due to noisy inputs and because of the random variance of their parameters. These two types of randomness affect the behavior of the system in ways that are intrinsically different. We examine the network topologies that accentuate or alleviate the effect of random variance in the network for both directed and undirected graphs, and find that increasing the crosstalk among different parts reduces the output variance but also contributes to a slower response.
https://thesis.library.caltech.edu/id/eprint/6749Learning and Using Taxonomies for Visual and Olfactory Classification
https://resolver.caltech.edu/CaltechTHESIS:05162013-152639568
Authors: {'items': [{'email': 'gsgriffin@gmail.com', 'id': 'Griffin-Gregory-Scott', 'name': {'family': 'Griffin', 'given': 'Gregory Scott'}, 'show_email': 'YES'}]}
Year: 2013
DOI: 10.7907/YTZH-HA75
Humans are able of distinguishing more than 5000 visual categories even in complex environments using a variety of different visual systems all working in tandem. We seem to be capable of distinguishing thousands of different odors as well. In the machine learning community, many commonly used multi-class classifiers do not scale well to such large numbers of categories. This thesis demonstrates a method of automatically creating application-specific taxonomies to aid in scaling classification algorithms to more than 100 cate- gories using both visual and olfactory data. The visual data consists of images collected online and pollen slides scanned under a microscope. The olfactory data was acquired by constructing a small portable sniffing apparatus which draws air over 10 carbon black polymer composite sensors. We investigate performance when classifying 256 visual categories, 8 or more species of pollen and 130 olfactory categories sampled from common household items and a standardized scratch-and-sniff test. Taxonomies are employed in a divide-and-conquer classification framework which improves classification time while allowing the end user to trade performance for specificity as needed. Before classification can even take place, the pollen counter and electronic nose must filter out a high volume of background “clutter” to detect the categories of interest. In the case of pollen this is done with an efficient cascade of classifiers that rule out most non-pollen before invoking slower multi-class classifiers. In the case of the electronic nose, much of the extraneous noise encountered in outdoor environments can be filtered using a sniffing strategy which preferentially samples the visensor response at frequencies that are relatively immune to background contributions from ambient water vapor. This combination of efficient background rejection with scalable classification algorithms is tested in detail for three separate projects: 1) the Caltech-256 Image Dataset, 2) the Caltech Automated Pollen Identification and Counting System (CAPICS) and 3) a portable electronic nose specially constructed for outdoor use.https://thesis.library.caltech.edu/id/eprint/7718Bootstrapping Vehicles: A Formal Approach to Unsupervised Sensorimotor Learning Based on Invariance
https://resolver.caltech.edu/CaltechTHESIS:10282012-082208075
Authors: {'items': [{'email': 'andrea@censi.org', 'id': 'Censi-Andrea', 'name': {'family': 'Censi', 'given': 'Andrea'}, 'show_email': 'NO'}]}
Year: 2013
DOI: 10.7907/PWVS-2Q74
Could a "brain in a jar" be able to control an unknown robotic body to which it is connected, and use it to achieve useful tasks, without any prior assumptions on the body's sensors and actuators? Other than of purely intellectual interest, this question is relevant to the medium-term challenges of robotics: as the complexity of robotics applications grows, automated learning techniques might reduce design effort and increase the robustness and reliability of the solutions. In this work, the problem of "bootstrapping" is studied in the context of the Vehicles universe, which is an idealization of simple mobile robots, after the work of Braitenberg. The first thread of results consists in analyzing such simple sensorimotor cascades and proposing models of varying complexity that can be learned from data. The second thread regards how to properly formalize the notions of "absence of assumptions", as a particular form of invariance that the bootstrapping agent must satisfy, and proposes some invariance-based design techniques.https://thesis.library.caltech.edu/id/eprint/7248Convex Analysis for Minimizing and Learning Submodular Set Functions
https://resolver.caltech.edu/CaltechTHESIS:05312013-151014984
Authors: {'items': [{'email': 'peterstobbe@gmail.com', 'id': 'Stobbe-Peter', 'name': {'family': 'Stobbe', 'given': 'Peter'}, 'show_email': 'NO'}]}
Year: 2013
DOI: 10.7907/1A1J-SA64
<p>The connections between convexity and submodularity are explored, for purposes of minimizing and learning submodular set functions.</p>
<p>First, we develop a novel method for minimizing a particular class of submodular functions, which can be expressed as a sum of concave functions composed with modular functions. The basic algorithm uses an accelerated first order method applied to a smoothed version of its convex extension. The smoothing algorithm is particularly novel as it allows us to treat general concave potentials without needing to construct a piecewise linear approximation as with graph-based techniques.</p>
<p>Second, we derive the general conditions under which it is possible to find a minimizer of a submodular function via a convex problem. This provides a framework for developing submodular minimization algorithms. The framework is then used to develop several algorithms that can be run in a distributed fashion. This is particularly useful for applications where the submodular objective function consists of a sum of many terms, each term dependent on a small part of a large data set.</p>
<p>Lastly, we approach the problem of learning set functions from an unorthodox perspective---sparse reconstruction. We demonstrate an explicit connection between the problem of learning set functions from random evaluations and that of sparse signals. Based on the observation that the Fourier transform for set functions satisfies exactly the conditions needed for sparse reconstruction algorithms to work, we examine some different function classes under which uniform reconstruction is possible.</p>
https://thesis.library.caltech.edu/id/eprint/7798New Directions In Sparse Sampling and Estimation For Underdetermined Systems
https://resolver.caltech.edu/CaltechTHESIS:06072013-153438961
Authors: {'items': [{'email': 'breakfast.at.flurys@gmail.com', 'id': 'Pal-Piya', 'name': {'family': 'Pal', 'given': 'Piya'}, 'show_email': 'YES'}]}
Year: 2013
DOI: 10.7907/P0E1-5G05
<p>A central objective in signal processing is to infer meaningful information from a set of measurements or data. While most signal models have an overdetermined structure (the number of unknowns less than the number of equations), traditionally very few statistical estimation problems have considered a data model which is underdetermined (number of unknowns more than the number of equations). However, in recent times, an explosion of theoretical and computational methods have been developed primarily to study underdetermined systems by imposing sparsity on the unknown variables. This is motivated by the observation that inspite of the huge volume of data that arises in sensor networks, genomics, imaging, particle physics, web search etc., their information content is often much smaller compared to the number of raw measurements. This has given rise to the possibility of reducing the number of measurements by down sampling the data, which automatically gives rise to underdetermined systems.</p>
<p>In this thesis, we provide new directions for estimation in an underdetermined system, both for a class of parameter estimation problems and also for the problem of sparse recovery in compressive sensing. There are two main contributions of the thesis: design of new sampling and statistical estimation algorithms for array processing, and development of improved guarantees for sparse reconstruction by introducing a statistical framework to the recovery problem.</p>
<p>We consider underdetermined observation models in array processing where the number of unknown sources simultaneously received by the array can be considerably larger than the number of physical sensors. We study new sparse spatial sampling schemes (array geometries) as well as propose new recovery algorithms that can exploit priors on the unknown signals and unambiguously identify all the sources. The proposed sampling structure is generic enough to be extended to multiple dimensions as well as to exploit different kinds of priors in the model such as correlation, higher order moments, etc.</p>
<p>Recognizing the role of correlation priors and suitable sampling schemes for underdetermined estimation in array processing, we introduce a correlation aware framework for recovering sparse support in compressive sensing. We show that it is possible to strictly increase the size of the recoverable sparse support using this framework provided the measurement matrix is suitably designed. The proposed nested and coprime arrays are shown to be appropriate candidates in this regard. We also provide new guarantees for convex and greedy formulations of the support recovery problem and demonstrate that it is possible to strictly improve upon existing guarantees.</p>
<p>This new paradigm of underdetermined estimation that explicitly establishes the fundamental interplay between sampling, statistical priors and the underlying sparsity, leads to exciting future research directions in a variety of application areas, and also gives rise to new questions that can lead to stand-alone theoretical results in their own right.</p>
https://thesis.library.caltech.edu/id/eprint/7870Transceiver Designs and Analysis for LTI, LTV and Broadcast Channels - New Matrix Decompositions and Majorization Theory
https://resolver.caltech.edu/CaltechTHESIS:05282013-115316389
Authors: {'items': [{'email': 'quepanda@gmail.com', 'id': 'Liu-Chih-Hao', 'name': {'family': 'Liu', 'given': 'Chih-Hao'}, 'show_email': 'NO'}]}
Year: 2013
DOI: 10.7907/2VFF-SZ70
<p>Signal processing techniques play important roles in the design of digital communication systems. These include information manipulation, transmitter signal processing, channel estimation, channel equalization and receiver signal processing. By interacting with communication theory and system implementing technologies, signal processing specialists develop efficient schemes for various communication problems by wisely exploiting various mathematical tools such as analysis, probability theory, matrix theory, optimization theory, and many others. In recent years, researchers realized that multiple-input multiple-output (MIMO) channel models are applicable to a wide range of different physical communications channels. Using the elegant matrix-vector notations, many MIMO transceiver (including the precoder and equalizer) design problems can be solved by matrix and optimization theory. Furthermore, the researchers showed that the majorization theory and matrix decompositions, such as singular value decomposition (SVD), geometric mean decomposition (GMD) and generalized triangular decomposition (GTD), provide unified frameworks for solving many of the point-to-point MIMO transceiver design problems.</p>
<p>In this thesis, we consider the transceiver design problems for linear time invariant (LTI) flat MIMO channels, linear time-varying narrowband MIMO channels, flat MIMO broadcast channels, and doubly selective scalar channels. Additionally, the channel estimation problem is also considered. The main contributions of this dissertation are the development of new matrix decompositions, and the uses of the matrix decompositions and majorization theory toward the practical transmit-receive scheme designs for transceiver optimization problems. Elegant solutions are obtained, novel transceiver structures are developed, ingenious algorithms are proposed, and performance analyses are derived.</p>
<p>The first part of the thesis focuses on transceiver design with LTI flat MIMO channels. We propose a novel matrix decomposition which decomposes a complex matrix as a product of several sets of semi-unitary matrices and upper triangular matrices in an iterative manner. The complexity of the new decomposition, generalized geometric mean decomposition (GGMD), is always less than or equal to that of geometric mean decomposition (GMD). The optimal GGMD parameters which yield the minimal complexity are derived. Based on the channel state information (CSI) at both the transmitter (CSIT) and receiver (CSIR), GGMD is used to design a butterfly structured decision feedback equalizer (DFE) MIMO transceiver which achieves the minimum average mean square error (MSE) under the total transmit power constraint. A novel iterative receiving detection algorithm for the specific receiver is also proposed. For the application to cyclic prefix (CP) systems in which the SVD of the equivalent channel matrix can be easily computed, the proposed GGMD transceiver has K/log_2(K) times complexity advantage over the GMD transceiver, where K is the number of data symbols per data block and is a power of 2. The performance analysis shows that the GGMD DFE transceiver can convert a MIMO channel into a set of parallel subchannels with the same bias and signal to interference plus noise ratios (SINRs). Hence, the average bit rate error (BER) is automatically minimized without the need for bit allocation. Moreover, the proposed transceiver can achieve the channel capacity simply by applying independent scalar Gaussian codes of the same rate at subchannels.</p>
<p>In the second part of the thesis, we focus on MIMO transceiver design for slowly time-varying MIMO channels with zero-forcing or MMSE criterion. Even though the GGMD/GMD DFE transceivers work for slowly time-varying MIMO channels by exploiting the instantaneous CSI at both ends, their performance is by no means optimal since the temporal diversity of the time-varying channels is not exploited. Based on the GTD, we develop space-time GTD (ST-GTD) for the decomposition of linear time-varying flat MIMO channels.
Under the assumption that CSIT, CSIR and channel prediction are available, by using the proposed ST-GTD, we develop space-time geometric mean decomposition (ST-GMD) DFE transceivers under the zero-forcing or MMSE criterion. Under perfect channel prediction, the new system minimizes both the average MSE at the detector in each space-time (ST) block (which consists of several coherence blocks), and the average per ST-block BER in the moderate high SNR region. Moreover, the ST-GMD DFE transceiver designed under an MMSE criterion maximizes Gaussian mutual information over the equivalent channel seen by each ST-block. In general, the newly proposed transceivers perform better than the GGMD-based systems since the super-imposed temporal precoder is able to exploit the temporal diversity of time-varying channels. For practical applications, a novel ST-GTD based system which does not require channel prediction but shares the same asymptotic BER performance with the ST-GMD DFE transceiver is also proposed.</p>
<p>The third part of the thesis considers two quality of service (QoS) transceiver design problems for flat MIMO broadcast channels. The first one is the power minimization problem (min-power) with a total bitrate constraint and per-stream BER constraints. The second problem is the rate maximization problem (max-rate) with a total transmit power constraint and per-stream BER constraints. Exploiting a particular class of joint triangularization (JT), we are able to jointly optimize the bit allocation and the broadcast DFE transceiver for the min-power and max-rate problems. The resulting optimal designs are called the minimum power JT broadcast DFE transceiver (MPJT) and maximum rate JT broadcast DFE transceiver (MRJT), respectively. In addition to the optimal designs, two suboptimal designs based on QR decomposition are proposed. They are realizable for arbitrary number of users.</p>
<p>Finally, we investigate the design of a discrete Fourier transform (DFT) modulated filterbank transceiver (DFT-FBT) with LTV scalar channels. For both cases with known LTV channels and unknown wide sense stationary uncorrelated scattering (WSSUS) statistical channels, we show how to optimize the transmitting and receiving prototypes of a DFT-FBT such that the SINR at the receiver is maximized. Also, a novel pilot-aided subspace channel estimation algorithm is proposed for the orthogonal frequency division multiplexing (OFDM) systems with quasi-stationary multi-path Rayleigh fading channels. Using the concept of a difference co-array, the new technique can construct M^2 co-pilots from M physical pilot tones with alternating pilot placement. Subspace methods, such as MUSIC and ESPRIT, can be used to estimate the multipath delays and the number of identifiable paths is up to O(M^2), theoretically. With the delay information, a MMSE estimator for frequency response is derived. It is shown through simulations that the proposed method outperforms the conventional subspace channel estimator when the number of multipaths is greater than or equal to the number of physical pilots minus one.</p>
https://thesis.library.caltech.edu/id/eprint/7758Development of Integrated Parylene Fluidic Devices for Use as a Microbial Monitoring System in Wastewater Treatment
https://resolver.caltech.edu/CaltechTHESIS:05312014-143352981
Authors: {'items': [{'email': 'penviphas@gmail.com', 'id': 'Satsanarukkit-Penvipha', 'name': {'family': 'Satsanarukkit', 'given': 'Penvipha'}, 'show_email': 'NO'}]}
Year: 2014
DOI: 10.7907/Z98G8HPQ
[Abstract Embargoed]https://thesis.library.caltech.edu/id/eprint/8456Community Sense and Response Systems
https://resolver.caltech.edu/CaltechTHESIS:04152014-111007328
Authors: {'items': [{'email': 'mnfaulk@gmail.com', 'id': 'Faulkner-Matthew-Nicholas', 'name': {'family': 'Faulkner', 'given': 'Matthew Nicholas'}, 'show_email': 'NO'}]}
Year: 2014
DOI: 10.7907/QFM5-FH06
<p>The proliferation of smartphones and other internet-enabled, sensor-equipped consumer devices enables us to sense and act upon the physical environment in unprecedented ways. This thesis considers Community Sense-and-Response (CSR) systems, a new class of web application for acting on sensory data gathered from participants' personal smart devices. The thesis describes how rare events can be reliably detected using a decentralized anomaly detection architecture that performs client-side anomaly detection and server-side event detection. After analyzing this decentralized anomaly detection approach, the thesis describes how weak but spatially structured events can be detected, despite significant noise, when the events have a sparse representation in an alternative basis. Finally, the thesis describes how the statistical models needed for client-side anomaly detection may be learned efficiently, using limited space, via coresets.</p>
<p>The Caltech Community Seismic Network (CSN) is a prototypical example of a CSR system that harnesses accelerometers in volunteers' smartphones and consumer electronics. Using CSN, this thesis presents the systems and algorithmic techniques to design, build and evaluate a scalable network for real-time awareness of spatial phenomena such as dangerous earthquakes.</p>https://thesis.library.caltech.edu/id/eprint/8188Optimal Data Distributions in Machine Learning
https://resolver.caltech.edu/CaltechTHESIS:05262015-094933189
Authors: {'items': [{'email': 'carlos.r.gonzalez@gmail.com', 'id': 'González-Palacios-Carlos-Roberto', 'name': {'family': 'González Palacios', 'given': 'Carlos Roberto'}, 'show_email': 'YES'}]}
Year: 2015
DOI: 10.7907/Z9DR2SD5
<p>In the first part of the thesis we explore three fundamental questions that arise naturally when we conceive a machine learning scenario where the training and test distributions can differ. Contrary to conventional wisdom, we show that in fact mismatched training and test distribution can yield better out-of-sample performance. This optimal performance can be obtained by training with the dual distribution. This optimal training distribution depends on the test distribution set by the problem, but not on the target function that we want to learn. We show how to obtain this distribution in both discrete and continuous input spaces, as well as how to approximate it in a practical scenario. Benefits of using this distribution are exemplified in both synthetic and real data sets.</p>
<p>In order to apply the dual distribution in the supervised learning scenario where the training data set is fixed, it is necessary to use weights to make the sample appear as if it came from the dual distribution. We explore the negative effect that weighting a sample can have. The theoretical decomposition of the use of weights regarding its effect on the out-of-sample error is easy to understand but not actionable in practice, as the quantities involved cannot be computed. Hence, we propose the Targeted Weighting algorithm that determines if, for a given set of weights, the out-of-sample performance will improve or not in a practical setting. This is necessary as the setting assumes there are no labeled points distributed according to the test distribution, only unlabeled samples.</p>
<p>Finally, we propose a new class of matching algorithms that can be used to match the training set to a desired distribution, such as the dual distribution (or the test distribution). These algorithms can be applied to very large datasets, and we show how they lead to improved performance in a large real dataset such as the Netflix dataset. Their computational complexity is the main reason for their advantage over previous algorithms proposed in the covariate shift literature.</p>
<p>In the second part of the thesis we apply Machine Learning to the problem of behavior recognition. We develop a specific behavior classifier to study fly aggression, and we develop a system that allows analyzing behavior in videos of animals, with minimal supervision. The system, which we call CUBA (Caltech Unsupervised Behavior Analysis), allows detecting movemes, actions, and stories from time series describing the position of animals in videos. The method summarizes the data, as well as it provides biologists with a mathematical tool to test new hypotheses. Other benefits of CUBA include finding classifiers for specific behaviors without the need for annotation, as well as providing means to discriminate groups of animals, for example, according to their genetic line.</p>https://thesis.library.caltech.edu/id/eprint/8888Advanced Monte Carlo Simulation and Machine Learning for Frequency Domain Optical Coherence Tomography
https://resolver.caltech.edu/CaltechTHESIS:03022016-094235703
Authors: {'items': [{'email': 'sinan.zhao@gmail.com', 'id': 'Zhao-Sinan', 'name': {'family': 'Zhao', 'given': 'Sinan'}, 'show_email': 'NO'}]}
Year: 2016
DOI: 10.7907/Z9X63JVM
<p>Optical Coherence Tomography(OCT) is a popular, rapidly growing imaging technique with an increasing number of bio-medical applications due to its noninvasive nature. However, there are three major challenges in understanding and improving an OCT system: (1) Obtaining an OCT image is not easy. It either takes a real medical experiment or requires days of computer simulation. Without much data, it is difficult to study the physical processes underlying OCT imaging of different objects simply because there aren't many imaged objects. (2) Interpretation of an OCT image is also hard. This challenge is more profound than it appears. For instance, it would require a trained expert to tell from an OCT image of human skin whether there is a lesion or not. This is expensive in its own right, but even the expert cannot be sure about the exact size of the lesion or the width of the various skin layers. The take-away message is that analyzing an OCT image even from a high level would usually require a trained expert, and pixel-level interpretation is simply unrealistic. The reason is simple: we have OCT images but not their underlying ground-truth structure, so there is nothing to learn from. (3) The imaging depth of OCT is very limited (millimeter or sub-millimeter on human tissues). While OCT utilizes infrared light for illumination to stay noninvasive, the downside of this is that photons at such long wavelengths can only penetrate a limited depth into the tissue before getting back-scattered. To image a particular region of a tissue, photons first need to reach that region. As a result, OCT signals from deeper regions of the tissue are both weak (since few photons reached there) and distorted (due to multiple scatterings of the contributing photons). This fact alone makes OCT images very hard to interpret.</p>
<p>This thesis addresses the above challenges by successfully developing an advanced Monte Carlo simulation platform which is 10000 times faster than the state-of-the-art simulator in the literature, bringing down the simulation time from 360 hours to a single minute. This powerful simulation tool not only enables us to efficiently generate as many OCT images of objects with arbitrary structure and shape as we want on a common desktop computer, but it also provides us the underlying ground-truth of the simulated images at the same time because we dictate them at the beginning of the simulation. This is one of the key contributions of this thesis. What allows us to build such a powerful simulation tool includes a thorough understanding of the signal formation process, clever implementation of the importance sampling/photon splitting procedure, efficient use of a voxel-based mesh system in determining photon-mesh interception, and a parallel computation of different A-scans that consist a full OCT image, among other programming and mathematical tricks, which will be explained in detail later in the thesis.</p>
<p>Next we aim at the inverse problem: given an OCT image, predict/reconstruct its ground-truth structure on a pixel level. By solving this problem we would be able to interpret an OCT image completely and precisely without the help from a trained expert. It turns out that we can do much better. For simple structures we are able to reconstruct the ground-truth of an OCT image more than 98% correctly, and for more complicated structures (e.g., a multi-layered brain structure) we are looking at 93%. We achieved this through extensive uses of Machine Learning. The success of the Monte Carlo simulation already puts us in a great position by providing us with a great deal of data (effectively unlimited), in the form of (image, truth) pairs. Through a transformation of the high-dimensional response variable, we convert the learning task into a multi-output multi-class classification problem and a multi-output regression problem. We then build a hierarchy architecture of machine learning models (committee of experts) and train different parts of the architecture with specifically designed data sets. In prediction, an unseen OCT image first goes through a classification model to determine its structure (e.g., the number and the types of layers present in the image); then the image is handed to a regression model that is trained specifically for that particular structure to predict the length of the different layers and by doing so reconstruct the ground-truth of the image. We also demonstrate that ideas from Deep Learning can be useful to further improve the performance.</p>
<p>It is worth pointing out that solving the inverse problem automatically improves the imaging depth, since previously the lower half of an OCT image (i.e., greater depth) can be hardly seen but now becomes fully resolved. Interestingly, although OCT signals consisting the lower half of the image are weak, messy, and uninterpretable to human eyes, they still carry enough information which when fed into a well-trained machine learning model spits out precisely the true structure of the object being imaged. This is just another case where Artificial Intelligence (AI) outperforms human. To the best knowledge of the author, this thesis is not only a success but also the first attempt to reconstruct an OCT image at a pixel level. To even give a try on this kind of task, it would require fully annotated OCT images and a lot of them (hundreds or even thousands). This is clearly impossible without a powerful simulation tool like the one developed in this thesis.</p>https://thesis.library.caltech.edu/id/eprint/9597Boosting Boosting
https://resolver.caltech.edu/CaltechTHESIS:06052017-221842127
Authors: {'items': [{'email': 'ronappel1@gmail.com', 'id': 'Appel-Ron', 'name': {'family': 'Appel', 'given': 'Ron'}, 'show_email': 'NO'}]}
Year: 2017
DOI: 10.7907/Z9N29V0J
<p>Machine learning is becoming prevalent in all aspects of our lives. For some applications, there is a need for simple but accurate white-box systems that are able to train efficiently and with little data.</p>
<p>"Boosting" is an intuitive method, combining many simple (possibly inaccurate) predictors to form a powerful, accurate classifier. Boosted classifiers are intuitive, easy to use, and exhibit the fastest speeds at test-time when implemented as a cascade. However, they have a few drawbacks: training decision trees is a relatively slow procedure, and from a theoretical standpoint, no simple unified framework for cost-sensitive multi-class boosting exists. Furthermore, (axis-aligned) decision trees may be inadequate in some situations, thereby stalling training; and even in cases where they are sufficiently useful, they don't capture the intrinsic nature of the data, as they tend to form boundaries that overfit.</p>
<p>My thesis focuses on remedying these three drawbacks of boosting.
Ch.III outlines a method (called QuickBoost) that trains identical classifiers at an order of magnitude faster than before, based on a proof of a bound. In Ch.IV, a unified framework for cost-sensitive multi-class boosting (called REBEL) is proposed, both advancing theory and demonstrating empirical gains. Finally, Ch.V describes a novel family of weak learners (called Localized Similarities) that guarantee theoretical bounds and outperform decision trees and Neural Nets (as well as several other commonly used classification methods) on a range of datasets. </p>
<p>The culmination of my work is an easy-to-use, fast-training, cost-sensitive multi-class boosting framework whose functionality is interpretable (since each weak learner is a simple comparison of similarity), and whose performance is better than Neural Networks and other competing methods. It is the tool that everyone should have in their toolbox and the first one they try.</p>https://thesis.library.caltech.edu/id/eprint/10292Advancing a Machine's Visual Awareness of People
https://resolver.caltech.edu/CaltechTHESIS:05292017-224523022
Authors: {'items': [{'email': 'dchall88@gmail.com', 'id': 'Hall-David-Christopher', 'name': {'family': 'Hall', 'given': 'David Christopher'}, 'orcid': '0000-0003-3244-5744', 'show_email': 'NO'}]}
Year: 2017
DOI: 10.7907/Z9ST7MWC
<p>Methods to advance a machine's visual awareness of people with a focus on understanding 'who is where' in video are presented. 'Who' is used in a broad sense that includes not only the identity of a person but attributes of that person as well. Efforts are focused on improving algorithms in four areas of visual recognition: detection, tracking, fine-grained classification and person reidentification.</p>
<p>Each of these problems appear to be quite different on the surface; however, there are two broader questions that are answered across each of the works. The first, the machine is able to make better predictions when it has access to the extra information that is available in video. The second, that it is possible to learn on-the-fly from single examples. How each work contributes to answering these over-arching questions as well as its specific contributions to the relevant problem domain are as follows:</p>
<p>The first problem studied is one-shot, real-time, instance detection. Given a single image of a person, the task for the machine is to learn a detector that is specific to that individual rather than to an entire category such as faces or pedestrians. In subsequent images, the individual detector indicates the size and location of that particular person in the image. The learning must be done in real-time. To solve this problem, the proposed method starts with a pre-trained boosted category detector from which an individual-object detector is trained, with near-zero computational cost, through elementary manipulations of the thresholds of the category detector. Experiments on two challenging pedestrian and face datasets indicate that it is indeed possible to learn identity classifiers in real-time; besides being faster-trained, the proposed classifier has better detection rates than previous methods.</p>
<p>The second problem studied is real-time tracking. Given the initial location of a target person, the task for the machine is to determine the size and location of the target person in subsequent video frames, in real-time. The method proposed for solving this problem treats tracking as a repeated detection problem where potential targets are identified with a pre-trained boosted person detector and identity across frames is established by individual-specific detectors. The individual-specific detectors are learnt using the method proposed to solve the first problem. The proposed algorithm runs in real-time and is robust to drift. The tracking algorithm is benchmarked against nine state-of-the-art trackers on two benchmark datasets. Results show that the proposed method is 10% more accurate and nearly as fast as the fastest of the competing algorithms, and it is as accurate but 20 times faster than the most accurate of the competing algorithms.</p>
<p>The third problem studied is the fine-grained classification of people. Given an image of a person, the task for the machine is to estimate characteristics of that person such as age, clothing style, sex, occupation, social status, ethnicity, emotional state and/or body type. Since fine-grained classification using the entire human body is a relatively unexplored area, a large video dataset was collected. To solve this problem, a method that uses deep neural networks and video of a person is proposed. Results show that the class average accuracy when combining information from a sequence of images of an individual and then predicting the label is 3.5-7.1% better than independently predicting the label of each image, when severely under-represented classes are ignored.</p>
<p>The final problem studied is person reidentification. Given an image of a person, the task for the machine is to find images that match the identity of that person from a large set of candidate images. This is a challenging task since images of the same individual can vary significantly due to changes in clothing, viewpoint, pose, lighting and background. The method proposed for solving this problem is a two-stage deep neural network architecture that uses body part patches as inputs rather than an entire image of a person. Experiments show that rank-1 matching rates increase by 22-25.6% on benchmark datasets when compared to state-of-the-art methods.</p>https://thesis.library.caltech.edu/id/eprint/10218Sparse Array Signal Processing: New Array Geometries, Parameter Estimation, and Theoretical Analysis
https://resolver.caltech.edu/CaltechTHESIS:05302018-095132389
Authors: {'items': [{'email': 'clliu144@gmail.com', 'id': 'Liu-Chun-Lin', 'name': {'family': 'Liu', 'given': 'Chun-Lin'}, 'orcid': '0000-0003-3135-9684', 'show_email': 'YES'}]}
Year: 2018
DOI: 10.7907/NSTQ-SD57
<p>Array signal processing focuses on an array of sensors receiving the incoming waveforms in the environment, from which source information, such as directions of arrival (DOA), signal power, amplitude, polarization, and velocity, can be estimated. This topic finds ubiquitous applications in radar, astronomy, tomography, imaging, and communications. In these applications, sparse arrays have recently attracted considerable attention, since they are capable of resolving <i>O</i>(<i>N</i><sup>2</sup>) uncorrelated source directions with <i>N</i> physical sensors. This is unlike the uniform linear arrays (ULA), which identify at most <i>N</i>-1 uncorrelated sources with <i>N</i> sensors. These sparse arrays include minimum redundancy arrays (MRA), nested arrays, and coprime arrays. All these arrays have an <i>O</i>(<i>N</i><sup>2</sup>)-long central ULA segment in the difference coarray, which is defined as the set of differences between sensor locations. This <i>O</i>(<i>N</i><sup>2</sup>) property makes it possible to resolve <i>O</i>(<i>N</i><sup>2</sup>) uncorrelated sources, using only <i>N</i> physical sensors.</p>
<p>The main contribution of this thesis is to provide a new direction for array geometry and performance analysis of sparse arrays in the presence of nonidealities. The first part of this thesis focuses on designing novel array geometries that are robust to effects of mutual coupling. It is known that, mutual coupling between sensors has an adverse effect on the estimation of DOA. While there are methods to counteract this through appropriate modeling and calibration, they are usually computationally expensive, and sensitive to model mismatch. On the other hand, sparse arrays, such as MRA, nested arrays, and coprime arrays, have reduced mutual coupling compared to ULA, but all of these have their own disadvantages. This thesis introduces a new array called the super nested array, which has many of the good properties of the nested array, and at the same time achieves reduced mutual coupling. Many theoretical properties are proved and simulations are included to demonstrate the superior performance of super nested arrays in the presence of mutual coupling.</p>
<p>Two-dimensional planar sparse arrays with large difference coarrays have also been known for a long time. These include billboard arrays, open box arrays (OBA), and 2D nested arrays. However, all of them have considerable mutual coupling. This thesis proposes new planar sparse arrays with the same large difference coarrays as the OBA, but with reduced mutual coupling. The new arrays include half open box arrays (HOBA), half open box arrays with two layers (HOBA-2), and hourglass arrays. Among these, simulations show that hourglass arrays have the best estimation performance in presence of mutual coupling.</p>
<p>The second part of this thesis analyzes the performance of sparse arrays from a theoretical perspective. We first study the Cramér-Rao bound (CRB) for sparse arrays, which poses a lower bound on the variances of unbiased DOA estimators. While there exist landmark papers on the study of the CRB in the context of array processing, the closed-form expressions available in the literature are not applicable in the context of sparse arrays for which the number of identifiable sources exceeds the number of sensors. This thesis derives a new expression for the CRB to fill this gap. Based on the proposed CRB expression, it is possible to prove the previously known experimental observation that, when there are more sources than sensors, the CRB stagnates to a constant value as the SNR tends to infinity. It is also possible to precisely specify the relation between the number of sensors and the number of uncorrelated sources such that these sources could be resolved.</p>
<p>Recently, it has been shown that correlation subspaces, which reveal the structure of the covariance matrix, help to improve some existing DOA estimators. However, the bases, the dimension, and other theoretical properties of correlation subspaces remain to be investigated. This thesis proposes generalized correlation subspaces in one and multiple dimensions. This leads to new insights into correlation subspaces and DOA estimation with prior knowledge. First, it is shown that the bases and the dimension of correlation subspaces are fundamentally related to difference coarrays, which were previously found to be important in the study of sparse arrays. Furthermore, generalized correlation subspaces can handle certain forms of prior knowledge about source directions. These results allow one to derive a broad class of DOA estimators with improved performance.</p>
<p>It is empirically known that the coarray structure is susceptible to sensor failures, and the reliability of sparse arrays remains a significant but challenging topic for investigation. This thesis advances a general theory for quantifying such robustness, by studying the effect of sensor failure on the difference coarray. We first present the (<i>k</i>-)essentialness property, which characterizes the combinations of the faulty sensors that shrink the difference coarray. Based on this, the notion of (<i>k</i>-)fragility is proposed to quantify the reliability of sparse arrays with faulty sensors, along with comprehensive studies of their properties. These novel concepts provide quite a few insights into the interplay between the array geometry and its robustness. For instance, for the same number of sensors, it can be proved that ULA is more robust than the coprime array, and the coprime array is more robust than the nested array. Rigorous development of these ideas leads to expressions for the probability of coarray failure, as a function of the probability of sensor failure.</p>
<p>The thesis concludes with some remarks on future directions and open problems.</p>https://thesis.library.caltech.edu/id/eprint/10970The Nested Periodic Subspaces: Extensions of Ramanujan Sums for Period Estimation
https://resolver.caltech.edu/CaltechTHESIS:06062018-132643508
Authors: {'items': [{'email': 'srikutv@gmail.com', 'id': 'Tenneti-Srikanth-Venkata', 'name': {'family': 'Tenneti', 'given': 'Srikanth Venkata'}, 'orcid': '0000-0002-5415-3681', 'show_email': 'YES'}]}
Year: 2018
DOI: 10.7907/1n4t-5876
<p>In the year 1918, the Indian mathematician Srinivasa Ramanujan proposed a set of sequences called Ramanujan Sums as bases to expand arithmetic functions in number theory. Today, exactly a 100 years later, we will show that these sequences re-emerge as exciting tools in a completely different context: For the extraction of periodic patterns in data. Combined with the state-of-the-art techniques of DSP, Ramanujan Sums can be used as the starting point for developing powerful algorithms for periodicity applications.</p>
<p>The primary inspiration for this thesis comes from a recent extension of Ramanujan sums to subspaces known as the Ramanujan subspaces. These subspaces were designed to span any sequence with integer periodicity, and have many interesting properties. Starting with Ramanujan subspaces, this thesis first develops an entire family of such subspace representations for periodic sequences. This family, called Nested Periodic Subspaces due to their unique structure, turns out to be the least redundant sets of subspaces that can span periodic sequences.</p>
<p>Three classes of new algorithms are proposed using the Nested Periodic Subspaces: dictionaries, filter banks, and eigen-space methods based on the auto-correlation matrix of the signal. It will be shown that these methods are especially advantageous to use when the data-length is short, or when the signal is a mixture of multiple hidden periods. The dictionary techniques were inspired by recent advances in sparsity based compressed sensing. Apart from the <i>l</i><sub>1</sub> norm based convex programs currently used in other applications, our dictionaries can admit <i>l</i><sub>2</sub> norm formulations that have linear and closed form solutions, even when the systems is under-determined. A new filter bank is also proposed using the Ramanujan sums. This, named the Ramanujan Filter Bank, can accurately track the instantaneous period for signals that exhibit time varying periodic nature. The filters in the Ramanujan Filter Bank have simple integer valued coefficients, and directly tile the period vs time plane, unlike classical STFT (Short Time Fourier Transform) and wavelets, which tile the time-frequency plane. The third family of techniques developed here are a generalization of the classic MUSIC (MUltiple SIgnal Classification) algorithm for periodic signals. MUSIC is one of the most popular techniques today for line spectral estimation. However, periodic signals are not just any unstructured line spectral signals. There is a nice harmonic spacing between the lines which is not exploited by plain MUSIC. We will show that one can design much more accurate adaptations of MUSIC using Nested Periodic Subspaces. Compared to prior variants of MUSIC for the periodicity problem, our approach is much faster and yields much more accurate results for signals with integer periods. This work is also the first extension of MUSIC that uses simple integer valued basis vectors instead of using traditional complex-exponentials to span the signal subspace. The advantages of the new methods are demonstrated both on simulations, as well as real world applications such as DNA micro-satellites, protein repeats and absence seizures.</p>
<p>Apart from practical contributions, the theory of Nested Periodic Subspaces offers answers to a number of fundamental questions that were previously unanswered. For example, what is the minimum contiguous data-length needed to be able to identify the period of a signal unambiguously? Notice that the answer we seek is a fundamental identifiability bound independent of any particular period estimation technique. Surprisingly, this basic question has never been answered before. In this thesis, we will derive precise expressions for the minimum necessary and sufficient datalengths for this question. We also extend these bounds to the context of mixtures of periodic signals. Once again, even though mixtures of periodic signals often occur in many applications, aspects such as the unique identifiability of the component periods were never rigorously analyzed before. We will present such an analysis as well.</p>
<p>While the above question deals with the minimum contiguous datalength required for period estimation, one may ask a slightly different question: If we are allowed to pick the samples of a signal in a non-contiguous fashion, how should we pick them so that we can estimate the period using the least number of samples? This question will be shown to be quite difficult to answer in general. In this thesis, we analyze a smaller case in this regard, namely, that of resolving between two periods. It will be shown that the analysis is quite involved even in this case, and the optimal sampling pattern takes an interesting form of sparsely located bunches. This result can also be extended to the case of multi-dimensional periodic signals.</p>
<p>We very briefly address multi-dimensional periodicity in this thesis. Most prior DSP literature on multi-dimensional discrete time periodic signals assumes the period to be parallelepipeds. But as shown by the artist M. C. Escher, one can tile the space using a much more diverse variety of shapes. Is it always possible to account for such other periodic shapes using the traditional notion of parallelepiped periods? An interesting analysis in this regard is presented towards the end of the thesis.</p>
https://thesis.library.caltech.edu/id/eprint/11029An Electrophysiological Study Of Voluntary Movement and Spinal Cord Injury
https://resolver.caltech.edu/CaltechTHESIS:06012018-140912331
Authors: {'items': [{'email': 'lsurban@gmail.com', 'id': 'Urban-Luke-Stuart', 'name': {'family': 'Urban', 'given': 'Luke Stuart'}, 'show_email': 'NO'}]}
Year: 2018
DOI: 10.7907/K6P2-ZH75
<p>Voluntary movement is generated from the interaction between neurons in our brain and the neurons in our spinal cord that engage our muscles. A spinal cord injury destroys the connection between these two regions, but parts of their underlying neural circuits survive. A new class of treatment (the brain-machine interface) takes advantage of this fact by either a) recording neural activity from the brain and predicting the intended movement (neural prosthetics) or b) stimulating neural activity in the spinal cord to facilitate muscle activity (spinal stimulation). This thesis covers new research studying the brain-machine interface and its application for spinal injury.</p>
<p>First, the electrical properties of the microelectrode (the main tool of the brain-machine interface) are studied during deep brain recording and stimulation. This work shows that the insulation coating the electrode forms a capacitor with the surrounding neural tissue. This capacitance causes large spikes of voltage in the surrounding tissue during deep brain stimulation, which will cause electrical artifacts in neural recordings and may damage the surrounding neurons. This work also shows that a coaxially shielded electrode will block this effect.</p>
<p>Second, the activity of neurons in the parietal cortex is studied during hand movements, which has applications for neural prosthetics. Prior work suggests that the parietal cortex encodes a state-estimator [1], which combines sensory feedback with the internal efference copy to predict the state of the hand. To test this idea, we used a visual lag to misalign sensory feedback from the efference copy. The expectation was that a state-estimator would unknowingly combine the delayed visual feedback with the current efference information, resulting in incorrect predictions of the hand. Our results show a drop in correlation between neural activity in the parietal cortex and hand movement during a visual lag, supporting the idea that the parietal cortex encodes a state-estimator. This correlation gradually recovers over time, showing that parietal cortex is adaptive to sensory delays.</p>
<p>Third, while the intention of spinal stimulation was to interact locally with neural circuits in the spinal cord, results from the clinic show that electrical stimulation of the lumbosacral enlargement enables paraplegic patients to regain voluntary movement of their legs [2]. This means that spinal stimulation facilitates communication across an injury site. To further study this effect, we developed a new behavioral task in the rodent. Rats were trained to kick their right hindlimb in response to an auditory cue. The animals then received a spinal injury that caused paraplegia. After injury, the animals recovered the behavior (they could kick in response to the cue), but only during spinal stimulation. Their recovered behavior was slower and more stereotyped than their pre-injury response. Administering quipazine to these rodents disrupted their ability to respond to the cue, suggesting that serotonin plays an important role in the recovered pathway. This work proves that the new behavioral task is a successful tool for studying the recovery of voluntary movement.</p>
<p>Future work will combine cortical recordings with this behavioral task in the rodent to study plasticity in the nervous system and improve treatment of spinal cord injuries.</p>
<p>[1] Mulliken, Grant H., Sam Musallam, and Richard A. Andersen. "Forward estimation of movement state in posterior parietal cortex." Proceedings of the National Academy of Sciences105.24 (2008): 8170-8177.</p>
<p>[2] Harkema, Susan, et al. "Effect of epidural stimulation of the lumbosacral spinal cord on voluntary movement, standing, and assisted stepping after motor complete paraplegia: a case study." The Lancet 377.9781 (2011): 1938-1947.</p>https://thesis.library.caltech.edu/id/eprint/11000Signals on Networks: Random Asynchronous and Multirate Processing, and Uncertainty Principles
https://resolver.caltech.edu/CaltechTHESIS:09242020-094028488
Authors: {'items': [{'email': 'ogteke@gmail.com', 'id': 'Teke-Oguzhan', 'name': {'family': 'Teke', 'given': 'Oguzhan'}, 'orcid': '0000-0002-1131-5206', 'show_email': 'NO'}]}
Year: 2021
DOI: 10.7907/44dx-3g83
<p>The processing of signals defined on graphs has been of interest for many years, and finds applications in a diverse set of fields such as sensor networks, social and economic networks, and biological networks. In graph signal processing applications, signals are not defined as functions on a uniform time-domain grid but they are defined as vectors indexed by the vertices of a graph, where the underlying graph is assumed to model the irregular signal domain. Although analysis of such networked models is not new (it can be traced back to the consensus problem studied more than four decades ago), such models are studied recently from the view-point of signal processing, in which the analysis is based on the "graph operator" whose eigenvectors serve as a Fourier basis for the graph of interest. With the help of graph Fourier basis, a number of topics from classical signal processing (such as sampling, reconstruction, filtering, etc.) are extended to the case of graphs.</p>
<p>The main contribution of this thesis is to provide new directions in the field of graph signal processing and provide further extensions of topics in classical signal processing. The first part of this thesis focuses on a random and asynchronous variant of "graph shift," i.e., localized communication between neighboring nodes. Since the dynamical behavior of randomized asynchronous updates is very different from standard graph shift (i.e., state-space models), this part of the thesis focuses on the convergence and stability behavior of such random asynchronous recursions. Although non-random variants of asynchronous state recursions (possibly with non-linear updates) are well-studied problems with early results dating back to the late 60's, this thesis considers the convergence (and stability) in the statistical mean-squared sense and presents the precise conditions for the stability by drawing parallels with switching systems. It is also shown that systems exhibit unexpected behavior under randomized asynchronicity: an unstable system (in the synchronous world) may be stabilized simply by the use of randomized asynchronicity. Moreover, randomized asynchronicity may result in a lower total computational complexity in certain parameter settings. The thesis presents applications of the random asynchronous model in the context of graph signal processing including an autonomous clustering of network of agents, and a node-asynchronous communication protocol that implements a given rational filter on the graph.</p>
<p>The second part of the thesis focuses on extensions of the following topics in classical signal processing to the case of graph: multirate processing and filter banks, discrete uncertainty principles, and energy compaction filters for optimal filter design. The thesis also considers an application to the heat diffusion over networks.</p>
<p>Multirate systems and filter banks find many applications in signal processing theory and implementations. Despite the possibility of extending 2-channel filter banks to bipartite graphs, this thesis shows that this relation cannot be generalized to <i>M</i>-channel systems on <i>M</i>-partite graphs. As a result, the extension of classical multirate theory to graphs is nontrivial, and such extensions cannot be obtained without certain mathematical restrictions on the graph. The thesis provides the necessary conditions on the graph such that fundamental building blocks of multirate processing remain valid in the graph domain. In particular, it is shown that when the underlying graph satisfies a condition called <i>M</i>-block cyclic property, classical multirate theory can be extended to the graphs.</p>
<p>The uncertainty principle is an essential mathematical concept in science and engineering, and uncertainty principles generally state that a signal cannot have an arbitrarily "short" description in the original basis and in the Fourier basis simultaneously. Based on the fact that graph signal processing proposes two different bases (i.e., vertex and the graph Fourier domains) to represent graph signals, this thesis shows that the total number of nonzero elements of a graph signal and its representation in the graph Fourier domain is lower bounded by a quantity depending on the underlying graph. The thesis also presents the necessary and sufficient condition for the existence of 2-sparse and 3-sparse eigenvectors of a connected graph. When such eigenvectors exist, the uncertainty bound is very low, tight, and independent of the global structure of the graph.</p>
<p>The thesis also considers the classical spectral concentration problem. In the context of polynomial graph filters, the problem reduces to the polynomial concentration problem studied more generally by Slepian in the 70's. The thesis studies the asymptotic behavior of the optimal solution in the case of narrow bandwidth. Different examples of graphs are also compared in order to show that the maximum energy compaction and the optimal filter depends heavily on the graph spectrum.</p>
<p>In the last part, the thesis considers the estimation of the starting time of a heat diffusion process from its noisy measurements when there is a single point source located on a known vertex of a graph with unknown starting time. In particular, the Cramér-Rao lower bound for the estimation problem is derived, and it is shown that for graphs with higher connectivity the problem has a larger lower bound making the estimation problem more difficult.</p>https://thesis.library.caltech.edu/id/eprint/13965Gravitational Wave Signatures of Black Hole Physics
https://resolver.caltech.edu/CaltechTHESIS:06072021-070015547
Authors: {'items': [{'email': 'zmarkgrel@gmail.com', 'id': 'Mark-Zachary-R', 'name': {'family': 'Mark', 'given': 'Zachary R.'}, 'orcid': '0000-0003-2300-893X', 'show_email': 'YES'}]}
Year: 2021
DOI: 10.7907/kh82-1q43
<p>Gravitational wave observations are opening the door to test general relativity in regimes far less common than the weak gravitational fields that we experience in the solar system. The first part of this thesis addresses the broad issue of how different exotic predictions of general relativity imprint themselves in gravitational waves.</p>
<p>The ringdown portion of a binary black hole merger is dominated by superposition of quasinormal modes, the resonant modes of a perturbed black hole. The quasinormal mode spectrum of a perturbed black hole mostly reflects the spacetime geometry near the photon orbits. Chapter 2 of this thesis develops a new method for calculating quasinormal mode frequencies for weakly charged, rotating black holes. Chapter 3 uses a variety of analytic approximations to calculate the charged, rotating quasinormal mode frequencies in other cases, including nearly extremal black holes.</p>
<p>The event horizon is one of the most unique predictions of general relativity and it unsurprisingly does not imprint itself in gravitational wave emission. However, alternatives to black holes known as exotic compact objects do leave a unique signature in the form of echoes following the initial signal. Chapter 4 develops a formalism to understand and calculate these echoes.</p>
<p>The second part of this thesis focuses on reducing the noise in gravitational wave measurements using neural networks. Chapter 5 demonstrates on mock data how simple neural networks can use auxiliary measurements from the detector to predict unmodeled noise which can be subtracted offline.</p>https://thesis.library.caltech.edu/id/eprint/14251Representation of the Semantic Structures: from Discovery to Applications
https://resolver.caltech.edu/CaltechTHESIS:12092021-125955775
Authors: {'items': [{'email': 'slryou41@gmail.com', 'id': 'Ryou-Serim', 'name': {'family': 'Ryou', 'given': 'Serim'}, 'orcid': '0000-0003-1344-1158', 'show_email': 'NO'}]}
Year: 2022
DOI: 10.7907/rvnc-dp57
<p>The world surrounding us is full of structured entities. Scenes can be structured as the sum of objects arranged in space, objects can be decomposed into parts, and even small molecules are composed of atoms. As humans can organize and structure many concepts into smaller components, structural representation has become a powerful tool for various applications. Computer vision utilizes the part-based representation for classical object detection and categorization tasks, and computational neuroscientists use the structural representation to achieve an interpretable and low-dimensional encoding for behavior analysis. Furthermore, structural encoding of the molecules allows the application of machine learning models to optimize experimental reaction conditions in organic chemistry.</p>
<p>To perform the high-level tasks described above, accurate detection of the structural component should be accomplished in advance. In this dissertation, we first propose methods to improve the pose estimation algorithm, where the task is to localize the semantic parts of the target instance from a 2D image. As the collection of a large number of human annotations is a prerequisite for the task to be successful, we aim to design a model that automatically discovers the structure information from the visual inputs without supervision. Lastly, we demonstrate the efficacy of the structural representation by applying it to various scientific applications such as behavior analysis and organic chemistry.</p>https://thesis.library.caltech.edu/id/eprint/14444Advancements in Hemodynamic Measurement: Arterial Resonance, Ultrasound, and Machine Learning
https://resolver.caltech.edu/CaltechTHESIS:06022023-215651797
Authors: {'items': [{'email': 'dominicyurk@gmail.com', 'id': 'Yurk-Dominic-Jeffrey', 'name': {'family': 'Yurk', 'given': 'Dominic Jeffrey'}, 'orcid': '0000-0002-2276-4189', 'show_email': 'YES'}]}
Year: 2023
DOI: 10.7907/q7j4-vj19
<p>This thesis covers two separate projects which both use ultrasound to measure a form of blood pressure in very different ways. The first project focuses on the noninvasive measurement of continuous arterial blood pressure via the previously unstudied phenomenon of arterial resonance. While prior research efforts have attempted many methods of noninvasive blood pressure measurement, none has been able to generate continuous, calibration-free measurements based on a first-principles physical model. This work describes the derivation of this resonance-based model, its <i>in vitro</i> validation, and its <i>in vivo</i> testing on 60 subjects. This testing resulted in robust resonance detection and accurate calculation of BP in the large majority of evaluated subjects, representing very promising performance for the first test of a new biomedical technology. The second study changes focus to the measurement of blood pressure in the right atrium of the heart, an important clinical indicator in heart disease patients. Rather than developing a new physical approach, this project used machine learning to model the existing assessments made by cardiologists. Comparison to gold standard invasive catheter measurements showed that model predictions were statistically indistinguishable from cardiologist measurements. Both of these projects represent significant advances in expanding precise blood pressure measurements beyond critical care units and expanding access to a much broader population.</p>https://thesis.library.caltech.edu/id/eprint/16066Learning and Control of Dynamical Systems
https://resolver.caltech.edu/CaltechTHESIS:05282023-011333603
Authors: {'items': [{'email': 'sahinlale93@gmail.com', 'id': 'Lale-Ali-Sahin', 'name': {'family': 'Lale', 'given': 'Ali Sahin'}, 'orcid': '0000-0002-7191-346X', 'show_email': 'YES'}]}
Year: 2023
DOI: 10.7907/rdhq-8a88
<p>Despite the remarkable success of machine learning in various domains in recent years, our understanding of its fundamental limitations remains incomplete. This knowledge gap poses a grand challenge when deploying machine learning methods in critical decision-making tasks, where incorrect decisions can have catastrophic consequences. To effectively utilize these learning-based methods in such contexts, it is crucial to explicitly characterize their performance. Over the years, significant research efforts have been dedicated to learning and control of dynamical systems where the underlying dynamics are unknown or only partially known a priori, and must be inferred from collected data. However, much of these classical results have focused on asymptotic guarantees, providing limited insights into the amount of data required to achieve desired control performance while satisfying operational constraints such as safety and stability, especially in the presence of statistical noise.</p>
<p>In this thesis, we study the statistical complexity of learning and control of unknown dynamical systems. By utilizing recent advances in statistical learning theory, high-dimensional statistics, and control theoretic tools, we aim to establish a fundamental understanding of the number of samples required to achieve desired (i) accuracy in learning the unknown dynamics, (ii) performance in the control of the underlying system, and (iii) satisfaction of the operational constraints such as safety and stability. We provide finite-sample guarantees for these objectives and propose efficient learning and control algorithms that achieve the desired performance at these statistical limits in various dynamical systems. Our investigation covers a broad range of dynamical systems, starting from fully observable linear dynamical systems to partially observable linear dynamical systems, and ultimately, nonlinear systems.</p>
<p>We deploy our learning and control algorithms in various adaptive control tasks in real-world control systems and demonstrate their strong empirical performance along with their learning, robustness, and stability guarantees. In particular, we implement one of our proposed methods, Fourier Adaptive Learning and Control (FALCON), on an experimental aerodynamic testbed under extreme turbulent flow dynamics in a wind tunnel. The results show that FALCON achieves state-of-the-art stabilization performance and consistently outperforms conventional and other learning-based methods by at least 37%, despite using 8 times less data. The superior performance of FALCON arises from its physically and theoretically accurate modeling of the underlying nonlinear turbulent dynamics, which yields rigorous finite-sample learning and performance guarantees. These findings underscore the importance of characterizing the statistical complexity of learning and control of unknown dynamical systems.</p>https://thesis.library.caltech.edu/id/eprint/15219