Book Section records
https://feeds.library.caltech.edu/people/Smyth-P/book_section.rss
A Caltech Library Repository Feedhttp://www.rssboard.org/rss-specificationpython-feedgenenThu, 30 Nov 2023 18:30:52 +0000An Information Theoretic Approach to Rule-Based Connectionist Expert Systems
https://resolver.caltech.edu/CaltechAUTHORS:20160107-155547718
Authors: Goodman, Rodney M.; Miller, John W.; Smyth, Padhraic
Year: 1989
We discuss in this paper architectures for executing probabilistic rule-bases in a parallel
manner, using as a theoretical basis recently introduced information-theoretic
models. We will begin by describing our (non-neural) learning algorithm and theory
of quantitative rule modelling, followed by a discussion on the exact nature of two
particular models. Finally we work through an example of our approach, going from
database to rules to inference network, and compare the network's performance with
the theoretical limits for specific problems.https://authors.library.caltech.edu/records/tz4df-tg596An Information Theoretic Approach to Modeling Neural Network Expert Systems
https://resolver.caltech.edu/CaltechAUTHORS:20170711-165746284
Authors: Goodman, Rodney M.; Miller, John W.; Smyth, Padhraic
Year: 1989
DOI: 10.1109/ITW.1989.761436
In this paper we propose several novel techniques for mapping rule bases, such as are used in rule based expert systems, onto neural network architectures. Our objective in doing this is to achieve a system capable of incremental learning, and distributed probabilistic inference. Such a system would be capable of performing inference many orders of magnitude faster than current serial rule based expert systems, and hence be capable of true real time operation. In addition, the rule based formalism gives the system an explicit knowledge representation, unlike current neural models. We propose an information-theoretic approach to this problem, which really has two aspects: firstly learning the model and, secondly, performing inference using this model. We will show a clear pathway to implementing an expert system starting from raw data, via a learned rule-based model, to a neural network that performs distributed inference.https://authors.library.caltech.edu/records/t79vm-e8688Objective Functions For Neural Network Classifier Design
https://resolver.caltech.edu/CaltechAUTHORS:20170620-163501947
Authors: Goodman, Rod; Miller, John W.; Smyth, Padhraic
Year: 1991
DOI: 10.1109/ISIT.1991.695143
Backpropagation was originally derived in the context of minimizing a mean-squared error (MSE) objective function. More recently there has been interest in objective functions that provide accurate class probability estimates. In this talk we derive necessary and sufficient conditions on the required form of an objective function to provide probability estimates. This leads to the definition of a general class of functions which includes MSE and cross entropy (CE) as two of the simplest cases. We establish the equivalence of these functions to Maximum Likelihood estimation and the more general principle of Minimum Description Length models. Empirical results are used to demonstrate the tradeoffs associated with the choice of objective functions which minimize to a probability.https://authors.library.caltech.edu/records/q4xgq-96c04Objective functions for probability estimation
https://resolver.caltech.edu/CaltechAUTHORS:20190314-142000675
Authors: Miller, John W.; Goodman, Rod; Smyth, Padhraic
Year: 1991
DOI: 10.1109/ijcnn.1991.155295
Backpropagation was originally derived in the context of minimizing a mean-squared error (MSE) objective function. More recently there has been interest in objective functions that provide accurate class probability estimates. In this paper we derive necessary and sufficient conditions on the required form of an objective function to provide probability estimates. This leads to the definition of a general class of functions which includes MSE and cross cutropy (CE) as two of the simplest cases.https://authors.library.caltech.edu/records/85qkn-3ga06Self-clustering recurrent networks
https://resolver.caltech.edu/CaltechAUTHORS:20190314-155127316
Authors: Zeng, Zheng; Goodman, Rodney M.; Smyth, Padhraic
Year: 1993
DOI: 10.1109/icnn.1993.298535
Recurrent neural networks have recently been shown to have the ability to learn finite state automata (FSA's) from examples. In this paper it is shown, based on empirical analyses, that second-order networks which are trained to learn FSA's tend to form discrete clusters as the state representation in the hidden unit activation space. This observation is used to define 'self-clustering' networks which automatically extract discrete state machines from the learned network. However, the problem of instability on long test strings is a factor in the generalization performance of recurrent networks - in essence, because of the analog nature of the state representation, the network gradually "forgets" where the individual state regions are. To address this problem a new network structure is introduced whereby the network uses quantization in the feedback path to force the learning of discrete states. Experimental results show that the new method learns FSA's just as well as existing methods in the literature but with the significant advantage of being stable on test strings of arbitrary length.https://authors.library.caltech.edu/records/2r5dm-sjr89Automating the Hunt for Volcanoes on Venus
https://resolver.caltech.edu/CaltechAUTHORS:20120306-142457025
Authors: Burl, M. C.; Fayyad, U. M.; Perona, P.; Smyth, P.; Burl, M. P.
Year: 1994
DOI: 10.1109/CVPR.1994.323844
Our long-term goal is to develop a trainable tool for locating patterns of interest in large image databases. Toward this goal we have developed a prototype system, based on classical filtering and statistical pattern recognition techniques, for automatically locating volcanoes in the Magellan SAR database of Venus. Training for the specific volcano-detection task is obtained by synthesizing feature templates (via normalization and principal components analysis) from a small number of examples provided by experts. Candidate regions identified by a focus of attention (FOA) algorithm are classified based on correlations with the feature templates. Preliminary tests show performance comparable to trained human observers.https://authors.library.caltech.edu/records/d52hq-zw836Automated analysis of radar imagery of Venus: handling lack of ground truth
https://resolver.caltech.edu/CaltechAUTHORS:20120306-150027112
Authors: Burl, M. C.; Fayyad, Usama M.; Perona, Pietro; Smyth, Padhraic
Year: 1994
DOI: 10.1109/ICIP.1994.413852
Lack of verifiable ground truth is a common problem in remote sensing image analysis. For example, consider the synthetic aperture radar (SAR) image data of Venus obtained by the Magellan spacecraft. Planetary scientists are interested in automatically cataloging the locations of all the small volcanoes in this data set; however, the problem is very difficult and cannot be performed with perfect reliability even by human experts. Thus, training and evaluating the performance of an automatic algorithm on this data set must be handled carefully. We discuss the use of weighted free-response receiver-operating characteristics (wFROCs) for evaluating detection performance when the "ground truth" is subjective. In particular, we evaluate the relative detection performance of humans and automatic algorithms. Our experimental results indicate that proper assessment of the uncertainty in "ground truth" is essential in applications of this nature.https://authors.library.caltech.edu/records/xa9d9-ych31Inferring Ground Truth from Subjective Labelling of Venus Images
https://resolver.caltech.edu/CaltechAUTHORS:20150305-153627706
Authors: Smyth, Padhraic; Fayyad, Usama; Burl, Michael; Perona, Pietro; Baldi, Pierre
Year: 1995
In remote sensing applications "ground-truth" data is often used as the basis for training pattern recognition algorithms to generate thematic maps or to detect objects of interest. In practical situations, experts may visually examine the images and provide a subjective noisy estimate of the truth. Calibrating the reliability
and bias of expert labellers is a non-trivial problem. In this paper we discuss some of our recent work on this topic in the context of detecting small volcanoes in Magellan SAR images of Venus. Empirical results (using the Expectation-Maximization procedure) suggest that accounting for subjective noise can be quite significant
in terms of quantifying both human and algorithm detection
performance.https://authors.library.caltech.edu/records/39t16-xsg91Gene Expression Clustering with Functional Mixture Models
https://resolver.caltech.edu/CaltechAUTHORS:20160309-105810912
Authors: Chudova, Darya; Hart, Christopher; Mjolsness, Eric; Smyth, Padhraic
Year: 2004
We propose a functional mixture model for simultaneous clustering and alignment of sets of curves measured on a discrete time grid. The model is specifically tailored to gene expression time course data. Each functional cluster center is a nonlinear combination of solutions of a simple
linear differential equation that describes the change of individual mRNA levels when the synthesis and decay rates are constant. The mixture of continuous time parametric functional forms allows one to (a) account for the heterogeneity in the observed profiles, (b) align the profiles in time by estimating real-valued time shifts, (c) capture the synthesis and decay of mRNA in the course of an experiment, and (d) regularize noisy profiles
by enforcing smoothness in the mean curves. We derive an EM algorithm for estimating the parameters of the model, and apply the proposed approach to the set of cycling genes in yeast. The experiments show consistent improvement in predictive power and within cluster variance compared to regular Gaussian mixtures.https://authors.library.caltech.edu/records/bcdmz-d7x31