Monograph records
https://feeds.library.caltech.edu/people/Abu-Mostafa-Y-S/monograph.rss
A Caltech Library Repository Feedhttp://www.rssboard.org/rss-specificationpython-feedgenenTue, 16 Apr 2024 13:15:21 +0000Two Theorems on Time Bounded Kolmogrov-Chaitin Complexity
https://resolver.caltech.edu/CaltechCSTR:1985.5205-tr-85
Authors: {'items': [{'id': 'Schweizer-D', 'name': {'family': 'Schweizer', 'given': 'David'}}, {'id': 'Abu-Mostafa-Y-S', 'name': {'family': 'Abu-Mostafa', 'given': 'Yaser'}}]}
Year: 1985
DOI: 10.7907/130cy-kxs54
An obvious extension of the KolmogorovChaitin notion of complexity is to require that the program which generates a string terminate within a prespecified time bound. We show that given a computable bound on the amount of time allowed for the production of a string from the program which generates it, there exist strings of arbitrarily low KolmogorovChaitin complexity which appear maximally random. That is, given a notion of fast, we show that there are strings which are generated by extremely short programs, but which are not generated by any fast programs shorter than the strings themselves. We show by enumeration that if we consider generating strings from programs some constant number of bits shorter than the strings themselves then these apparently random strings are significant (i.e are a proper fraction of all strings of a given length).https://authors.library.caltech.edu/records/130cy-kxs54No Free Lunch for Early Stopping
https://resolver.caltech.edu/CaltechCSTR:1998.cs-tr-98-02
Authors: {'items': [{'id': 'Çataltepe-Zehra-Kök', 'name': {'family': 'Çataltepe', 'given': 'Zehra'}}, {'id': 'Abu-Mostafa-Y-S', 'name': {'family': 'Abu-Mostafa', 'given': 'Yaser S.'}}, {'id': 'Magdon-Ismail-M', 'name': {'family': 'Magdon-Ismail', 'given': 'Malik'}}]}
Year: 2001
DOI: 10.7907/Z9B8565P
We show that, with a uniform prior on hypothesis functions having the same training error, early stopping at some fixed training error above the training error minimum results in an increase in the expected generalization error. We also show that regularization methods are equivalent to early stopping with certain non-uniform prior on the early stopping solutions.https://authors.library.caltech.edu/records/f2t2r-bk048CGBoost: Conjugate Gradient in Function Space
https://resolver.caltech.edu/CaltechCSTR:2003.007
Authors: {'items': [{'id': 'Li-L', 'name': {'family': 'Li', 'given': 'Ling'}}, {'id': 'Abu-Mostafa-Y-S', 'name': {'family': 'Abu-Mostafa', 'given': 'Yaser S.'}}, {'id': 'Pratap-A', 'name': {'family': 'Pratap', 'given': 'Amrit'}}]}
Year: 2003
DOI: 10.7907/yryfk-z8a36
The superior out-of-sample performance of AdaBoost has been attributed to the fact that it minimizes a cost function based on margin, in that it can be viewed as a special case of AnyBoost, an abstract gradient descent algorithm. In this paper, we provide a more sophisticated abstract boosting algorithm, CGBoost, based on conjugate gradient in function space. When the AdaBoost exponential cost function is optimized, CGBoost generally yields much lower cost and training error but higher test error, which implies that the exponential cost is vulnerable to overfitting. With the optimization power of CGBoost, we can adopt more "regularized" cost functions that have better out-of-sample performance but are difficult to optimize. Our experiments demonstrate that CGBoost generally outperforms AnyBoost in cost reduction. With suitable cost functions, CGBoost can have better out-of-sample performance.https://authors.library.caltech.edu/records/yryfk-z8a36The Bin Model
https://resolver.caltech.edu/CaltechCSTR:2004.002
Authors: {'items': [{'id': 'Abu-Mostafa-Y-S', 'name': {'family': 'Abu-Mostafa', 'given': 'Yaser'}}, {'id': 'Song-Xubo', 'name': {'family': 'Song', 'given': 'Xubo'}}, {'id': 'Nicholson-A', 'name': {'family': 'Nicholson', 'given': 'Alexander'}}, {'id': 'Magdon-Ismail-M', 'name': {'family': 'Magdon-Ismail', 'given': 'Malik'}}]}
Year: 2004
DOI: 10.7907/Z9222RR7
We propose a novel theoretical framework for understanding learning and generalization which we will call the bin model. Using the bin model, a closed form is derived for the generalization error that estimates the out-of-sample performance in terms of the in-sample performance. We address the problem of overfitting, and show that using a simple exhaustive learning algorithm it does not arise. This is independent of the target function, input distribution and learning model, and remains true even with noisy data sets. We apply our analysis to both classification and regression problems and give an example of how it may be used effectively in practice.https://authors.library.caltech.edu/records/n7j7x-tmq04Data complexity in machine learning
https://resolver.caltech.edu/CaltechCSTR:2006.004
Authors: {'items': [{'id': 'Li-Ling', 'name': {'family': 'Li', 'given': 'Ling'}}, {'id': 'Abu-Mostafa-Y-S', 'name': {'family': 'Abu-Mostafa', 'given': 'Yaser S.'}}]}
Year: 2006
DOI: 10.7907/Z9319SW2
We investigate the role of data complexity in the context of binary classification problems. The universal data complexity is defined for a data set as the Kolmogorov complexity of the mapping enforced by the data set. It is closely related to several existing principles used in machine learning such as Occam's razor, the minimum description length, and the Bayesian approach. The data complexity can also be defined based on a learning model, which is more realistic for applications. We demonstrate the application of the data complexity in two learning problems, data decomposition and data pruning. In data decomposition, we illustrate that a data set is best approximated by its principal subsets which are Pareto optimal with respect to the complexity and the set size. In data pruning, we show that outliers usually have high complexity contributions, and propose methods for estimating the complexity contribution. Since in practice we have to approximate the ideal data complexity measures, we also discuss the impact of such approximations.https://authors.library.caltech.edu/records/ypj0s-xex59County-Specific, Real-Time Projection of the Effect of Business Closures on the COVID-19 Pandemic
https://resolver.caltech.edu/CaltechAUTHORS:20210212-100829452
Authors: {'items': [{'id': 'Yurk-Dominic', 'name': {'family': 'Yurk', 'given': 'Dominic'}}, {'id': 'Abu-Mostafa-Y-S', 'name': {'family': 'Abu-Mostafa', 'given': 'Yaser'}}]}
Year: 2021
DOI: 10.1101/2021.02.10.21251533
Public health policies such as business closures have been one of our most effective tools in slowing the spread of COVID-19, but they also impose costs. This has created demand from policy makers for models which can predict when and where such policies will be most effective to head off a surge and where they could safely be loosened. No current model combines data-driven, real-time policy effect predictions with county-level granularity. We present a neural net-based model for predicting the effect of business closures or re-openings on the COVID-19 time-varying reproduction number Rt in real time for every county in California. When trained on data from May through September the model accurately captured relative county dynamics during the October/November California COVID-19 surge (r2=0.76), indicating robust out-of-sample performance. To showcase the model's potential utility we present a case study of various counties in mid-October. Even when counties imposed similar restrictions at the time, our model successfully distinguished counties in need of drastic and immediate action to head off a surge from counties in less dire need of intervention. While this study focuses on business closures in California, the presented model architecture could be applied to other policies around world.https://authors.library.caltech.edu/records/tehhm-n2q81