Submitted - 1305.2714v5.pdf

", "abstract": "Denoising has to do with estimating a signal x_0 from its noisy observations y = x_0 + z. In this paper, we focus on the \"structured denoising problem,\" where the signal x_0 possesses a certain structure and z has independent normally distributed entries with mean zero and variance \u03c3^2. We employ a structure-inducing convex function f(\u22c5) and solve min_x {1/2 \u2225y\u2212x\u2225^2_2 +\u03c3\u03bbf(x)}to estimate x_0, for some \u03bb>0. Common choices for f(\u22c5) include the \u2113_1 norm for sparse vectors, the \u2113_1 \u2212\u2113_2 norm for block-sparse signals and the nuclear norm for low-rank matrices. The metric we use to evaluate the performance of an estimate x\u2217 is the normalized mean-squared error NMSE(\u03c3)=E\u2225x\u2217 \u2212 x_0\u2225^2_2/\u03c3^2. We show that NMSE is maximized as \u03c3\u21920 and we find the exact worst-case NMSE, which has a simple geometric interpretation: the mean-squared distance of a standard normal vector to the \u03bb-scaled subdifferential \u03bb\u2202f(x_0). When \u03bb is optimally tuned to minimize the worst-case NMSE, our results can be related to the constrained denoising problem min_(f(x)\u2264f(x_0)){\u2225y\u2212x\u22252}. The paper also connects these results to the generalized LASSO problem, in which one solves min_(f(x)\u2264f(x_0)){\u2225y\u2212Ax\u22252} to estimate x_0 from noisy linear observations y=Ax_0 + z. We show that certain properties of the LASSO problem are closely related to the denoising problem. In particular, we characterize the normalized LASSO cost and show that it exhibits a \"phase transition\" as a function of number of observations. We also provide an order-optimal bound for the LASSO error in terms of the mean-squared distance. Our results are significant in two ways. First, we find a simple formula for the performance of a general convex estimator. Secondly, we establish a connection between the denoising and linear inverse problems.", "date": "2016-08", "date_type": "published", "publication": "Foundations of Computational Mathematics", "volume": "16", "number": "4", "publisher": "Springer", "pagerange": "965-1029", "id_number": "CaltechAUTHORS:20160825-102038882", "issn": "1615-3375", "official_url": "https://resolver.caltech.edu/CaltechAUTHORS:20160825-102038882", "rights": "No commercial reproduction, distribution, display or performance rights in this work are provided.", "funders": { "items": [ { "agency": "NSF", "grant_number": "CCF-0729203" }, { "agency": "NSF", "grant_number": "CNS-0932428" }, { "agency": "NSF", "grant_number": "CIF-1018927" }, { "agency": "Office of Naval Research (ONR)", "grant_number": "N00014-08-1-0747" }, { "agency": "Qualcomm Inc." } ] }, "doi": "10.1007/s10208-015-9278-4", "primary_object": { "basename": "1305.2714v5.pdf", "url": "https://authors.library.caltech.edu/records/fbe5r-xn288/files/1305.2714v5.pdf" }, "resource_type": "article", "pub_year": "2016", "author_list": "Oymak, Samet and Hassibi, Babak" } ]