Book Section records
https://feeds.library.caltech.edu/people/HassibiB/book_section.rss
A Caltech Library Repository Feedhttp://www.rssboard.org/rssspecificationpythonfeedgenenFri, 08 Dec 2023 12:14:39 +0000Second Order Derivatives for Network Pruning: Optimal Brain Surgeon
https://resolver.caltech.edu/CaltechAUTHORS:20150219075206704
Authors: Hassibi, Babak; Stork, David G.
Year: 1993
We investigate the use of information from all second order derivatives of the error function to perform network pruning (i.e., removing unimportant weights from a trained
network) in order to improve generalization, simplify networks, reduce hardware or storage requirements, increase the speed of further training, and in some cases enable rule
extraction. Our method, Optimal Brain Surgeon (OBS), is Significantly better than magnitudebased methods and Optimal Brain Damage [Le Cun, Denker and Sol1a, 1990],
which often remove the wrong weights. OBS permits the pruning of more weights than
other methods (for the same error on the training set), and thus yields better generalization on test data. Crucial to OBS is a recursion relation for calculating the
inverse Hessian matrix H^(1) from training data and structural information of the net. OBS
permits a 90%, a 76%, and a 62% reduction in weights over backpropagation with weigh decay on three benchmark MONK's problems [Thrun et aI., 1991]. Of OBS, Optimal
Brain Damage, and magnitudebased methods, only OBS deletes the correct weights from a trained XOR network in every case. Finally, whereas Sejnowski and Rosenberg [1987J
used 18,000 weights in their NETtalk network, we used OBS to prune a network to just 1560 weights, yielding better generalization.https://authors.library.caltech.edu/records/d0dqgkr329Optimal Brain Surgeon and general network pruning
https://resolver.caltech.edu/CaltechAUTHORS:20150219074543150
Authors: Hassibi, Babak; Stork, David G.; Wolff, Gregory J.
Year: 1993
DOI: 10.1109/ICNN.1993.298572
The use of information from all secondorder derivatives of the error function to perform network pruning (i.e., removing unimportant weights from a trained network) in order to improve generalization, simplify networks, reduce hardware or storage requirements, increase the
speed of further training, and, in some cases, enable rule extraction, is investigated. The method, Optimal Brain Surgeon (OBS), is significantly better than magnitudebased methods and Optimal Brain Damage, which often remove the wrong weights. OBS, permits pruning of
more weights than other methods (for the same error on the training set), and thus yields better generalization on test data. Crucial to OBS is a recursion relation for calculating the inverse Hessian matrix
H^1 from training data and structural information of the set. OBS permits a 76%, a 62%, and a 90% reduction in weights over backpropagation with weight decay on
three benchmark MONK'S problems. Of OBS, Optimal Brain Damage, and a magnitudebased method, only OBS deletes the correct weights from a trained XOR network in every case.
Finally, whereas Sejnowski and Rosenberg used 18,000
weights in their NETtalk network, we used OBS to prune
a network to just 1,560 weights, yielding better generalization.https://authors.library.caltech.edu/records/tzehbvz706Blind identification of FIR channels via antenna arrays
https://resolver.caltech.edu/CaltechAUTHORS:20150219074206061
Authors: Khalaj, Babak H.; Hassibi, Babak; Paulraj, Arogyaswami; Kailath, Thomas
Year: 1993
DOI: 10.1109/ACSSC.1993.342615
We propose a new method for blind identification of possibly nonminimum phase FIR channels using antenna arrays.
Based on a frequency domain formulation, we show that even
a nonminimum phase channel driven by stationary sequences
can be identified from the secondorder statistics. Although
in the single antenna case it is necessary to use cyclostationary signals or higher order statistics to identify the magnitude and phase of the channel, we circumvent such a requirement by exploiting multichannel properties of the array. We also present necessary and
sufficient conditions for channel identifiability, and
develop an identification algorithm.https://authors.library.caltech.edu/records/kmfrv4ex13Recursive linear estimation in Krein spaces  Part I. Theory
https://resolver.caltech.edu/CaltechAUTHORS:20150219072749955
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1993
DOI: 10.1109/CDC.1993.325867
We develop a selfcontained theory for linear estimation in Krein spaces. The theory is based on simple concepts such as projections and matrix factorizations, and leads to an interesting connection between Krein space projection and the computation of the stationary points of
certain second order (or quadratic) forms. We use the innovations process to obtain a rather general recursive linear estimation algorithm, which when specialized to a state space model yields a Krein space generalization of the celebrated Kalman filter with applications
in several areas such as H^∞filtering and control, game
problems, risk sensitive control, and adaptive filtering.https://authors.library.caltech.edu/records/jhsqtsvr31Recursive Linear Estimation In Krein Spaces  Part II: Applications
https://resolver.caltech.edu/CaltechAUTHORS:20150213072028236
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1993
DOI: 10.1109/CDC.1993.325855
We show that several applications recently considered in the context of H^∞ filtering and game theory, risk sensitive control and estimation, follow as special cases of the Krein space Kalman filter developed in the companion paper [1]. We show that these problems can be cast into the problem of calculating the stationary points of certain second order forms, and that by considering appropriate state space models and error Gramians, we can use the Krein space Kalman filter to recursively compute these stationary points and to study their properties.https://authors.library.caltech.edu/records/1y5yj9b056LMS is H^∞ optimal
https://resolver.caltech.edu/CaltechAUTHORS:20150219073038887
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1993
DOI: 10.1109/CDC.1993.325187
We show that the celebrated LMS (LeastMean Squares) adaptive algorithm is an H^∞ optimal filter. In other words, the LMS algorithm, which has long been regarded as an approximate leastmean squares solution, is in fact a minimizer of the H^∞ error norm and not the H^2 norm.
In particular, the LMS minimizes the energy gain from the
disturbances to the predicted errors, while the normalized
LMS minimizes the energy gain from the disturbances
to the filtered errors. Moreover, since these algorithms
are central H^∞ filters, they are also risksensitive
optimal and minimize a certain exponential
cost function. We discuss various implications of
these results, and show how they provide theoretical
justification for the widely observed excellent robustness properties of the LMS filter.https://authors.library.caltech.edu/records/q7s6ndjz32SpatioTemporal Blind Identification Of FIR Channels For Multiple Users
https://resolver.caltech.edu/CaltechAUTHORS:20150218072357194
Authors: Hassibi, B.; Aghajan, H.; Khalaj, B.; Paulraj, A.; Kailath, T.
Year: 1994
A new method is proposed for blind identification of possibly nonminimum phase FIR channels with multiple users. The technique exploits the structure of the signals received by an antenna array in both the temporal and spatial frequency domains. Although in the single antenna case it is necessary to use cyclostationary signals or higher order statistics to identify the magnitude and phase of the channel, we circumvent such a requirement by exploiting certain multichannel features of the array. We show that if multiple users are present, the nonminimum phase channels associated with each user can still be identified from the secondorder statistics, provided additional spatial structure exists.https://authors.library.caltech.edu/records/xg5c3dhd35Recursive linear estimation in Krein spaces with applications to H∞ problems,
https://resolver.caltech.edu/CaltechAUTHORS:20150302171722259
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1994
We develop a selfcontained theory for linear estimation in Krein space with applications in several areas such as H∞filtering and control, game problems, risk sensitive control, and adaptive filtering.https://authors.library.caltech.edu/records/ww4zs97r65Optimal Brain Surgeon: Extensions and performance comparison.
https://resolver.caltech.edu/CaltechAUTHORS:20150219074935323
Authors: Hassibi, Babak; Stork, David G.; Wolff, Gregory; Watanabe, Takahiro
Year: 1994
We extend Optimal Brain Surgeon (OBS)  a secondorder method for pruning networks  to allow for general error measures, and explore a reduced computational and storage implementation via a dominant eigenspace decomposition. Simulations on nonlinear, noisy pattern classification problems reveal that OBS does lead to improved generalization, and performs favorably in comparison with Optimal Brain Damage (OBD). We find that the required retraining steps in OBD may lead to inferior generalization, a result that can be interpreted as due to injecting noise back into the system. A common technique is to stop training of a large network at the minimum validation error. We found that the test error could be reduced even further by means of OBS (but not OBD) pruning. Our results justify the t → o approximation used in OBS and indicate why retraining in a highly pruned network may lead to inferior performance.https://authors.library.caltech.edu/records/aj32vgbd85LMS and backpropagation are minimax filters
https://resolver.caltech.edu/CaltechAUTHORS:20150223074318755
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1994
DOI: 10.1007/9781461526964_12
An important problem that arises in many applications is the following adaptive problem: given a sequence of n × 1 input column vectors {h_i}, and a corresponding sequence of desired scalar responses {d_i}, find an estimate of an n × 1 column vector of weights w such that the sum of squared errors, ∑^N_i=0 d_i−h^T_iw^2 , is minimized. The {h_i, d_i } are most often presented sequentially, and one is therefore required to find an adaptive scheme that recursively updates the estimate of w. The leastmeansquares (LMS) algorithm was originally conceived as an approximate solution to the above adaptive problem. It recursively updates the estimates of the weight vector along the direction of the instantaneous gradient of the sum squared error [1]. The introduction of the LMS adaptive filter in 1960 came as a significant development for a broad range of engineering applications since the LMS adaptive linearestimation procedure requires essentially no advance knowledge of the signal statistics. The LMS, however, has been long thought to be an approximate minimizing solution to the above squared error criterion, and a rigorous minimization criterion has been missing.https://authors.library.caltech.edu/records/jmnma7fe88H∞ Optimality Criteria for LMS and Backpropagation
https://resolver.caltech.edu/CaltechAUTHORS:20150302170824137
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1994
We have recently shown that the widely known LMS algorithm
is an H∞ optimal estimator. The H∞ criterion has been
introduced, initially in the control theory literature, as
a means to ensure robust performance in the face of
model uncertainties and lack of statistical information
on the exogenous signals. We extend here our analysis to the nonlinear setting often encountered in neural networks,
and show that the backpropagation algorithm is locally
H∞ optimal. This fact provides a theoretical justification of the widely observed excellent robustness properties
of the LMS and backpropagation algorithms. We further
discuss some implications of these results.https://authors.library.caltech.edu/records/mh8h55d812H^∞ bounds for the recursiveleastsquares algorithm
https://resolver.caltech.edu/CaltechAUTHORS:20150219072113191
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1994
DOI: 10.1109/CDC.1994.411555
We obtain upper and lower bounds for the H^∞ norm
of the RLS (recursiveleastsquares) algorithm. The H^∞
norm may be regarded as the worstcase energy gain from the disturbances to the prediction errors, and is therefore a measure of the robustness of an algorithm to perturbations and model uncertainty. Our results allow one to compare the robustness of RLS compared to the LMS (leastmeansquares) algorithm, which is known to minimize the H^∞ norm. Simulations are presented to show the behaviour of RLS relative to these bounds.https://authors.library.caltech.edu/records/h3z6k9ef74On a closed form solution to the constant modulus factorization problem
https://resolver.caltech.edu/CaltechAUTHORS:20150218075405541
Authors: Hassibi, Babak; Paulraj, Arogyaswami; Kailath, Thomas
Year: 1994
DOI: 10.1109/ACSSC.1994.471567
We consider the problem of separating independent constant modulus signals received by an antenna array. Assuming that the statistics of the phases of the signals are known, we derive a closed form solution for the array response vector from which the original signals can be recovered. Our method is based on estimating the higher order statistics
of the received signals and the estimate of the array response vector is shown to be asymptotically unbiased. Simulation results are included to demonstrate the feasibility of the algorithm.https://authors.library.caltech.edu/records/vf2wmx4f09Adaptive filtering with a H^∞ criterion
https://resolver.caltech.edu/CaltechAUTHORS:20150219073604688
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1994
DOI: 10.1109/ACSSC.1994.471704
H^∞ optimal estimators guarantee the smallest
possible estimation error energy over all possible disturbances of fixed energy, and are therefore robust with respect to model uncertainties and lack of statistical information on the exogenous signals. We have
previously shown that if the prediction error is considered, then the celebrated LMS adaptive filtering algorithm is H^∞ optimal. In this paper we consider prediction of the filter weight vector itself, and for the purpose of coping with timevariations,
exponentially weighted, finitememory and timevarying adaptive filtering. This results in some new adaptive filtering algorithms that may be useful in uncertain and nonstationary environment. Simulation
results are given to demonstrate the feasibility of the algorithm and to compare them with wellknown H^2 (or leastsquares based) adaptive filters.https://authors.library.caltech.edu/records/qxv84bks68Squareroot arrays and Chandrasekhar recursions for H∞ problems
https://resolver.caltech.edu/CaltechAUTHORS:20150219071844925
Authors: Hassibi, Babak; Sayed, Ali H.; Kailath, Thomas
Year: 1994
DOI: 10.1109/CDC.1994.411487
Using their previous observation that H∞ filtering coincides with Kalman filtering in Krein space the authors develop squareroot arrays and Chandrasekhar recursions for H∞ filtering problems. The H∞ squareroot algorithms involve propagating the indefinite squareroot of the quantities of interest and have the property that the appropriate inertia of these quantities is preserved. For systems that are constant, or whose timevariation is structured in a certain way, the Chandrasekhar recursions allow a reduction in the computational effort per iteration from O(n^3) to O(n^2), where n is the number of states. The H∞ squareroot and Chandrasekhar recursions both have the interesting feature that one does not need to explicitly check for the positivity conditions required of the H∞ filters. These conditions are built into the algorithms themselves so that an H∞ estimator of the desired level exists if, and only if, the algorithms can be executed.https://authors.library.caltech.edu/records/xwss43ht77Blind identification of FIR channels with multiple users via spatiotemporal processing
https://resolver.caltech.edu/CaltechAUTHORS:20150219073856149
Authors: Aghajan, H.; Hassibi, B.; Khalaj, B.; Paulraj, A.; Kailath, T.
Year: 1994
DOI: 10.1109/GLOCOM.1994.513200
A new method is proposed for blind identification of possibly nonminimum phase FIR channels with multiple users. The technique exploits the structure of the signals received by an antenna array in both the temporal and spatial frequency domains. Although in the single
antenna case it is necessary to use cyclostationary signals or higher order statistics to identify the magnitude and phase of the channel, the present authors circumvent such a requirement by exploiting certain multichannel features of the array. They show that if multiple users are
present, the nonminimum phase channels associated with each user can still be identified from the secondorder statistics, provided additional spatial structure exists.https://authors.library.caltech.edu/records/v3fxtx2k15H∞ Adaptive Filtering
https://resolver.caltech.edu/CaltechAUTHORS:20150218074309000
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1995
DOI: 10.1109/ICASSP.1995.480332
H∞ optimal estimators guarantee the smallest possible estimation error energy over all possible disturbances of fixed energy, and are therefore robust with respect to model uncertainties and lack of statistical information on the exogenous signals. We have shown that if the prediction error is considered, then the celebrated LMS adaptive filtering algorithm is H∞ optimal. We consider prediction of the filter weight vector itself, and for the purpose of coping with timevariations, exponentially weighted, finitememory and timevarying adaptive filtering. This results in some new adaptive filtering algorithms that may be useful in uncertain and nonstationary environments. Simulation results are given to demonstrate the feasibility of the algorithms and to compare them with wellknown H^2 (or leastsquares based) adaptive filters.https://authors.library.caltech.edu/records/aygn3amx16H^∞ Optimal Training Algorithms and their Relation to Backpropagation
https://resolver.caltech.edu/CaltechAUTHORS:20150218074822311
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1995
We derive global H^∞ optimal training algorithms for neural networks. These algorithms guarantee the smallest possible prediction error energy over all possible disturbances of fixed energy, and are therefore robust with respect to model uncertainties and lack of statistical information on the exogenous signals. The ensuing estimators are infinitedimensional, in the sense that updating the weight vector estimate requires knowledge of all previous weight esimates. A certain finitedimensional approximation to these estimators is the backpropagation algorithm. This explains the local H6∞ optimality of backpropagation that has been previously demonstrated.https://authors.library.caltech.edu/records/gqzyx18n23Fundamental inertia conditions for the solution of H^∞problems
https://resolver.caltech.edu/CaltechAUTHORS:20150219071332166
Authors: Sayed, Ali H.; Hassibi, Babak; Kailath, Thomas
Year: 1995
DOI: 10.1109/ACC.1995.532764
We study the relation between the solutions
of two minimization problems with indefinite quadratic forms. We show that a complete link between both solutions can be established by invoking a fundamental set of inertia conditions. While these inertia conditions are automatically satisfied in a standard Hilbert space setting, they nevertheless turn out to mark the differences between the two optimization problems in indefinite metric spaces. They also include, as special cases, the wellknown conditions for the existence of H^∞filters and controllers.https://authors.library.caltech.edu/records/5qvnxfed82Mixed H^2/H^∞ Estimation: Preliminary Analytic Characterization And A Numerical Solution
https://resolver.caltech.edu/CaltechAUTHORS:20150218075106274
Authors: Halder, B.; Hassibi, B.; Kailath, T.
Year: 1996
We introduce and motivate the problem of mixed H^2/H∞ estimation by studying the stochastic and deterministic approaches of H^2 and H^∞ estimation. Mixed H^2/H^∞ estimators have the property that they have the best average performance over all estimators that achieve a certain worstcase performance bound. They thus allow a tradeoff between average and worstcase performances. In the finite horizon case, we obtain a numerical solution (based on convex optimization methods) for the optimal mixed H^2/H^∞ estimator. We also give some analytic characterizations, both on this optimal solution, and on the set of all estimators achieving a guaranteed worstcase bound. A numerical example is also provided.https://authors.library.caltech.edu/records/9x1893va02Inertia conditions for the minimization of quadratic forms in indefinite metric spaces
https://resolver.caltech.edu/CaltechAUTHORS:20150223074538319
Authors: Sayed, Ali H.; Hassibi, Babak; Kailath, Thomas
Year: 1996
DOI: 10.1007/9783034890359_15
Using results on indefinite metric space theory, two minimization problems are considered. Under a fundamental set of inertia conditions, a complete link between both solutions can be established. A very nice translation of prediction and filtering notions in the language of indefinite metric notions can be found. Applications to H^∞ filtering and approximate total least squares methods are presented.https://authors.library.caltech.edu/records/s17tv1h838Statespace structure of finite horizon optimal mixed H_2/H_∞ filters
https://resolver.caltech.edu/CaltechAUTHORS:20150302161737951
Authors: Halder, B.; Hassibi, B.; Kailath, T.
Year: 1997
DOI: 10.1109/ACC.1997.611909
We show that finite horizon optimal mixed H_2/H_∞ filters are not of fixed order. Moreover, when the underlying systems have statespace models of order n, the optimal finite horizon mixed H_2/H_∞, filter has statespace model of order no greater than n+J where J is the multiplicity of the maximum singular value of T_2(K) equal to the H_∞ bound γ.https://authors.library.caltech.edu/records/4ghv0nja86On nonlinear filters for mixed H^2/H^∞ estimation
https://resolver.caltech.edu/CaltechAUTHORS:20150302162239004
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1997
DOI: 10.1109/ACC.1997.611970
We study the problem of mixed leastmeansquares H^∞ optimal (or mixed H^2/H^∞optimal) estimation of signals generated by discretetime, finitedimensional, linear statespace models. The major result is that, for finitehorizon problems, and when the stochastic disturbances have Gaussian distributions, the optimal solutions have finitedimensional (i.e., boundedorder) nonlinear statespace structure of order 2n+1 (where n is the dimension of the underlying statespace model). Being nonlinear, the filters do not minimize an H2 norm subject to an H^∞ constraint, but instead minimize the leastmeansquares estimation error (given a certain a priori probability distribution on the disturbances) subject to a given constraint on the maximum energy gain from disturbances to estimation errors. The mixed filters therefore have the property of yielding the best average (leastmeansquares) performance over all filters that achieve a certain worstcase (H^∞) boundhttps://authors.library.caltech.edu/records/z7x9wxgf38On adaptive filtering with combined leastmeansquares and H^∞ criteria
https://resolver.caltech.edu/CaltechAUTHORS:20150218070609299
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1997
DOI: 10.1109/ACSSC.1997.679167
We study the possibility of combining leastmeansquares, or
stochastic, performance with H^∞optimal, or worstcase,
performance in adaptive filtering. The resulting adaptive algorithms allow for a tradeoff between average and worstcase performances and are most applicable in situations, such as mobile communications, where,
due to modeling errors and rapid timevariation of system parameters, the exact statistics and distributions of the underlying signals are not known. We mention some of the open problems in this field, and construct a nonlinear adaptive filter (requiring O(n^2) operations per
iteration, where n is the number of filter weights) that recursively minimizes the leastmeansquares error over all filters that guarantee a specified worstcase H^∞ bound. We also present some simple examples to compare the algorithm's behaviour with standard leastsquares and H^∞ adaptive filters.https://authors.library.caltech.edu/records/j50bppxg29Mixed leastmeansquares/H^∞optimal adaptive filtering
https://resolver.caltech.edu/CaltechAUTHORS:20150218073848656
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1997
DOI: 10.1109/ACSSC.1996.600941
We construct a socalled mixed leastmean squares/H∞optimal (or mixed H^2/H^∞optimal) algorithm for adaptive filtering. The resulting adaptive algorithm is nonlinear and requires O(n^2) (where n is the number of filter weights) operations per iteration. Such mixed algorithms have the properly of yielding the best average (leastmeansquares) performance over all algorithms that achieve a certain worstcase (H^∞optimal) bound. They thus allow a tradeoff between average and worstcase performance and are most applicable in situations where the exact statistics and distributions of the underlying signals are not known. Simple simulations are also presented to compare the algorithm's behaviour with standard leastsquares and H^∞ adaptive filters.https://authors.library.caltech.edu/records/zqpxwt7611Tracking with an H^∞ criterion
https://resolver.caltech.edu/CaltechAUTHORS:20150218070205310
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1997
DOI: 10.1109/CDC.1997.652411
We study the problem of tracking a reference signal from the
H∞ point of view. As opposed to general H^∞
problems, where only suboptimal solutions are obtained, we show that, for both the full information and measurement feedback tracking problems the H^∞optimal solutions can be explicitly found. The results also indicate an interesting dichotomy between minimum phase and nonminimum phase plants: for minimum phase plants the best causal tracker performs as well as the best noncausal tracker,
whereas for nonminimum phase plants, causal trackers cannot reduce the H^∞ norms from their a priori values. We also mention some remedies for the nonminimum phase case, such as adding more actuators (control inputs) or allowing for some finite delay. For causal tracking of nonminimum phase plants, we show that a delay equal to at
least the number of nonminimum phase zeros of the plant is required.https://authors.library.caltech.edu/records/p1b16br891Linearly Combined Suboptimal Mixed H_2/H_∞ Controllers
https://resolver.caltech.edu/CaltechAUTHORS:20150218071918325
Authors: Halder, B.; Hassibi, B.; Kailath, T.
Year: 1997
DOI: 10.1109/CDC.1997.650663
In this paper we consider the problem of minimizing the H_2 norm of a closedloop map over all static state feedback controllers while satisfying an H_∞ constraint on another closedloop map. We propose a readily computable suboptimal solution to the pure mixed H_2/H_∞ problem by restricting the search to a class of linearly combined controllers. Such mixed linearly combined controllers yield smaller closedloop H_2 norms than those obtained by using the central solutions of the H_∞ problem. Moreover, the mixed controllers achieve the optimal H2 performance whenever the optimal H_2 controller satisfies the H_∞ bound.https://authors.library.caltech.edu/records/w2e1mqm107H^∞ equalization of communication channels
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958476
Authors: Erdogan, Alper T.; Hassibi, Babak; Kailath, Thomas
Year: 1998
DOI: 10.1109/icassp.1998.679618
As an alternative to existing techniques and algorithms, we investigate the merit of the Hinfinity approach to the equalization of communication channels. We first look at causal Hinfinity equalization problem and then look at the improvement due to finite delay. By introducing the risk sensitive property, we compare the average performance of the central Hinfinity equalizer with the MMSE equalizer in equalizing minimum phase channels.https://authors.library.caltech.edu/records/bvhjrnv046On Optimal Solutions to TwoBlock H∞ Problems
https://resolver.caltech.edu/CaltechAUTHORS:20150218072113882
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1998
DOI: 10.1109/ACC.1998.707368
In this paper we obtain a new formula for the minimum achievable disturbance attenuation in twoblock H∞ problems.
This new formula has the same structure as the optimal H∞ norm formula for noncausal problems, except that doubly infinite (socalled Laurent) operators must be replaced by
semiinfinite (socalled Toeplitz) operators. The benefit of the new formula is that it allows us to find explicit expressions for the optimal H∞ norm in several important cases: the equalization problem (or its dual, the tracking problem), and the problem of filtering signals in additive noise. Furthermore, it leads us to the concepts of "worstcase nonestimability", corresponding to when causal filters cannot reduce the H∞ norms from their a priori values, and "worstcase complete estimability", corresponding to when causal filters offer the same H∞ performance as noncausal ones. We also obtain an explicit characterization of worstcase nonestimability and study the consequences to the problem of equalization with finite delay.https://authors.library.caltech.edu/records/swprhwcd26Multiobjective H_2/H_∞optimal control via finite dimensional Qparametrization and linear matrix inequalities
https://resolver.caltech.edu/CaltechAUTHORS:20150217073735707
Authors: Hindi, Haitham A.; Hassibi, Babak; Boyd, Stephen P.
Year: 1998
DOI: 10.1109/ACC.1998.688463
The problem of multiobjective H_2/H_∞
optimal controller design is reviewed. There is as yet no exact solution to this problem. We present a method based on that proposed by Scherer (1995). The problem is formulated as a convex semidefinite program (SDP)
using the LMI formulation of the H_2 and H_∞ norms. Suboptimal solutions are computed using finite dimensional
Qparametrization. The objective value of the suboptimal Q's converges to the true optimum as the dimension of and is increased. State space representations are presented which are the analog of those given by
Khargonekar and Rotea (1991) for the H_2 case. A simple
example computed using finite impulse response Qs is presented.https://authors.library.caltech.edu/records/qzdabt0664Design of optimal mixed H_2/H_∞ static state feedback controllers
https://resolver.caltech.edu/CaltechAUTHORS:20150302070217718
Authors: Halder, B.; Hassibi, B.; Kailath, T.
Year: 1998
DOI: 10.1109/ACC.1998.688462
Despite advances in robust control theory, the robust performance problem formulated in the mixed H_2/H_∞
framework largely remains an open problem. In this approach, one seeks a controller that minimizes the H_2 norm of a closedloop map over all admissible controllers while satisfying an H_∞ constraint on another closedloop map. Unlike the optimal H_2 problem or the γlevel suboptimal H_∞ problem, the mixed H_2/H_∞ problem does not have a readily computable solution. In the paper we restrict consideration to static state feedback controllers and propose an efficient iterative algorithm for computing the optimal H_2/H_∞ solution.https://authors.library.caltech.edu/records/xfwcyveb47An estimationbased approach to the design of adaptive IIR filters
https://resolver.caltech.edu/CaltechAUTHORS:20150217073428156
Authors: Sayyarrodsari, Bijan; How, Jonathan P.; Hassibi, Babak; Carrier, Alain
Year: 1998
DOI: 10.1109/ACC.1998.688442
We present an estimationbased approach to the design of adaptive IIR filters. We also use this approach to design adaptive filters when a feedback signal from the output of the adaptive filter contaminates the reference signal. We use an H∞ criterion to cast the problem as a nonlinear H∞ filtering problem, and present an approximate linear H∞ filtering solution. This linear filtering solution is then used to adapt the adaptive IIR Filter. The presentation of the proposed adaptive algorithm is done in the context of an adaptive active noise cancellation problem. Simulations are used to examine the performance of the proposed estimationbased adaptive algorithm.https://authors.library.caltech.edu/records/0kpmy7nf03An H_∞optimal alternative to the FxLMS algorithm
https://resolver.caltech.edu/CaltechAUTHORS:20150217073042063
Authors: Sayyarrodsari, Bijan; How, Jonathan P.; Hassibi, Babak; Carrier, Alain
Year: 1998
DOI: 10.1109/ACC.1998.703585
We study a general setting of active noise cancellation problems from the H_∞ point of view and present a solution that optimally limits the worst case energy gain from the interfering measurement errors, external disturbances, and initial condition uncertainty to the residual noise. The optimal bounding of this energy gain is the main characteristic of the proposed solution. To impose a
finite impulse response structure on the controller, we suggest an adaptation scheme for the weight vector of an FIR filter that approximates the H_∞optimal solution. Our discussions explain: 1) why and how this new adaptive scheme generalizes previous results on the H_∞optimality of the LMS algorithm; 2) why it is an alternative to the widely used filteredX leastmeansquares (FxLMS) algorithm; and 3) how the formulation
provides an appropriate framework to address the issues of modeling error and robustness. Simulations are used to compare the performance of the proposed H_∞optimal adaptive scheme with the FxLMS algorithm.https://authors.library.caltech.edu/records/2rw35vyn70Equalization with an H^∞ criterion
https://resolver.caltech.edu/CaltechAUTHORS:20150302065953699
Authors: Hassibi, Babak; Erdogan, Alper T.; Kailath, Thomas
Year: 1998
DOI: 10.1109/ISIT.1998.709054
We study the problem of linear equalization from an H^∞
point of view and parameterize, in closed form, all possible H^∞optimal equalizers. The results indicate an interesting dichotomy between minimum phase and nonminimum phase channels: for minimum phase channels the best causal equalizer performs as well as the best noncausal equalizer, whereas for nonminimum phase channels, causal equalizers cannot reduce the estimation error bounds from their a priori values.https://authors.library.caltech.edu/records/gcd9d20y30Upper bounds for mixed H^2/H^∞ control
https://resolver.caltech.edu/CaltechAUTHORS:20150217075137959
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1998
DOI: 10.1109/CDC.1998.760757
We consider the mixed H^2/H^∞ control problem of choosing a controller to minimize the H^2 norm of a given closedloop map, subject to the H^∞ norm of another closedloop map being less than a prescribed value γ. Let d_2 and γ_2 denote the H^2 and H^∞ norms for the pure H^2optimal solution (without any H^∞ constraint), and let d_c and
γ_c < γ denote the H^2 and H^∞ norms for any solution that yields an H^∞ norm strictly less than γ (such as, say, the central solution). Then if d_m denotes the optimal H2 norm
that can be achieved in the mixed H^2/H^∞ control problem, we show that (d^2_m  d^2_2)/(d^_c  d^2_2) ⩽ ((γ_2  γ)/(γ_2  γ_c))^2 < ((γ^2_2  γ^2)/(γ^2_2  γ^2_c))^2 < 1.https://authors.library.caltech.edu/records/f57hsd7831On robust twoblock problems
https://resolver.caltech.edu/CaltechAUTHORS:20150218070415882
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1998
DOI: 10.1109/CDC.1998.758669
In this paper we consider the following robust twoblock problem that arises in estimation and in fullinformation control: minimize the worstcase H^∞ norm of a twoblock transfer matrix whose elements contain H∞normbounded modeling errors. We show that, when the underlying systems are singleinput/singleoutput, and if the modeling errors are "small enough", then the robust, twoblock problem can be solved by solving a onedimensional family of appropriatelyweighted "modelingerrorfree" twoblock problems. We also study the consequences of this result to a robust tracking problem, where the optimal solution can be explicitly found.https://authors.library.caltech.edu/records/awjncm0f84H^∞optimality of H^2 predictors
https://resolver.caltech.edu/CaltechAUTHORS:20150302070441915
Authors: Hassibi, Babak; Kailath, Thomas
Year: 1998
DOI: 10.1109/CDC.1998.760752
Given past observations of a process, {y_j,jhttps://authors.library.caltech.edu/records/z7qe66de25An LMI formulation for the estimationbased approach to the design of adaptive filters
https://resolver.caltech.edu/CaltechAUTHORS:20150217074721805
Authors: Sayyarrodsari, Bijan; How, Jonathan P.; Hassibi, Babak; Carrier, Alain
Year: 1998
DOI: 10.1109/CDC.1998.760610
We present a linear matrix inequalities (LMI) formulation for the estimationbased approach to the design of adaptive FIR and IIR filters. LMI provide a convenient framework for the synthesis of multiobjective (H_2/H∞) control problems. Therefore, the H∞ disturbance attenuation criterion in the
estimationbased adaptive algorithm can be easily augmented with the appropriate H2 performance constraints. The question of internal stability of the overall system is also directly addressed as a byproduct of the Lyapunovbased nature of the LMI formulation. We use
an active noise cancellation scenario to study the main features of the proposed LMI solution.https://authors.library.caltech.edu/records/3b00w66q70On H^∞ optimal signal reconstruction in noisy filter banks
https://resolver.caltech.edu/CaltechAUTHORS:20150227074916986
Authors: Vikalo, H.; Hassibi, B.; Kailath, T.
Year: 1999
DOI: 10.1109/ICASSP.1999.756266
We study the design of synthesis filters in noisy filter bank systems using an H^∞ point of view. For unitary analysis polyphase matrices we obtain an explicit expression for the minimum achievable disturbance attenuation. Numerical examples and comparisons with existing methods are also included.https://authors.library.caltech.edu/records/x2fsayw085Kalman Filters
https://resolver.caltech.edu/CaltechAUTHORS:20150227074717012
Authors: Kailath, Thomas; Sayed, Ali H.; Hassibi, Babak
Year: 1999
DOI: 10.1002/047134608X.W7210
[no abstract]https://authors.library.caltech.edu/records/32k5pzpx04Estimationbased synthesis of H_∞optimal adaptive equalizers over wireless channels
https://resolver.caltech.edu/CaltechAUTHORS:20150227075402686
Authors: MalekiTehrani, Ardavan; Sayyarrodsari, Bijan; Hassibi, Babak; How, Jonathan P.; Cioffi, John M.
Year: 1999
DOI: 10.1109/GLOCOM.1999.831681
This paper presents a systematic synthesis procedure for
H_∞optimal adaptive FIR equalizers over a timevarying
wireless channel. The channel is assumed to be frequency selective with Rayleigh fading. The proposed equalizer structure consists of the series connection of an adaptive FIR filter and a fixed equalizer (designed for
the nominal channel). Adaptation of the weight vector of the adaptive FIR filter is achieved using the H_∞optimal solution of an estimationbased interpretation of the channel equalization problem. Due to its H_∞optimality, the proposed solution is robust to exogenous disturbances, and enables fast adaptation (i.e., a short training period) without compromising the steadystate performance
of the equalization. Preliminary simulation are presented to support the above claims.https://authors.library.caltech.edu/records/swvvgyv260Array algorithms for H^2 and H^∞ estimation
https://resolver.caltech.edu/CaltechAUTHORS:20150223071202056
Authors: Hassibi, Babak; Kailath, Thomas; Sayed, Ali H.
Year: 1999
DOI: 10.1007/9781461205715_2
Currently, the preferred method for implementing H^2 estimation algorithms is what is called the array form, and includes two main families: squareroot array algorithms, that are typically more stable than conventional ones, and fast array algorithms, which, when the system is timeinvariant, typically offer an order of magnitude reduction in the computational effort. Using our recent observation that H^∞ filtering coincides with Kalman filtering in Krein space, in this chapter we develop array algorithms for H^∞ filtering. These can be regarded as natural generalizations of their H^2 counterparts, and involve propagating the indefinite square roots of the quantities of interest. The H^∞ squareroot and fast array algorithms both have the interesting feature that one does not need to explicitly check for the positivity conditions required for the existence of H^∞ filters. These conditions are built into the algorithms themselves so that an H^∞ estimator of the desired level exists if, and only if, the algorithms can be executed. However, since H^∞ squareroot algorithms predominantly use Junitary transformations, rather than the unitary transformations required in the H^2 case, further investigation is needed to determine the numerical behavior of such algorithms.https://authors.library.caltech.edu/records/rmd00b6t44Adaptive equalization of multipleinput multipleoutput (MIMO) frequency selective channels
https://resolver.caltech.edu/CaltechAUTHORS:20150217072128821
Authors: MalekiTehrani, Ardavan; Hassibi, Babak; Cioffi, John M.
Year: 1999
DOI: 10.1109/ACSSC.1999.832390
The purpose of this paper is to propose and investigate a new approach to adaptive spatiotemporal equalization for MIMO (multipleinput multipleoutput) channels. A system with n transmit and m (n≥m) receiver antennas is assumed. An adaptive MIMO linear equalizer has been considered.
For the considered equalizer a least squares solution is formulated, based on which a recursive solution using Riccati recursions is proposed. The solutions are tested by simulating the MIMO system. It is shown that the adaptive solutions will achieve the same performance as the optimum least squares solutions. The effect of the nondiagonal channel elements (acting as interference) on the system performance is also studied. It has been shown that in order to achieve better performance, the interference from nondiagonal channel elements needs to be minimized. This can be done by using orthogonal transmission. Moreover the proposed solutions do not require channel identification and will also enable equalizer adaptation to channel changes.https://authors.library.caltech.edu/records/m0zsxwj840Spacetime autocoding: arbitrarily reliable communication in a single fading interval
https://resolver.caltech.edu/CaltechAUTHORS:20150227073515010
Authors: Marzetta, Thomas L.; Hochwald, Bertrand; Hassibi, Babak
Year: 2000
DOI: 10.1109/ISIT.2000.866611
Prior treatments of spacetime communications in Rayleigh flat fading generally assume that channel coding covers either one fading intervalin which case there is a nonzero "outage capacity"or multiple fading intervalsin which case there is a nonzero Shannon capacity. However, we establish conditions under which channel codes span only one fading interval and yet are arbitrarily
reliable. In short, spacetime signals are their own channel codes. We call this phenomenon spacetime autocoding, and the accompanying capacity the spacetime autocapacity.https://authors.library.caltech.edu/records/53nxze3k86Optimal training in spacetime systems
https://resolver.caltech.edu/CaltechAUTHORS:20150213072339075
Authors: Hassibi, Babak; Hochwald, Bertrand
Year: 2000
DOI: 10.1109/ACSSC.2000.911051
Multipleantenna wireless communication links promise very high data rates with low error probabilities, especially when the wireless channel response is known at the receiver. In practice, knowledge of the channel is often obtained by sending known training symbols to the receiver. We show how training affects the capacity of a fading channeltoo little training and the channel is improperly learned too much training and there is no time left for data transmission before the channel changes. We use an informationtheoretic approach to compute the optimal amount of training as a function of the received signaltonoise ratio, fading coherence time, and number of transmitter antennas. When the training and data powers are allowed to vary, we show that the optimal number of training symbols is equal to the number of transmit antennasthis number is also the smallest training interval length that guarantees meaningful estimates of the channel matrix. When the training and data powers are instead required to be equal, the optimal number of symbols may be larger than the number of antennas. We further conclude that at high SNR trainingbased schemes can capture most of the channel capacity, whereas at low SNR they can be highly suboptimal.https://authors.library.caltech.edu/records/0k04jm1s71On robust multiuser detection
https://resolver.caltech.edu/CaltechAUTHORS:20150213074424107
Authors: Vikalo, Haris; Hassibi, Babak; Kailath, Thomas
Year: 2000
DOI: 10.1109/ACSSC.2000.910698
We study the design of multiuser detectors from an H∞ point of view. The H∞ approach is most appropriate in the situations where the statistical properties of the disturbances are not known or are too hard to model and analyze. The design of the H∞ optimal FIR multiuser detectors can be efficiently performed using numerical methods. Exploiting the inherent nonuniqueness of the H∞ solution, we additionally optimize for an average performance thus obtaining mixed H^2/H∞ optimal multiuser detector. Recursive solutions, allowing for computationally efficient implementation of the decisionfeedback detectors, is briefly discussed.https://authors.library.caltech.edu/records/jth0nz4156Multiple antennas and representation theory
https://resolver.caltech.edu/CaltechAUTHORS:20150227073943509
Authors: Hassibi, Babak; Hochwald, Bertrand; Shokrollahi, Amin; Sweldens, Wim
Year: 2000
DOI: 10.1109/ISIT.2000.866635
Multiple antennas can greatly increase the data rate and
reliability of a wireless communication link in a fading environment, but the practical success of using multiple antennas depends crucially on our ability to design highrate spacetime constellations with low
encoding and decoding complexity. It has been shown that full transmitter diversity, where the constellation is a set of unitary matrices whose differences have nonzero
determinant, is a desirable property for good performance.
We use the powerful theory of fixedpointfree groups and their representations to design highrate
constellations with full diversity. Furthermore, we thereby classify all fulldiversity constellations that form a group, for all rates and numbers of transmitter antennas. The group structure makes the constellations especially suitable for differential modulation and
lowcomplexity decoding algorithms.
The classification also reveals that the number of different group structures with full diversity is very limited when the number of transmitter antennas is large and odd. We therefore also consider extensions of the constellation designs to nongroups. We conclude by showing that many of our designed
constellations perform excellently on both simulated and real wireless channels.https://authors.library.caltech.edu/records/4nvxcjfq28Mixed H^2/H^∞ optimal signal reconstruction in noisy filter banks
https://resolver.caltech.edu/CaltechAUTHORS:20150227073327948
Authors: Vikalo, Haris; Hassibi, Babak; Kailath, Thomas
Year: 2000
DOI: 10.1109/ICASSP.2000.862027
We study the design of synthesis filters in noisy filter bank systems using an H^∞ estimation point of view. The
H^∞ approach is most promising in situations where the
statistical properties of the disturbances (arising from quantization, compression, etc.) in each subband of the filter bank are unknown, or are too difficult to model and analyze. For arbitrary analysis polyphase matrices, standard statespace H∞ techniques can be
employed to obtain numerical solutions. When the synthesis filters are restricted to being FIR, as is often the case in practice, the design can be cast as a finitedimensional semidefinite program. In this case, we can effectively exploit the inherent nonuniqueness of the
H∞ solution to optimize for an additional average
performance and thus obtain mixed H^2/H^∞
optimal FIR synthesis filters.https://authors.library.caltech.edu/records/27x5cyt607FIR H∞ equalization
https://resolver.caltech.edu/CaltechAUTHORS:20150218162033552
Authors: Erdogan, Alper T.; Hassibi, Babak; Kailath, Thomas
Year: 2000
DOI: 10.1109/ICASSP.2000.861055
We approach FIR equalization problem from an H∞ perspective.
First, we formulate the calculation of the optimal H∞ performance for a given equalization setting as a semidefinite programming (SDP) problem. H∞ criterion provides a set of FIR equalizers with different optimality properties.
Among
these,
we
formulate
the
calculation
of risk sensitive
or
minimum
entropy
FIR
filter
as
the
constrained analytic centring
problem
and
mixed
H2/H"
problem
as
another
SDP. We
provide an
example
to
il
lustrate the
procedures
we
described.https://authors.library.caltech.edu/records/r2pyx1ew66Estimationbased multichannel adaptive algorithm for filteredLMS problems
https://resolver.caltech.edu/CaltechAUTHORS:20150213073143336
Authors: Sayyarrodsari, Bijan; How, Jonathan P.; Hassibi, Babak; Carrier, Alain
Year: 2000
DOI: 10.1109/ACC.2000.879154
This paper presents an estimationbased adaptive
filtering algorithm for the multichannel FilteredLMS
problems where a number of adaptively controlled secondary
sources use multiple reference signals to cancel the effect
of a number of primary sources (i.e. disturbance sources)
as seen by a number of error sensors. We show that our
estimation based approach easily extends to the multichannel case, and that it maintains all of the stability and
performance features of the singlechannel solution.
The problem of noise cancellation in a one dimensional acoustic duct, and a structural vibration control problem
are chosen to examine the main characteristics of the proposed multichannel adaptive algorithm. The
performance of the new multichannel adaptive algorithm is compared to the performance of a multichannel implementation of the FAMS algorithm in these cases, and it is shown that the new algorithm provides a faster response, with improved transient behavior and steadystate performance.https://authors.library.caltech.edu/records/by8cvd3g25Codes for differential signaling with many antennas
https://resolver.caltech.edu/CaltechAUTHORS:20150227073723026
Authors: Hassibi, Babak; Hochwald, Bertrand; Shokrollahi, Amin; Sweldens, Wim
Year: 2000
DOI: 10.1109/WCNC.2000.904593
We construct signal constellations for differential transmission with multiple basestation antennas. The signals are derived using the theory of fixedpointfree groups and are especially suitable for mobile
cellular applications because they do not require the handset to have more than one antenna or to know the timevarying propagation environment. Yet we achieve full transmitter diversity and excellent performance gains over a singleantenna system.https://authors.library.caltech.edu/records/frr2h57z93An estimationbased approach to multipleinput multipleoutput (MIMO) channel equalization
https://resolver.caltech.edu/CaltechAUTHORS:20150213072636206
Authors: Tehrani, Ardavan Maleki; Hassibi, Babak; Cioffi, John
Year: 2000
DOI: 10.1109/SAM.2000.877961
The purpose of this paper is to propose and investigate a new approach to implementing a spatiotemporal decision feedback equalizer (DFE) for MIMO (multipleinput multipleoutput) channels. A system with an array of n transmit and m receiver antennas where (m ≥ n) is assumed. Both finitelength (finite horizon) and infinitelength
(infinite horizon) MIMO decision feedback equalizers are considered. We also assume an ISI (intersymbolinterference) MIMO channel, which means
the channel matrix elements are frequency selective.
For the infinitelength case the DFE problem leads to solving a matrix spectral factorization. For the finitelength case the DFE problem leads to solving a corresponding Cholesky factorization.
Using the estimationbased spectral factorization we have shown that the solution to the infinitelength MIMO DFE is not unique. In the finitelength case the estimationbased approach leads to a recursive algorithm to perform
the Cholesky factorization. The proposed recursive algorithm has low complexity and is also simple to implement. Moreover it leads to a
closed form solution for the MIMO DFE matrices.https://authors.library.caltech.edu/records/f58be0nf98An efficient squareroot algorithm for BLAST
https://resolver.caltech.edu/CaltechAUTHORS:20150213075135707
Authors: Hassibi, Babak
Year: 2000
DOI: 10.1109/ICASSP.2000.859065
Bell Labs Layered SpaceTime (BLAST) is a scheme for transmitting information over a richscattering wireless environment using multiple receive and transmit antennas. The main computational bottleneck in the
BLAST algorithm is a "nulling and cancellation" step, where
the optimal ordering for the sequential estimation and detection of the received signals is determined. To reduce the computational cost of BLAST, we develop an efficient squareroot algorithm for the nulling and cancellation step. The main features of the algorithm include
efficiency: the computational cost is reduced by 0.7 M, where M is the number of transmit antennas, and numerical stability: the algorithm is divisionfree and uses only orthogonal transformations. In a 14 antenna
system designed for transmission of 1 Mbit/s over a 30 kHz channel, the nulling and cancellation computation is reduced from 190 MFlops/s to 19 MFlops/s, with the overall computations being reduced from 220 MFlops/s
to 49 MFlops/s. The numerical stability of the algorithm also make it attractive for implementation in fixedpoint (rather than floatingpoint) architectures.https://authors.library.caltech.edu/records/a3qmqhye34Adaptive equalization of multipleinput multipleoutput (MIMO) channels
https://resolver.caltech.edu/CaltechAUTHORS:20150213074719604
Authors: MalekiTehrani, Ardavan; Hassibi, Babak; Cioffi, John M.
Year: 2000
DOI: 10.1109/ICC.2000.853778
The paper proposes and investigates a new approach to adaptive spatiotemporal equalization for MIMO (multipleinput multipleoutput) channels. A system with n transmit and m (m≥n) receiver antennas is
assumed.
A decision Feedback equalizer is considered. A least squares
solution is first formulated, based on which a recursive solution using Riccati recursions is proposed. The proposed solution is tested by simulating the MIMO system. It is shown that the adaptive solution achieves the same performance as the optimum least squares solution. The
effect of the nondiagonal channel elements (acting as interference) on the system performance is also studied. It has been shown that in order to achieve better performance, the interference from nondiagonal channel elements needs to be minimized. This can be done by using orthogonal
transmission. Moreover the proposed solution do not require channel identification and will also enable equalizer adaptation to channel changes.https://authors.library.caltech.edu/records/7wq1vatp94A fast squareroot implementation for BLAST
https://resolver.caltech.edu/CaltechAUTHORS:20150213075410053
Authors: Hassibi, Babak
Year: 2000
DOI: 10.1109/ACSSC.2000.910764
Bell Labs Layered SpaceTime (BLAST) is a scheme for transmitting information over a richscattering wireless environment using multiple receive and transmit antennas. The main computational bottleneck in the BLAST algorithm is a "nulling and cancellation" step, where the optimal ordering for the sequential estimation and detection of the received signals is determined. To reduce the computational cost of BLAST we develop an efficient squareroot algorithm for the nulling and cancellation step. The main features of the algorithm include efficiency: the computational cost is reduced by 0.7M, where M is the number of transmit antennas, and numerical stability; and the algorithm is divisionfree and uses only orthogonal transformations. In a 14 antenna system designed for transmission of 1 Mbit/sec over a 30 kHz channel, the nulling and cancellation computation is reduced from 190 MFlops/sec to 19 MFlops/sec, with the overall computations being reduced from 220 MFlops/sec to 49 MFlops/sec. The numerical stability of the algorithm also make it attractive for implementation in fixedpoint (rather than floatingpoint) architectures.https://authors.library.caltech.edu/records/bswam6zx63Exponentialquadratic optimal signal reconstruction in noisy filter banks
https://resolver.caltech.edu/CaltechAUTHORS:20150227072806962
Authors: Vikalo, Haris; Erdogan, Alper T.; Hassibi, Babak
Year: 2000
DOI: 10.1117/12.408589
We consider the design of synthesis filters in noisy filter bank system, using an exponentialquadratic criterion. We assume that the analysis filters have been design to achieve good coding of the input signal.
Then we design the synthesis filters to minimize reconstruction error according to the adopted criterion. When the synthesis filters are restricted to be FIR, the design can be cast as a constraint analytic
centering problem. To this end, we first employ standard statespace techniques to obtain a set of H(infinity) optimal FIR synthesis filters. Among these, we select the socalled risksensitive synthesis filters by
additionally minimizing exponentialquadratic cost function. We provide numerical examples to illustrate the procedure.https://authors.library.caltech.edu/records/wrzge28m88Spacetime autocoding constellations with pairwiseindependent signals
https://resolver.caltech.edu/CaltechAUTHORS:20150227072531001
Authors: Marzetta, Thomas L.; Hassibi, Babak; Hochwald, Bertrand M.
Year: 2001
DOI: 10.1109/ISIT.2001.936189
The spacetime autocoding effect implies that arbitrarily
reliable communication is possible within a single
coherence interval in Rayleigh flat fading as the
symbolduration of the coherence interval and the
number of transmit antennas grow simultaneously. For
relatively short (e.g., 16symbol) coherence intervals,
a codebook of isotropically random unitary spacetime
signals theoretically supports transmission rates
that are a significant fraction of autocapacity with
an extremely low probability of error. However
a constellation of the required size (typically L = 280)
is impossible to generate and store, and due to lack
of structure there is little hope of finding a fast
decoding scheme. We propose a random, but highly structured,
constellation that is completely specified by log,
L independent isotropically distributed unitary matrices.
The distinguishing property of this construction
is that any two signals in the constellation are
pairwise statistically independent and isotropically
distributed. Thus, the pairwise probability of error,
and hence the union bound on the block probability
of error, of the structured constellation is identical
to that of a fully random constellation of independent
signals.https://authors.library.caltech.edu/records/02mhz4q181Optimal training for frequencyselective fading channels
https://resolver.caltech.edu/CaltechAUTHORS:20150212075427673
Authors: Vikalo, H.; Hassibi, B.; Hochwald, B.; Kailath, T.
Year: 2001
DOI: 10.1109/ICASSP.2001.940408
Many communications systems employ training, ie, the transmission of known signals, so that the channel parameters may be learned at the receiver. This has a dual effect: too little training and the channel is improperly learned, too much training and there is no time left for data transmission before the channel changes. We use an informationtheoretic approach to find the optimal amount of training for frequency selective channels described by a blockfading model. When the training and data powers are allowed to vary, we show that the optimal number of training symbols is equal to the length of the channel impulse response. When the training and data powers are instead required to be equal, the optimal number of symbols may be larger. We further show that at high SNR trainingbased schemes are capable of capturing most of the channel capacity, whereas at low SNR they are highly suboptimal.https://authors.library.caltech.edu/records/5cgasqvh83On the expected complexity of sphere decoding
https://resolver.caltech.edu/CaltechAUTHORS:20150212074845925
Authors: Hassibi, Babak; Vikalo, Haris
Year: 2001
DOI: 10.1109/ACSSC.2001.987655
The problem of finding the leastsquares solution to a system of linear equations where the unknown vector is comprised of integers, but the matrix coefficient and given vector are comprised of real numbers, arises in many applications: communications, cryptography, GPS, to name a few. The problem is equivalent to finding the closest lattice point to a given point and is known to be NPhard. In communications applications, however, the given vector is not arbitrary, but rather is an unknown lattice point that has been perturbed by an additive noise vector whose statistical properties are known. Therefore in this paper, rather than dwell on the worstcase complexity of the integerleastsquares problem, we study its expected complexity, averaged over the noise and over the lattice. For the "sphere decoding" algorithm of Fincke and Pohst (1995) we find a closedform expression for the expected complexity and show that for a wide range of noise variances the expected complexity is polynomial, in fact often subcubic. Since many communications systems operate at noise levels for which the expected complexity turns out to be polynomial, this suggests that maximumlikelihood decoding, which was hitherto thought to be computationally intractable, can in fact be implemented in realtimea result with many practical implications.https://authors.library.caltech.edu/records/9k98414325Linear dispersion codes
https://resolver.caltech.edu/CaltechAUTHORS:20150227072321684
Authors: Hassibi, Babak; Hochwald, Bertrand M.
Year: 2001
DOI: 10.1109/ISIT.2001.936188
Multipleantenna systems that operate at high rates require simple yet effective spacetime transmission schemes to handle the large traffic volume in real time. At rates of tens of bits/sec/Hz, VBLAST, where every antenna transmits its own independent substream of data, has been shown to have good performance and simple encoding and decoding.
Yet VBLAST suffers from its inability to work with fewer receive antennas than transmit antennas. Furthermore, because VBLAST transmits independent data streams on its antennas there is no builtin spatial coding to guard against deep fades from any given transmit antenna. On
the other hand, there are many previouslyproposed spacetime codes that have good fading resistance and simple decoding, but these codes generally have poor performance at high data rates or with many
antennas.
We propose a highrate coding scheme that can handle any configuration of transmit and receive antennas and that subsumes both VBLAST and many proposed spacetime block codes as special cases. The scheme transmits substreams of data in linear combinations over space
and time. The codes are designed to optimize the mutual information between the transmitted and received signals. Because of their linear structure, the codes retain the decoding simplicity of VBLAST, and because of their information theoretic optimality, they possess many
coding advantages. We give examples of the codes and show that their performance is generally superior to earlier proposed methods over a wide range of rates and SNRs.https://authors.library.caltech.edu/records/se6e93vw40Highrate linear spacetime codes
https://resolver.caltech.edu/CaltechAUTHORS:20150212075147443
Authors: Hassibi, B.; Hochwald, B.
Year: 2001
DOI: 10.1109/ICASSP.2001.940499
Multipleantenna systems that operate at high
rates require simple yet effective spacetime transmission schemes to handle the large traffic volume in real time.
VBLAST, where every antenna transmits its own independent substream of data, has been shown to have good performance and simple encoding and decoding. Yet its drawbacks include its inability to work with fewer receive antennas than transmit antennas, and its absence of builtin spatial coding. On the other hand, there are many previously proposed spacetime codes that have good fading resistance and simple decoding, but generally poor performance at
high data rates or with many antennas.
We propose a highrate coding scheme that can handle any configuration of transmit and receive antennas and that subsumes both VBLAST and many proposed spacetime codes
as special cases. The scheme transmits substreams of data in
linear combinations over space and time and the codes are designed to optimize the mutual information between the
transmitted and received signals. Because of their
linear structure, the codes retain the decoding simplicity of VBLAST, and because of their informationtheoretic
optimality, they possess many coding advantages.https://authors.library.caltech.edu/records/a8e1ef6944Fullydiverse multipleantenna signal constellations and fixedpointfree Lie groups
https://resolver.caltech.edu/CaltechAUTHORS:20150212075856105
Authors: Hassibi, Babak; Khorrami, Mohammad
Year: 2001
DOI: 10.1109/ISIT.2001.936062
A group of unitary matrices is called fixedpointfree (fpf) if all nonidentity elements of the group have no eigenvalues at unity. Such groups are useful in multiple antenna communications, especially in multipleantenna differential modulation, since they constitute a
fullydiverse constellation. In this note we consider infinite groups and, in particular, their most interesting case, Lie groups. Two such fpf Lie groups are currently widely used in communications: the group of
unit modulus scalars, from which various phase modulation schemes, such as QPSK, are derived, and the 2×2 orthogonal designs of Alamouti, on which many twotransmitantenna schemes are based. In Liegrouptheoretic jargon these are referred to as U(1) and SU(2). A natural question is whether there exist other fpf Lie groups. We answer this question in the negative: U(1) and SU(2) are all there are.https://authors.library.caltech.edu/records/237f3fyy27Cayley codes for multiantenna differential modulation
https://resolver.caltech.edu/CaltechAUTHORS:20150213065942830
Authors: Hassibi, Babak; Hochwald, Bertrand M.
Year: 2001
DOI: 10.1109/ACSSC.2001.986911
Multiple antenna differential modulation using unitary matrices requires no channel knowledge at the receiver, and so is ideal for use on wireless links where channel tracking is undesirable or infeasible, either because of rapid changes in the channel characteristics or because of limited system resources. Although this basic principle is well understood, it is not known how to generate goodperforming constellations of unitary matrices, for any number of transmit and receive antennas and especially at high rates. We propose a class of Cayley codes that works with any number of antennas, and allows for polynomialtime nearmaximumlikelihood decoding based on either successive nulling/cancelling or sphere decoding. The codes use the Cayley transform, which maps the highly nonlinear Stiefel manifold of unitary matrices to the linear space of skewHermitian matrices. This leads to a simple linear constellation structure in the Cayley transform domain and to an informationtheoretic design criterion based on emulating a Cauchy random matrix. Simulations show that Cayley codes allow efficient and effective highrate data transmission in multiantenna communication systems without knowing the channel.https://authors.library.caltech.edu/records/909jvjew52Multipleantennas and isotropicallyrandom unitary inputs: the received signal density in closedform
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958724
Authors: Hassibi, Babak; Marzetta, Thomas L.
Year: 2001
DOI: 10.1109/isit.2001.936204
Computing the capacity of a multiple antenna wireless link subject to flat Rayleigh blockfading, with no channelstate information available either to the transmitter or to the receiver, is an important open problem. The isotropicallyrandom (i.r.) unitary matrixhaving orthonormal columns, and a probability density that is invariant to premultiplication by an independent unitary matrixplays a central role in the calculation of capacity and in some special cases is capacityachieving. We take an important step towards computing this capacity by obtaining, in closedform, the probability density of the received signal when transmitting i.r. unitary matrices. This enables us to evaluate the mutual information for any case of interest, something that could previously only be done for single transmit and receive antennas. Simulation results show that at high SNR the mutual information is maximized for M=min(N, T/2) transmit antennas, where N is the number of receive antennas and T is the length of the coherence interval, whereas at low SNR the mutual information is maximized by allocating all transmit power to a single antenna.https://authors.library.caltech.edu/records/t6q2z83a39Unitary spacetime codes and the Cayley transform
https://resolver.caltech.edu/CaltechAUTHORS:20150212072748572
Authors: Hassibi, Babak; Jing, Yindi
Year: 2002
DOI: 10.1109/ICASSP.2002.5745132
A previously proposed method for communicating with multiple antennas over block fading channels is unitary spacetime modulation (USTM), socalled because the transmitted signals form a matrix with orthonormal columns. Since channel knowledge is not required at the receiver, USTM schemes are suitable for use on wireless links where channel tracking is undesirable or infeasible. Results have shown that, if suitably designed, USTM schemes can achieve full channel capacity at high SNR. While all this is well recognized, what is not clear is how to generate good performing constellations of (nonsquare) unitary matrices, that lend themselves to efficient encoding/decoding. The schemes proposed so far either exhibit poor performance, especially at high rates, or have no efficient decoding algorithms. In this paper, we propose to use the Cayley transform to design USTM constellations. This work is a generalization, to the nonsquare case, of the Cayley codes that have been proposed for differential USTM. The codes are designed based on an informationtheoretic criterion, and lend themselves to polynomialtime (often cubic) nearmaximumlikelihood decoding using a sphere decoding algorithm.https://authors.library.caltech.edu/records/9nt5ey8f61Towards closing the capacity gap on multiple antenna channels
https://resolver.caltech.edu/CaltechAUTHORS:20150212073434297
Authors: Vikalo, Haris; Hassibi, Babak
Year: 2002
DOI: 10.1109/ICASSP.2002.5745126
In recent years, soft iterative decoding techniques have been shown to greatly improve the bit error rate performance of various communication systems. For multiple antenna systems, however, it is not clear what is the best way to obtain the softinformation required of the iterative scheme with low complexity. In this paper,
we propose a modification of the FinckePohst (sphere decoder) algorithm to estimate the MAP probability of the received symbol sequence. The new algorithm solves a nonlinear integer leastsquares problem and, over a wide range of rates and SNRs, has polynomialtime (often cubic) complexity. The performance of the algorithm, combined with convolutional, turbo, and LDPC codes is demonstrated on several multiple antenna channels.https://authors.library.caltech.edu/records/wbnrsqn570On the expected complexity of integer leastsquares problems
https://resolver.caltech.edu/CaltechAUTHORS:20150212073945154
Authors: Hassibi, Babak; Vikalo, Haris
Year: 2002
DOI: 10.1109/ICASSP.2002.5744897
The problem of finding the leastsquares solution to a system of linear equations where the unknown vector is comprised of integers, but the matrix coefficient and given vector are comprised of real numbers, arises in many applications: communications, cryptography, GPS, to name a few. The problem is equivalent to finding the closest lattice point to a given point and is known to be NPhard. In communications applications, however, the given vector is not arbitrary, but rather is an unknown lattice point that has been perturbed by an additive noise vector whose statistical properties are known. Therefore in this paper, rather than dwell on the worstcase complexity of the integerleastsquares problem, we study its expected complexity, averaged over the noise and over the lattice. For the "sphere decoding" algorithm of Fincke and Pohst (1985) we find a closedform expression for the expected complexity and show that for a wide range of noise variances the expected complexity is polynomial, in fact often subcubic. Since many communications systems operate at noise levels for which the expected complexity turns out to be polynomial, this suggests that maximumlikelihood decoding, which was hitherto thought to be computationally intractable, can in fact be implemented in realtimea result with many practical implications.https://authors.library.caltech.edu/records/h26xma1t37Multiantenna Cayley differential codes
https://resolver.caltech.edu/CaltechAUTHORS:20150212073144332
Authors: Hassibi, Babak; Hochwald, Bertrand M.
Year: 2002
DOI: 10.1109/ICASSP.2002.5745082
Multiple antenna differential modulation using unitary matrices requires no channel knowledge at the receiver, and so is ideal for use on wireless links where channel tracking is undesirable or infeasible, either because of rapid changes in the channel characteristics or because of limited system resources. Although this basic principle is well understood, it is not known how to generate goodperforming constellations of unitary matrices, for any number of transmit and receive antennas and especially at high rates. We propose a class of Cayley codes that works with any number of antennas, and allows for polynomialtime nearmaximumlikelihood decoding based on either successive ing/cancelling or sphere decoding. The codes use the Cayley transform, which maps the highly nonlinear Stiefel manifold of unitary matrices to the linear space of skewHermitian matrices. This leads to a simple linear constellation structure in the Cayley transform domain and to an informationtheoretic design criterion based on emulating a Cauchy random matrix. Simulations show that Cayley codes allow efficient and effective highrate data transmission in multiantenna communication systems without knowing the channel.https://authors.library.caltech.edu/records/z2ye3hnc87Modified finckepohst algorithm for lowcomplexity iterative decoding over multiple antenna channels
https://resolver.caltech.edu/CaltechAUTHORS:20150227072029585
Authors: Vikalo, Haris; Hassibi, Babak
Year: 2002
DOI: 10.1109/ISIT.2002.1023662
In recent years, soft iterative decoding techniques have been shown to greatly improve the bit error rate performance of various communication systems. For multiple antenna systems employing spacetime codes, however, it is not clear what is the best way to obtain the softinformation required of the iterative scheme with low complexity. In this paper, we propose a modification of the FinckePohst (sphere decoding) algorithm to estimate the maximum a posteriori (MAP) probability of the received symbol sequence. The new algorithm (FPMAP) solves a nonlinear integer leastsquares problem and, over a wide range of rates and SNRs, has polynomialtime (often cubic) expected complexity. The FPMAP algorithm provides soft detection information for the soft channel decoder. The soft decoder's output is then fed back to the FPMAP, and iterated on. The performance of the FPMAP algorithm on a multiple antenna system employing turbo code is demonstrated.https://authors.library.caltech.edu/records/n0t10k6r16Fullydiverse multiantenna spacetime codes based on Sp(2)
https://resolver.caltech.edu/CaltechAUTHORS:20150227071829673
Authors: Jing, Yindi; Hassibi, Babak
Year: 2002
DOI: 10.1109/ACSSC.2002.1196962
Fullydiverse constellations, i.e., a set of unitary matrices whose pairwise differences are nonsingular, are useful in multiantenna communications, especially in multiantenna differential modulation, since they have good pairwise error properties. Group theoretic ideas, especially fixedpointfree (FPF) groups, have been used to design fullydiverse constellations of unitary matrices. We construct fourtransmitantenna constellations appropriate for differential modulation based on the symplectic group Sp(2). These can be regarded as extensions of Alamouti's (1998) celebrated twotransmitantenna orthogonal design which can be constructed from the group Sp(1). We further show that the structure of the code tends itself to efficient maximum likelihood (ML) decoding via the sphere decoding algorithm. Finally, the performance of the code is compared with existing methods including Alamouti's scheme, Cayley differential unitary spacetime codes and groupbased codes.https://authors.library.caltech.edu/records/ykegwqkr34Unitary spacetime modulation via the Cayley transform
https://resolver.caltech.edu/CaltechAUTHORS:20190308135957863
Authors: Hassibi, Babak; Jing, Yindi
Year: 2002
DOI: 10.1109/isit.2002.1023406
A method of generating good performing USTM constellations using the Cayley transform is proposed. The codes, which can be used for any number of transmit and receive antennas without channel knowledge, are designed based on an informationtheoretic criterion, and lend themselves to polynomialtime nearmaximumlikelihood decoding using the sphere decoding algorithm.https://authors.library.caltech.edu/records/zj797dxx52Lowcomplexity iterative detection and decoding of multiantenna systems employing channel and spacetime codes
https://resolver.caltech.edu/CaltechAUTHORS:20150227071614629
Authors: Vikalo, Haris; Hassibi, Babak
Year: 2002
DOI: 10.1109/ACSSC.2002.1197194
We study multiple antenna systems that employ spacetime modulation schemes and transmit data encoded by powerful channel codes. Decoders for such codes require probabilistic (soft) information about transmitted bits. We use a modification of the FinckePohst algorithm to solve maximum a posteriori detection problem and efficiently approximate soft information. Simulation results that illustrate performance of the proposed system are presented.https://authors.library.caltech.edu/records/mhh3x3pw44Stability analysis of stochastically varying formations of dynamic agents
https://resolver.caltech.edu/CaltechAUTHORS:20150211070320141
Authors: Gupta, Vijay; Hassibi, Babak; Murray, Richard M.
Year: 2003
DOI: 10.1109/CDC.2003.1272613
We analyze a network of dynamic agents where the topology of the network specifies the information flow between the agents. We present an analysis method for such a system for both consensus and formation stabilization problems. To consider the general features introduced by the information flow topology, we consider the case of agent dynamics being a single integrator. Then we show that the method of analysis can be extended to more general cases of complicated agent dynamics, nonideal links for information flow, etc. We also consider the case when the topology of the network is changing over time. The focus of the paper is on obtaining conditions for the stability of the formation that can be checked in a decentralized way.https://authors.library.caltech.edu/records/tp3v1zfe08Should we break a wireless network into subnetworks?
https://resolver.caltech.edu/CaltechAUTHORS:20150212071001793
Authors: Dana, Amir F.; Gowaikar, Radhika; Hassibi, Babak; Effros, Michelle; Médard, Muriel; Koetter, Ralf
Year: 2003
In this paper we show that to achieve capacity of a wireless network, the global structure of the network should be considered. In other words, achieving capacity on the subnetworks of a wireless network does not guarantee achieving capacity globally. We illustrate this fact by some examples. Then we consider packet erasure wireless networks with limited sets of operations allowed at each node. We pose some interesting problems related to the optimal achievable rate of these networks and provide partial answers to some of them.https://authors.library.caltech.edu/records/bqkfqxaa81On the control of jump linear Markov systems with Markov state estimation
https://resolver.caltech.edu/CaltechAUTHORS:20150211075328058
Authors: Gupta, Vijay; Murray, Richard M.; Hassibi, Babak
Year: 2003
DOI: 10.1109/ACC.2003.1243762
We analyze a jump linear Markov system being stabilized using a linear controller. We consider the case when the Markov state is associated with the probability distribution of a measured variable. We assume that the Markov state is not known, but rather is being estimated based on the observations of the variable. We present conditions for the stability of such a system and also solve the optimal LQR control problem for the case when the state estimate update uses only the last observation value. In particular we consider a suboptimal version of the casual Viterbi estimation algorithm and show that a separation property does not hold between the optimal control and the Markov state estimate. Some simple examples are also presented.https://authors.library.caltech.edu/records/wyqkkpmv02On the Robustness of LMS Filters
https://resolver.caltech.edu/CaltechAUTHORS:20150225073400276
Authors: Hassibi, Babak
Year: 2003
[no abstract]https://authors.library.caltech.edu/records/bfjsg5f837Maximumlikelihood decoding and integer leastsquares: the expected complexity
https://resolver.caltech.edu/CaltechAUTHORS:20150225074538196
Authors: Hassibi, Babak; Vikalo, Haris
Year: 2003
The problem of finding the leastsquares solution to a system of linear equations where the unknown
vector is comprised of integers, but the matrix coefficient and given vector are comprised of real numbers,
arises in many applications: communications, cryptography, GPS, to name a few. The problem is equivalent
to finding the closest lattice point to a given point and is known to be NPhard. In communications
applications, however, the given vector is not arbitrary, but rather is an unknown lattice point that has been
perturbed by an additive noise vector whose statistical properties are known. Therefore in this paper, rather
than dwell on the worstcase complexity of the integerleastsquares problem, we study its expected complexity,
averaged over the noise and over the lattice. For the "sphere decoding" algorithm of Fincke and
Pohst we find a closedform expression for the expected complexity and show that, for a wide range of noise
variances and dimensions, the expected complexity is polynomial, in fact often roughly cubic. Since many
communications systems operate at noise levels for which the expected complexity turns out to be polynomial,
this suggests that maximumlikelihood decoding, which was hitherto thought to be computationally
intractable, can in fact be implemented in realtime—a result with many practical implications.https://authors.library.caltech.edu/records/1mys10hy93Linear network codes: A unified framework for source, channel, and network coding
https://resolver.caltech.edu/CaltechAUTHORS:20150210075301992
Authors: Effros, Michelle; Médard, Muriel; Ho, Tracey; Ray, Siddharth; Karger, David; Koetter, Ralf; Hassibi, Babak
Year: 2003
We examine the issue of separation and code design for network data transmission environments. We demonstrate that sourcechannel separation holds for several canonical network channel models when the whole network operates over a common finite field. Our approach uses linear codes. This simple, unifying framework allows us to reestablish with economy the optimality of linear codes for single transmitter channels and for SlepianWolf source coding. It also enables us to establish the optimality of linear codes for multiple access channels and for erasure broadcast channels. Moreover, we show that sourcechannel separation holds for these networks. This robustness of separation we show to be strongly predicated on the fact that noise and inputs are independent. The linearity of source, channel, and network coding blurs the delineation between these codes, and thus we explore joint linear design. Finally, we illustrate the fact that design for individual network modules may yield poor results when such modules are concatenated, demonstrating that endtoend coding is necessary. Thus, we argue, it is the lack of decomposability into canonical network modules, rather than the lack of separation between source and channel coding, that presents major challenges for coding in networks.https://authors.library.caltech.edu/records/9557k6zr11Joint maximumlikelihood channel estimation and signal detection for SIMO channels
https://resolver.caltech.edu/CaltechAUTHORS:20150211075755481
Authors: Stoica, P.; Vikalo, H.; Hassibi, B.
Year: 2003
DOI: 10.1109/ICASSP.2003.1202529
In wireless communication systems, channel state information is often assumed to be available at the receiver. Traditionally, a training sequence is used to obtain the estimate of the channel. Alternatively, the channel can be identified using known properties of the transmitted signal. However, the computational effort required to find the joint ML solution to the symbol detection and channel estimation problem increases exponentially with the dimension of the problem. To significantly reduce this computational effort, we formulate the aforementioned problem in a way that makes it possible to solve it via the use of sphere decoding, an algorithm that has polynomial expected complexity. We also provide simulation results and a complexity discussion.https://authors.library.caltech.edu/records/f4j20x4037Highrate spacetime codes motivated by SU(3)
https://resolver.caltech.edu/CaltechAUTHORS:20150211073959812
Authors: Jing, Yindi; Hassibi, Babak
Year: 2003
DOI: 10.1109/ACSSC.2003.1292293
Fullydiverse constellations, i.e., a set of unitary matrices whose pairwise differences are nonsingular,
are useful in multiantenna communications especially in
multiantenna differential modulation, since they have
good pairwise error properties. Recently,group theoretic
ideas, especially fixedpointfree (fpf) groups,
have been used to design fullydiverse constellations of unitary matrices. Here we give systematic methods to design spacetime codes which are appropriate for threetransmit
antenna differential modulation. The structures of the codes
are motivated by the Lie group SU(3). One of the codes, called the AB code, has a fast decoding algorithm using the complex sphere decoder. The diversity products of the codes can be easily calculated and simulated performances show that the codes are better than the groupbased codes
[1] especially at high rates and as good as the elaboratelydesigned nongroup codes[1].https://authors.library.caltech.edu/records/cymqy27n22Efficient statistical pruning for maximum likelihood decoding
https://resolver.caltech.edu/CaltechAUTHORS:20150212070415519
Authors: Gowaikar, Radhika; Hassibi, Babak
Year: 2003
DOI: 10.1109/ICASSP.2003.1199865
In many communications problems, maximumlikelihood (ML) decoding reduces to finding the closest (skewed) lattice point in Ndimensions to a given point x∈C^N. In its full generality, this problem is known to be NPcomplete and requires exponential complexity in N. Recently, the expected complexity of the sphere decoder, a particular algorithm that solves the ML problem exactly, has been computed; it is shown that, over a wide range of rates, SNRs and dimensions, the expected complexity is polynomial in N. We propose an algorithm that, for large N, offers substantial computational savings over the sphere decoder, while maintaining performance arbitrarily close to ML. The method is based on statistically pruning the search space. Simulations are presented to show the algorithm's performance and the computational savings relative to the sphere decoder.https://authors.library.caltech.edu/records/q8sdfnab82Efficient nearML decoding via statistical pruning
https://resolver.caltech.edu/CaltechAUTHORS:20150227071358104
Authors: Gowaikar, Radhika; Hassibi, Babak
Year: 2003
DOI: 10.1109/ISIT.2003.1228289
Maximumlikelihood (ML) decoding often reduces to finding
the closest (skewed) lattice point in Ndimensions to
a given point x ϵ C^N. Sphere decoding is an
algorithm that does this. We modify the sphere decoder to
reduce the computational complexity of decoding while
maintaining nearML performance.https://authors.library.caltech.edu/records/6j1684va36Sphereconstrained ML detection for frequencyselective channels
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958122
Authors: Vikalo, H.; Hassibi, B.; Mitra, U.
Year: 2003
DOI: 10.1109/icassp.2003.1202526
Maximumlikelihood (ML) detection problem for channels with memory is investigated. The Viterbi algorithm provides an elegant solution, but is computationally inefficient when employed for detection on long channels. On the other hand, sphere decoding solves the ML detection problem in polynomial expected time over a wide range of SNRs. The sphereconstrained search strategy of sphere decoding is combined with the dynamic programming principles of the Viterbi algorithm. The resulting algorithm has the worstcase complexity of the Viterbi algorithm, but significantly lower expected complexity.https://authors.library.caltech.edu/records/em373ee357Design of fullydiverse multiantenna codes based on Sp(2)
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958032
Authors: Jing, Yindi; Hassibi, Babak
Year: 2003
DOI: 10.1109/icassp.2003.1202534
Fullydiverse constellations, i.e., a set of unitary matrices whose pairwise differences are nonsingular, are useful in multiantenna communications, especially in multiantenna differential modulation, since they have good pairwise error properties. Recently, group theoretic ideas, especially fixedpointfree (FPF) groups, have been used to design fullydiverse constellations of unitary matrices. Here we construct fourtransmitantenna constellations appropriate for differential modulation based on the symplectic group Sp(2) These can be regarded as extensions of S.M. Alamouti's celebrated twotransmitantenna orthogonal design which can be constructed from the group Sp(1) (see IEEE J. Sel. Area Commun., p.14518, 1998). We further show that the structure of the code lends itself to efficient maximum likelihood (ML) decoding via the sphere decoding algorithm. Finally, the performance of the code is compared with existing methods including Alamouti's scheme, Cayley differential unitary spacetime codes and group based codes.https://authors.library.caltech.edu/records/mx3rm5tn34A deterministic algorithm that achieves the PMEPR of c log n for multicarrier signals
https://resolver.caltech.edu/CaltechAUTHORS:SHAicassp03
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2003
Multicarrier signals often exhibit large peak to mean envelope power ratios (PMEPR) which can be problematic in practice. In this paper, we study adjusting the sign of each subcarrier in order to reduce the PMEPR of a multicarrier signal with n subcarriers. Considering that any randomly chosen codeword has PMEPR of log n with probability one and for large values of n [1], randomly choosing signs should lead to the PMEPR of log n in the probability sense. Based on the derandomization algorithm suggested in [2], we propose a deterministic and efficient algorithm to design signs such that the PMEPR of the resulting codeword is less than c log n for any n where c is a constant independent of n. By using a symmetric qary constellation, this algorithm in fact constructs a code with rate 1  logq 2, PMEPR of c log n, and with simple encoding and decoding. We then present simulation results for our algorithm.https://authors.library.caltech.edu/records/nqq35ayg43On the average power of multiple subcarrier intensity modulated optical signals: Nehari's problem and coding bounds
https://resolver.caltech.edu/CaltechAUTHORS:SHAicc03
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2003
DOI: 10.1109/ICC.2003.1204608
Multiple subcarrier modulation (MSM) is an attractive technique for optical wireless communication for high speed applications. The main disadvantage of this scheme is its low average power efficiency which is an analogous problem to the high peak to mean envelope power ratio (PMEPR) of multicarrier signals. In this paper, we consider the achievable average power reduction of MSM signals by using optimized reserved carriers and coding methods. Based on Nehari's result we present a lower bound for the maximum average power of the signal after adding the reserved carriers. It is shown that the mean value of the average required power behaves very close to √2n log log n for a BPSK constellation where n is the number of subcarriers. We then consider finding the optimum values for the carriers and the effect of having finite bandwidth for reserved carriers. In the next section, mainly based on recent coding results for the PMEPR of multicarrier signals, we show the existence of very high rate codes with average power of O(√n log n) for large values of n, and furthermore the existence of codes with nonvanishing to zero rate and average power of O(√n) asymptotically.https://authors.library.caltech.edu/records/mp3qy5p568Asymptotic probability bounds on the peak distribution of complex multicarrier signals without Gaussian assumption
https://resolver.caltech.edu/CaltechAUTHORS:SHAasilo02.951
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2003
DOI: 10.1109/ACSSC.2002.1197170
Multicarrier signals exhibit a large peak to mean envelope power ratio (PMEPR). In this paper, we derive the lower and upper probability bounds for the PMEPR distribution when entries of the codeword, C, are chosen independently from a symmetric qary PSK or QAM constellation, C /spl isin/ /spl Qscr/;/sup nq/, or C is chosen from a complex n dimensional sphere, /spl Omega//sup n/ when the number of subcarriers, n, is large and without any Gaussian assumption on either the joint distribution or any sample of the multicarrier signal. Even though the worst case PMEPR is of the order of n, the main result is that the PMEPR of a random codeword C chosen from /spl Qscr/;/sup nq/ or /spl Omega//sup n/ is log n with probability one, asymptotically. A VarsharmovGilbert (VG) style bound for the achievable rate and minimum Hamming distance of codes chosen from /spl Qscr/;/sup nq/, with PMEPR of less than log n is obtained. It is proved that asymptotically, the VG bound remains the same for the codes chosen from /spl Qscr/;/sup nq/ with PMEPR of less than log n.https://authors.library.caltech.edu/records/2mgpzwsg43On the existence of codes with constant bounded PMEPR for multicarrier signals
https://resolver.caltech.edu/CaltechAUTHORS:SHAisit03
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2003
DOI: 10.1109/ISIT.2003.1228144
It has been shown that with probability one the peak to mean envelope power ratio (PMEPR) of any random codeword chosen from a symmetric QAM/PSK constellation is log n where n is the number of subcarriers [1]. In this paper, the existence of
codes with nonzero rate and PMEPR bounded by a constant is established.https://authors.library.caltech.edu/records/tbv0h6vk42On the power efficiency of sensory and adhoc wireless networks
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958221
Authors: Hassibi, Babak; Dana, Amir
Year: 2003
DOI: 10.1109/isit.2003.1228429
This paper discusses the power efficiency of a communications channel, i.e., the maximum bit rate that can be achieved per unit power (energy rate). For AWGN channels, it is well known that power efficiency is attained in the low SNR regime. In this paper we show that for a random sensory, or adhoc, wireless network with n users (nodes), with high probability converging to one as n grows, the power efficiency scales at least by a factor of √n. In other words, each user in a wireless channel with n nodes can support the same communications rate as a single user system, but by expending only 1/√n the energy.https://authors.library.caltech.edu/records/cs5ede2r41On joint ML detection and decoding for linear block codes
https://resolver.caltech.edu/CaltechAUTHORS:20150225075547820
Authors: Vikalo, Haris; Hassibi, Babak
Year: 2003
DOI: 10.1109/ISIT.2003.1228290
We consider joint maximumlikelihood (ML) detection and decoding in multipleinput multipleoutput (MIMO) systems. The information data is encoded by a linear block errorcorrecting code and then transmitted across the MIMO channel in AWGN. Geometrically, the transmitted symbol is a
point in a highdimensional lattice. The received symbol
is the lattice point perturbed by an additive noise.
The joint detection and decoding problem is equivalent
to the search for the closest lattice point that is
an admissible codeword. We propose an algorithm
which performs a search constrained by a sphere centered
at the observed point. The radius of the sphere
is determined according to the statistics of the noise.
Thus the computational complexity of the algorithm
is a random variable. We quantify it by means of its
first moment which, for binary codes, we And analytically.
The expected complexity of the proposed
algorithm is polynomial in the length of the uncoded
information word over a wide range of SNRs.https://authors.library.caltech.edu/records/bwcm1kpm62Fullydiverse Sp(2) code design
https://resolver.caltech.edu/CaltechAUTHORS:20150227071011013
Authors: Jing, Yindi; Hassibi, Babak
Year: 2003
DOI: 10.1109/ISIT.2003.1228314
A fullydiverse code that is suitable for differential
modulation for fourtransmitantenna communication systems
is constructed based on the symplectic group Sp(2).
The code can be regarded as an extension of Alamouti's celebrated twotransmitantenna orthogonal design
which can be constructed from the group Sp(1). The structure
of the code lends itself to efficient ML decoding via the sphere decoding algorithm.https://authors.library.caltech.edu/records/3nz37qar78Analysis of multiple antenna wireless links at low SNR
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958304
Authors: Rao, Chaitanya; Hassibi, Babak
Year: 2003
DOI: 10.1109/isit.2003.1228485
In the low signaltonoise (SNR) regime when the channel is not known to both transmitter and receiver, we show that the capacity of a multiple antenna Rayleigh fading link is asymptotically quadratic in the SNR for all practical input distributions. This is much less than the known channel case where it exhibits a linear growth in the SNR. Under various signaling constraints we show that mutual information is maximized by using a single transmit antenna. Also sending training symbols offers no advantage at low SNR.https://authors.library.caltech.edu/records/rhk7337527Sphereconstrained ML detection for channels with memory
https://resolver.caltech.edu/CaltechAUTHORS:20150211073457103
Authors: Vikalo, Haris; Hassibi, Babak; Mitra, Urbashi
Year: 2003
DOI: 10.1109/ACSSC.2003.1291996
The maximumlikelihood (ML) detection problem for channels with memory is investigated. The Viterbi algorithm (VA) provides an exact solution. Its computational complexity is linear in the length of the transmitted sequence but exponential in the channel memory length. Hence, the VA can be computationally inefficient when employed for detection on long channels. On the other hand, the sphere decoding (SD) algorithm also solves the ML detection problem exactly and has expected complexity which is polynomial (often cubic) in the length of the transmitted sequence over a wide range of signaltonoise ratios (SNR). We combine the sphereconstrained search strategy of SD with the dynamic programming principles of the VA. The resulting algorithm has the worstcase complexity of the VA, but often significantly lower expected complexity.https://authors.library.caltech.edu/records/k01e159120On the capacity of MIMO broadcast channel with partial side information
https://resolver.caltech.edu/CaltechAUTHORS:SHAasilo03
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2003
DOI: 10.1109/ACSSC.2003.1292058
Since having full channel state information in the transmitter is not reasonable in many applications and lack of channel knowledge does not lead to linear growth of the sum rate capacity as the number transmit antennas increases, it is therefore of interest to investigate transmission schemes that employ only partial CSI. In this paper, we propose a scheme that constructs M random beams and that transmits information to the users with the highest signaltonoiseplusinferference ratios (SINRs), which can be made available to the transmitter with very little feedback. For fixed M and n increasing, the sumrate capacity of our scheme scales as M log log n, which is precisely the same scaling obtained with perfect channel information. We furthermore show that linear increase in capacity can be obtained provided that M does not not grow faster than O(log n). We also study the fairness of our scheduling scheme and show that, when M is large enough, the system becomes interferencedominated and the probability of transmitting to any user converges to 1/n, irrespective of its pathloss. In fact, using M = α log n transmit antennas emerges as a desirable operating point, both in terms of providing linear increase in capacity as well as in guaranteeing fairness.https://authors.library.caltech.edu/records/j040fdre81Is broadcast plus multiaccess optimal for Gaussian wireless networks?
https://resolver.caltech.edu/CaltechAUTHORS:DANasilo03
Authors: Dana, Amir F.; Sharif, Masoud; Gowaikar, Radhika; Hassibi, Babak; Effros, Michelle
Year: 2003
DOI: 10.1109/ACSSC.2003.1292284
In this paper we show that "separation"based approaches in wireless networks do not necessarily give good performance in terms of the capacity of the network. Therefore in optimal design of a wireless network, its total structure should be considered. In other words, achieving capacity on the subnetworks of a wireless network does not guarantee globally achieving capacity. We will illustrate this fact by considering some examples of multistage Gaussian wireless relay networks. We will consider a wireless Gaussian relay network with one stage in both fading and nonfading environment. We show that as the number of relay nodes, n, grows large, the capacity of this network scales like log n. We then show that with the "separation"based scheme, in which the network is viewed as the concatenation of a broadcast and a multiaccess network, the achievable rate scales as log log n and as a constant for fading and nonfading environment, respectively, which is clearly suboptimal.https://authors.library.caltech.edu/records/bym5bd8p25Nucleic Acid Detection Using Bioluminescence Regenerative Cycle and Statistical Signal Processing
https://resolver.caltech.edu/CaltechAUTHORS:20150211073018535
Authors: Vikalo, H.; Hassibi, A.; Hassibi, B.
Year: 2004
An important emerging research area is the study and development of signal processing techniques for rapid real time nucleic acid detection (1). In this paper, we focus on the newly developed bioluminescence regenerative cycle (BRC) technique, and apply statistical signal processing to the data identification problem. This extended summary provides a description of the BRC platform and experiments, the statistical model employed for analysis, and some preliminary experimental results.https://authors.library.caltech.edu/records/1b0vrg5272Spacetime code design for threetransmitantenna systems
https://resolver.caltech.edu/CaltechAUTHORS:20150210074713367
Authors: Jing, Yindi; Hassibi, Babak
Year: 2004
DOI: 10.1109/ICASSP.2004.1326862
Fully diverse constellations, i.e., a set of unitary matrices whose pairwise differences are nonsingular, are useful in multiantenna communications especially in multiantenna differential modulation, since they have good pairwise error properties. Recently, group theoretic ideals, especially fixedpointfree (fpf) groups, have been used to design fully diverse constellations of unitary matrices. Here we give a systematic method to design spacetime codes which are appropriate for threetransmitantenna differential modulation. The structure of the code is motivated by the Lie group SU(3). The code has a fast decoding algorithm using sphere decode. The diversity product of the code can be easily calculated and simulated performance shows that the code is better than the groupbased codes especially at high rates and is as good as the elaboratelydesigned nongroup code.https://authors.library.caltech.edu/records/0ehbj3xr29Sensor scheduling algorithms requiring limited computation
https://resolver.caltech.edu/CaltechAUTHORS:20150211071543730
Authors: Gupta, Vijay; Chung, Timothy; Hassibi, Babak; Murray, Richard M.
Year: 2004
DOI: 10.1109/ICASSP.2004.1326672
In this paper, we consider the scenario where many sensors cooperate to estimate a process. Only one sensor can take a measurement at any time step. We wish to come up with optimal sensor scheduling algorithms. The problem is motivated by the use of sonar rangefinders used by the vehicles on the Caltech MultiVehicle Wireless Testbed. We see that this problem involves searching a tree in general and propose and analyze two strategies for pruning the tree to keep the computation limited. The first is a sliding window strategy motivated by the Viterbi algorithm, and the second one uses thresholding. We also study a technique that employs choosing the sensors randomly from a probability distribution which can then be optimized. The performance of the algorithms are illustrated with the help of numerical examples.https://authors.library.caltech.edu/records/vswwftx625Scaling laws of sum rate using timesharing, DPC, and beamforming for MIMO broadcast channels
https://resolver.caltech.edu/CaltechAUTHORS:SHAisit04a
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2004
DOI: 10.1109/ISIT.2004.1365214
We derive the scaling laws of the sum rate throughput for MIMO Gaussian broadcast channels using timesharing to the strongest user, dirty paper coding (DPC), and beamforming when the number of users (receivers) n is large. Assuming a fixed total average transmit power, we show that for a system with M transmit antennas and users equipped
with N antennas, the sum rate scales like M log log nN
for DPC and beamforming when M is fixed and for any N (either growing to infinity or not). On the other hand, when both M and N are fixed, the sum rate of timesharing to the strongest user scales like min(M,N) log log n. It is also shown that if M grows as log n, the sum rate of DPC and beamforming will grow linearly in M, but with different constant multiplicative factors. In this region, the sum rate capacity of timesharing scales like N log log n.https://authors.library.caltech.edu/records/qfycw25h90Peak to average power reduction using amplitude and sign adjustment
https://resolver.caltech.edu/CaltechAUTHORS:SHAicc04
Authors: Sharif, Masoud; Florens, Cedric; Fazel, Maryam; Hassibi, Babak
Year: 2004
DOI: 10.1109/ICC.2004.1312619
In this paper, we propose a method to reduce the peak to mean envelope power ratio (PMEPR) of multicarrier signals by modifying the constellation. For MPSK constellations,
we minimize the maximum of the multicarrier signal over the
sign and amplitude of each subcarrier. In order to find an efficient solution to the aforementioned nonconvex optimization problem, we present a suboptimal solution by first optimizing over the signs using the result of [1], and then optimizing over the amplitudes given the signs. We prove that the minimization of the maximum of a multicarrier signal over the amplitude of each subcarrier can be written as a convex optimization problem with linear matrix inequality constraints. We also generalize the idea to other
constellations such as 16QAM. Simulation results show that by an average power increase of 0.21 db and not sending information over the sign of each subcarrier, PMEPR can be decreased by 5.1 db for a system with 128 subcarriers.https://authors.library.caltech.edu/records/3gy93d9867Delay guarantee versus throughput in broadcast fading channels
https://resolver.caltech.edu/CaltechAUTHORS:SHAisit04b
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2004
DOI: 10.1109/ISIT.2004.1365282
We consider a singleantenna broadcast fading channel with n backlogged users. Assuming the transmission is packetbased, we define the delay as the minimum number of channel uses that guarantees all n users successfully receive m packets. A delay optimal strategy such as roundrobin achieves
the delay of mn. For the optimal throughput strategy
(i.e. transmitting to the user with the best channel
condition at each channel use), we derive the mean
and variance of the delay for any m and n. For large
n, it is proved that the expected delay in receiving the
first packet in all users scales like n log n as opposed to
n for the roundrobin scheduling.https://authors.library.caltech.edu/records/w4cb3bsy76The Gaussian interference channel at low SNR
https://resolver.caltech.edu/CaltechAUTHORS:20150210074255750
Authors: Rao, Chaitanya; Hassibi, Babak
Year: 2004
DOI: 10.1109/ISIT.2004.1365453
For the twouser Gaussian interference channel we show that, up to second order in the signaltonoise ratio (SNR), the mutual information between each transmitter and intended receiver depends only on the covariance matrices of the input distributions. Our analysis suggests that in the low SNR regime there is an interference threshold above which it is better for the users to alternate signal transmission than to transmit simultaneously.https://authors.library.caltech.edu/records/g43wmvwe25Statistical approach to ML decoding of linear block codes on symmetric channels
https://resolver.caltech.edu/CaltechAUTHORS:20150210073800640
Authors: Vikalo, Haris; Hassibi, Babak
Year: 2004
DOI: 10.1109/ISIT.2004.1365558
Maximumlikelihood (ML) decoding of linear block codes on a symmetric channel is studied. Exact ML decoding is known to be computationally difficult. We propose an algorithm that finds the exact solution to the ML decoding problem by performing a depthfirst search on a tree. The tree is designed from the code generator matrix and pruned based on the statistics of the channel noise. The complexity of the algorithm is a random variable. We characterize the complexity by means of its first moment, which for binary symmetric channels we find in closedform. The obtained results indicate that the expected complexity of the algorithm is low over a wide range of system parameters.https://authors.library.caltech.edu/records/qp64xjyx49Power bandwidth tradeoff for sensory and adhoc wireless networks
https://resolver.caltech.edu/CaltechAUTHORS:20150225074130900
Authors: Dana, Amir F.; Hassibi, Babak
Year: 2004
DOI: 10.1109/ISIT.2004.1365506
We look at the power bandwidth tradeoff in random sensory and adhoc wireless networks with n users and r ≤ √n simultaneous source/destination pairs. Under a specific protocol, we show that the minimum power required for maintaining an achievable scaling law of R_(sum)=Θ(f(n)) for the sumrate in the network, scales like Θ(f(n)/√n). The required bandwidth B in this case is of order Θ(f(n)/r). It is also proved that the minimum achievable energy per information bit (E_b/N_0)min for this protocol, scales as Θ(1/√n) and in this case the spectral efficiency is nonzero and is of constant order.https://authors.library.caltech.edu/records/fwk9n8zz71On the synthesis of control laws for a network of autonomous agents
https://resolver.caltech.edu/CaltechAUTHORS:20150211070629438
Authors: Gupta, Vijay; Hassibi, Babak; Murray, Richard M.
Year: 2004
We study the synthesis problem of a LQR controller when the matrix describing the control law is additionally constrained to lie in a particular vector space. Our motivation is the use of such control laws to stabilize networks of autonomous agents in a decentralized fashion; with the information flow being dictated by the constraints of a prespecified topology. We formulate the problem as an optimization problem and provide numerical procedures to solve it. Then, we apply the technique to the decentralized vehicle formation control problem and show that the topology can have a significant effect on the optimal cost.https://authors.library.caltech.edu/records/s5ytfw0r28On the capacity of wireless erasure networks
https://resolver.caltech.edu/CaltechAUTHORS:GOWisit04
Authors: Gowaikar, Radhika; Dana, Amir F.; Palanki, Ravi; Hassibi, Babak; Effros, Michelle
Year: 2004
DOI: 10.1109/ISIT.2004.1365437
We determine the capacity of a certain class of wireless erasure relay networks. We first find a suitable definition for the "cutcapacity" of erasure networks with broadcast at transmission and no interference at reception. With this definition, a maxflow mincut capacity result holds for the capacity of these networks.https://authors.library.caltech.edu/records/f0krw81g56Distributed spacetime codes in wireless relay networks
https://resolver.caltech.edu/CaltechAUTHORS:20150225073914994
Authors: Jing, Yindi; Hassibi, Babak
Year: 2004
DOI: 10.1109/SAM.2004.1502947
We apply the idea of spacetime coding devised for multiple antenna systems to the communication over a wireless relay network. We use a two stage protocol, where in one stage the transmitter sends information and in the other, the relay nodes encode their received signals into a "distributed" linear dispersion code, and then transmit the coded signals to the receiver. We show that for high SNR the pairwise error probability (PEP) behaves as (log P/P)^(min{T,R}) with T the coherence interval, R the number of relay nodes and P the total transmit power. Thus, apart from the log P factor and assuming T≥R, the system has the same diversity as a multiantenna system with R transmit antennas, which is the same as assuming that the R relay nodes can fully cooperate and have full knowledge of the transmitted signal. We further show that for a fixed total transmit power across the entire network, the optimal power allocation is for the transmitter to expend half the power and for the relays to collectively expend the other half. We also show that at low and high SNR, the coding gain is the same as that of a multiantenna system with R antennas. At intermediate SNR, it can be quite different, which has implications for the design of distributed spacetime codes.https://authors.library.caltech.edu/records/yrvm2x9g93Wireless networks, diversity and spacetime codes
https://resolver.caltech.edu/CaltechAUTHORS:20150210073202979
Authors: Jing, Yindi; Hassibi, Babak
Year: 2004
DOI: 10.1109/ITW.2004.1405348
We apply the idea of spacetime coding devised for multipleantenna systems to the problem of communications over wireless relay networks. A twostage protocol is used, where in one stage the transmitter sends information and in the other, the relay nodes encode their received signals into a "distributed" linear dispersion code, and then transmit the coded signals to the receiver. We show that for high SNR the proposed system has a diversity of order α_0 min{T, R}, with T the coherence interval, R the number of relay nodes, and α0 the solution to the equation α + (log α)/(log P) = 1  (log log P)/(log P), where P is the total transmit power in the network. In particular, we show that the pairwise error probability (PEP) decays no slower than ((log P)/P)^(min{T,R}). Thus, apart from the log P factor and assuming T ≥ R, the system has the same diversity as a multipleantenna system with R transmit antennas and one receive antenna, which is the same as assuming that the R relay nodes can fully cooperate and have full knowledge of the transmit signal. We further show that for a fixed total transmit power across the entire network, the optimal power allocation is for the transmitter to expend half the power and for the relays to collectively expend the other half. We also show that at low and high SNR, the coding gain is the same as that of multipleantenna systems. However, at intermediate SNR, it can be quite different. We discuss some of the ramifications of using different spacetime codes and finally verify our analysis through the simulation or randomly generated distributed spacetime codes.https://authors.library.caltech.edu/records/cxe45drs16Rate maximization in multiantenna broadcast channels with linear preprocessing
https://resolver.caltech.edu/CaltechAUTHORS:20190308135958382
Authors: Stojnic, Mihailo; Vikalo, Haris; Hassibi, Babak
Year: 2004
DOI: 10.1109/glocom.2004.1379110
The sum rate capacity of the multiantenna broadcast channel has recently been computed. However, the search for efficient practical schemes that achieve it is still ongoing. In this paper, we focus on schemes with linear preprocessing of the transmitted data. We propose two criteria for precoding matrix design, one maximizing the sum rate and the other maximizing the minimum rate among all users. The latter problem is shown to be quasiconvex and is solved exactly via the bisection method. In addition to precoding, we employ a signal scaling scheme that minimizes average biterrorrate (BER). The signal scaling scheme is posed as a convex optimization problem, and thus can be solved exactly via efficient interiorpoint methods. In terms of achievable sum rate, the proposed technique significantly outperforms traditional channel inversion methods, while having comparable (in fact, often superior) BER performance.https://authors.library.caltech.edu/records/y2kkgt7619Differentiated Rate Sceduling for MIMO Broadcast Channels
https://resolver.caltech.edu/CaltechAUTHORS:20150223070813329
Authors: Vakili, Ali; Dana, Amir; Sharif, Masoud; Hassibi, Babak
Year: 2005
We consider the problem of differentiated rate scheduling for the fading MIMO Gaussian broadcast channel, in the sense that the rates required by different users must satisfy certain rational rate constraints. When full channel state information (CSI) is available at the transmitter, the problem can be readily solved using dirty paper coding (DPC) and con vex optimization techniques on the dual multipleaccess channel (MAC). However, since in many practical applications full CSI is not feasible, and since the computational complexity may be prohibitive when the number of users is large, we focus on two simple schemes that require very little CSI: timedivision opportunistic (TO) beamforming where in different timeslots the transmitter performs opportunistic beamforing only to users requiring the same rate, and weighted opportunistic (WO) beamforing where the random beams are assigned to those users having the largest weighted SINR. In both cases we determine explicit schedules to guarantee the rate constraints and show that, in the limit of a large number of users, the throughput loss compared to the unconstrained sumrate capacity tends to zero. As a side result, we show that, in this regime, the sumrate of opportunistic beamforming converges to the optimal sumrate achieved by DPC, which is a stronger result than the orderoptimal results of (10, 13).https://authors.library.caltech.edu/records/nnrw78xz86A delay analysis for opportunistic transmission in fading broadcast channels
https://resolver.caltech.edu/CaltechAUTHORS:SHAinfocomm05
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2005
DOI: 10.1109/INFCOM.2005.1498555
We consider a singleantenna broadcast block fading channel (downlink scheduling) with n users where the transmission is packetbased and all users are backlogged. We define the delay as the minimum number of channel uses that guarantees all n users successfully receive m packets. This is a more stringent notion of delay than average delay and is the worst case delay among the users. A delay optimal scheduling scheme, such as roundrobin, achieves the delay of mn. In a heterogeneous network and for the optimal throughput strategy where the transmitter sends the packet to the user with the best channel conditions, we derive the moment generating function of the delay for any m and n. For large n and in a homogeneous network, the expected delay in receiving one packet by all the receivers scales as n log n, as opposed to n for the roundrobin scheduling. We also show that when m grows faster than (log n)^r, for some r > 1, then the expected value of delay scales like mn. This roughly determines the timescale required for the system
to behave fairly in a homogeneous network. We then propose a
scheme to significantly reduce the delay at the expense of a small throughput hit.
We further look into two generalizations of our work: i) the
effect of temporal channel correlation and ii) the advantage of multiple transmit antennas on the delay. For a channel with memory of two, we prove that the delay scales again like n log n no matter how severe the correlation is. For a system with M transmit antennas, we prove that the expected delay in receiving one packet by all the users scales like (n log n)/(M +O((M^2)/n) for large n and when M is not growing faster than log n. Thus, when the temporal channel correlation is zero, multiple transmit antenna systems do not reduce the delay significantly. However, when channel correlation is present, they can lead to significant gains
by "decorrelating" the effective channel through means such as random beamforming.https://authors.library.caltech.edu/records/h4a6r2x828A branch and bound approach to speed up the sphere decoder
https://resolver.caltech.edu/CaltechAUTHORS:20150210071852248
Authors: Stojnic, M.; Vikalo, H.; Hassibi, B.
Year: 2005
DOI: 10.1109/ICASSP.2005.1415738
In many communications applications, maximumlikelihood decoding reduces to solving an integer leastsquares problem which is NP hard in the worstcase. However, as has recently been shown, over a wide range of dimensions and SNRs, the sphere decoder can be used to find the exact solution with an expected complexity that is roughly cubic in the dimension of the problem. However, the computational complexity becomes prohibitive if the SNR is too low and/or if the dimension of the problem is too large. We target these two regimes and attempt to find faster algorithms by pruning the search tree beyond what is done in the standard sphere decoder. The search tree is pruned by computing lower bounds on the possible optimal solution as we proceed down the tree. We observe a tradeoff between the computational complexity required to compute the lower bound and the size of the pruned tree: the more effort spent computing a tight lower bound, the more branches that can be eliminated in the tree. Thus, even though it is possible to prune the search tree (and hence the number of points visited) by several orders of magnitude, this may be offset by the computations required to perform the pruning. All of which suggests the need for computationallyefficient tight lower bounds. We present three different lower bounds (based on sphericalrelaxation, polytoperelaxation and duality), simulate their performances and discuss their relative merits.https://authors.library.caltech.edu/records/xz0mk7y361Scheduling for Distributed Sensor Networks with Single Sensor Measurement per Time Step
https://resolver.caltech.edu/CaltechAUTHORS:20150211072421791
Authors: Chung, Timothy H.; Gupta, Vijay; Hassibi, Babak; Burdick, Joel; Murray, Richard M.
Year: 2005
DOI: 10.1109/ROBOT.2004.1307149
We examine the problem of distributed estimation when only one sensor can take a measurement per time step. We solve for the optimal recursive estimation algorithm when the sensor switching schedule is given. We then consider the effect of noise in communication channels. We also investigate the problem of determining an optimal sensor switching strategy. We see that this problem involves searching a tree in general and propose two strategies for pruning the tree to minimize the computation. The first is a sliding window strategy motivated by the Viterbi algorithm, and the second one uses thresholding. The performance of the algorithms is illustrated using numerical examples.https://authors.library.caltech.edu/records/6tvvjz0f93Towards reducing the gap between PMEPR of multicarrier and single carrier signals
https://resolver.caltech.edu/CaltechAUTHORS:SHAspawc05
Authors: Sharif, Masoud; Hassibi, Babak
Year: 2005
DOI: 10.1109/SPAWC.2005.1506051
It has recently been shown that by altering the sign of each subcarrier in a multicarrier system significant reduction in the peak to mean envelope power (PMEPR) can be obtained. In fact, the PMEPR can even be made a constant independent of the number of subcarriers n. However, finding the best sign requires a search over 2^n possible signs which is computationally prohibitive. In this paper, we first propose a greedy algorithm to choose the signs based on pnorm minimization and we prove that it can achieve a PMEPR of order log n. We further decrease the PMEPR by enlarging the search space considered by the greedy algorithm. By ignoring peaks with probability less than l0^3, simulation results show that the PMEPR of a multicarrier system with 128 subcarriers each one modulated by 64QAM constellations is reduced to 3.4. This implies that at the cost of one bit of information per subcarrier (i.e., not sending information over the sign of each subcarrier) and modest computational complexity in the transmitter, the PMEPR can be reduced from 12.5 to 3.4 which is within 1.6 dB of the PMEPR of a single carrier system with 64QAM modulation.https://authors.library.caltech.edu/records/8zwbrwq545Throughput analysis in wideband MIMO broadcast channels with partial feedback
https://resolver.caltech.edu/CaltechAUTHORS:20110817075126475
Authors: Fakhereddin, Maralle J.; Sharif, Masoud; Hassibi, Babak
Year: 2005
DOI: 10.1109/SPAWC.2005.1506244
It has been recently shown that opportunistic transmit beamforming using partial channel state information (CSI) achieves the same throughput scaling obtained from dirty paper coding for a broadcast channel with fixed number of transmit antennas and many receivers M. Sharif et al., (2005). In this paper, we study the generalization of this scheme to wideband broadcast channels. By using orthogonal frequency division multiplexing, an Ltap wideband channel can be decomposed to N parallel narrowband channels, where N is larger than L. Neighboring subchannels are therefore highly correlated, and it is intuitive to say that each group of neighboring subchannels (forming a cluster) can be characterized by one channel quality. We show in this paper that users need only feedback the best signaltonoiseplusinterference ratio at the center of each cluster. Our results indicate that for cluster size of order ^N/_(L√(log K)), where K is the number of users, this feedback scheme maintains the same throughput scaling as when full CSI is known. Simulation results show that larger cluster sizes (N/(2L)) can also be implemented for a small throughput hit.https://authors.library.caltech.edu/records/k961katx70On LQG control across a stochastic packetdropping link
https://resolver.caltech.edu/CaltechAUTHORS:20150210072226646
Authors: Gupta, Vijay; Spanos, Demetri; Hassibi, Babak; Murray, Richard M.
Year: 2005
DOI: 10.1109/ACC.2005.1469960
In this paper, we consider the problem of
optimal Linear Quadratic Gaussian control of a system
in which communication between the sensor and the
controller occurs across a packetdropping link. We first
prove a separation principle that allows us to solve this
problem using a standard LQR statefeedback design,
along with an optimal algorithm for propagating and
using the information across the unreliable link. Then
we present one such optimal algorithm, which consists
of a Kalman filter at the sensor side of the link, and a
switched linear filter at the controller side. Our design
does not assume any statistical model of the packet drop
events, and is thus optimal for any arbitrary packet
drop pattern. Further, the solution is appealing from a
practical point of view because it can be implemented as
a small modification of an existing LQG control design.https://authors.library.caltech.edu/records/etf1773312Invited paper: An efficient H∞ estimation approach to speed up the sphere decoder
https://resolver.caltech.edu/CaltechAUTHORS:20110825134003288
Authors: Stojnic, Mihailo; Vikalo, Haris; Hassibi, Babak
Year: 2005
DOI: 10.1109/WIRLES.2005.1549632
Maximumlikelihood (ML) decoding often reduces to solving an integer leastsquares problem, which is NP hard in the worstcase. On the other hand, it has recently been shown that, over a wide range of dimensions and signaltonoise ratios (SNR), the sphere decoding algorithm finds the exact solution with an expected complexity that is roughly cubic in the dimension of the problem. However, the computational complexity of sphere decoding becomes prohibitive if the SNR is too low and/or if the dimension of the problem is too large. In this paper, we target these two regimes and attempt to find faster algorithms by pruning the search tree beyond what is done in the standard sphere decoder. The search tree is pruned by computing lower bounds on the possible optimal solution as we proceed to go down the tree. Using ideas from H∞ estimation theory, we have developed a general framework to compute the lower bound on the integer leastsquares. Several special cases of lower bounds were derived from this general framework. Clearly, the tighter the lower bound, the more branches can be eliminated from the tree. However, finding a tight lower bound requires significant computational effort that might diminish the savings obtained by additional pruning. In this paper, we propose the use of a lower bound which can be computed with only linear complexity. Its use for tree pruning results in significantly speeding up the basic sphere decoding algorithm.https://authors.library.caltech.edu/records/a13sch9y38An Hinfinity based lower bound to speed up the sphere decoder
https://resolver.caltech.edu/CaltechAUTHORS:20150210071542063
Authors: Stojnic, M.; Vikalo, H.; Hassibi, B.
Year: 2005
DOI: 10.1109/SPAWC.2005.1506240
It is well known that maximumlikelihood (ML) decoding in many digital communication schemes reduces to solving an integer least problem, which is NP hard in the worstcase. On the other hand, it has recently been shown that, over a wide range of dimensions and signaltonoise ratios (SNR), the sphere decoder can be used to find the exact solution with an expected complexity that is roughly cubic in the dimension of the problem. However, the computational complexity of sphere decoding becomes prohibitive if the SNR is too low and/or if the dimension of the problem is too large. In recent work M. Stonjic et al. (2005), we have targeted these two regimes and attempted to find faster algorithms by employing a branchandbound technique based on convex relaxations of the original integer leastsquares problem. In this paper, using ideas from H∞ estimation theory, we propose new lower bounds that are generally tighter than the ones obtained in M. Stonjic et al. (2005). Simulation results snow the advantages, in terms of computational complexity, of the new H∞based branchandbound algorithm over the ones based on convex relaxation, as well as the original sphere decoder.https://authors.library.caltech.edu/records/48qhgj5y43The capacity region of multiple input erasure broadcast channels
https://resolver.caltech.edu/CaltechAUTHORS:20150210070820341
Authors: Dana, Amir F.; Hassibi, Babak
Year: 2005
DOI: 10.1109/ISIT.2005.1523761
In this paper, we look at the capacity region of a
special class of broadcast channels with multiple inputs at the transmitter and a number of receivers. The channel between an input of the transmitter and a receiver is modelled as an independent memoryless erasure channel. We assume that the signals coming from different inputs to the receiver do not interfere with each other. Also for each input, the transmitter sends the same signal through the channels outgoing from that input. This class of broadcast channels does not necessarily belong to the class of "more capable". We will show that the capacity region of these broadcast channels is achieved by timesharing between the receivers at each input. Finally, the implications of these results to the more general network setup are discussed.https://authors.library.caltech.edu/records/r6b0wf6q60Differentiated rate scheduling for Gaussian broadcast channels
https://resolver.caltech.edu/CaltechAUTHORS:SHAisit05
Authors: Sharif, Masoud; Dana, Amir F.; Hassibi, Babak
Year: 2005
DOI: 10.1109/ISIT.2005.1523733
In this paper, we consider a fading broadcast channel where users have different rate demands. In particular, we assume users are divided into M groups, each group of which requires the same rate, and where the ratio of the rates of the groups are given. The transmitter would like to maximize the throughput (sum of the rates to all users) while maintaining the rational rate constraints. In general, this problem appears to be computationally intractable since the ergodic capacity region is described as the convex hull of (an infinite) set of rates. In this paper, we therefore, focus on the asymptotic regime of many users (large n) where explicit results can be found. In particular, we propose three scheduling schemes to provide the rational rate constraints namely, weighted opportunistic (WO), time division opportunistic (TO), and superposition coding (SC). The WO scheduling is a generalization of the opportunistic scheduling in which we transmit to only the user that has the maximum weighted signal to noise ratio (SNR). In TO, each group has its own time slot in which the transmitter chooses the user with the best SNR from the corresponding group. Superposition coding is the one that achieves the capacity region. For each scheduling we give explicit scheme to guarantee the rational rate constraints. We also analyze the throughput loss due to rate constraints for different schemes. In particular, we show that the throughput loss compared to the maximum throughput (i.e., the sum rate capacity without any rate constraints) tends to zero for large n, and finally, we analyze the convergence rate of all the schemes.https://authors.library.caltech.edu/records/6m5s0jhj84Cooperative diversity in wireless relay networks with multipleantenna nodes
https://resolver.caltech.edu/CaltechAUTHORS:20150210070058052
Authors: Jing, Yindi; Hassibi, Babak
Year: 2005
DOI: 10.1109/ISIT.2005.1523450
In [1], the idea of distributed spacetime coding
was proposed to achieve a degree of cooperative diversity in
a wireless relay network. In particular, for a relay network with a singleantenna transmitter and receiver and
R singleantenna relays, it was shown that the pairwise error probability (PEP) decays as ((log P)/P)^R, where
P is the total transmit power. In this paper, we extend the results to wireless relay networks where the transmitter, receiver, and/or relays may have multiple antennas.
Assuming that the transmitter has M antennas, the receiver has N antennas, the sum of all the antennas at the relay nodes is R, and the coherence interval is long enough, we show that the PEP behaves as (1/P)^(min{M,N}R), if
M ≠ N, and ((log^(1/M)P)/p)^(MR), if M=N. Therefore, for the case of M ≠ N, distributed spacetime coding has the same PEP performance as a multipleantenna system with
min{M, N}R transmit and a single receive antenna.
For the case of M = N, the penalty on the PEP compared to a
multipleantenna system is a log^(1/M) P factor, which is negligible at high SNR. We also show that for a fixed total transmit power across the entire network, the optimal power allocation is for the transmitter to expend half the power and for the relays to share the other half with the power used by each relay being proportional to the number of antennas it has.https://authors.library.caltech.edu/records/gcznmhy415Bounds on the performance of sphere decoding of linear block codes
https://resolver.caltech.edu/CaltechAUTHORS:20110725143155106
Authors: ElKhamy, Mostafa; Vikalo, Haris; Hassibi, Babak
Year: 2005
DOI: 10.1109/ITW.2005.1531854
A sphere decoder searches for the closest lattice point within a certain search radius. The search radius provides a tradeoff between performance and complexity. We derive tight upper bounds on the performance of sphere decoding of linear block codes. The performance of softdecision sphere decoding on AWGN channels as well as that of harddecision sphere decoding on binary symmetric channels is analyzed.https://authors.library.caltech.edu/records/yr20jwkw42An achievability result for random networks
https://resolver.caltech.edu/CaltechAUTHORS:20150210070353237
Authors: Gowaikar, Radhika; Hochwald, Bertrand; Hassibi, Babak
Year: 2005
DOI: 10.1109/ISIT.2005.1523477
We analyze a network of nodes in which pairs communicate over a shared wireless medium. We are interested
in the maximum total aggregate traffic flow that is possible
through the network. Our model differs substantially from the many existing approaches in that the channel connections in our network are entirely random: we assume that, rather than being governed by geometry and a decay law, the strength of the connections between nodes is drawn independently from a common distribution. Such a model is appropriate for environments where the first order effect that governs the signal strength at a receiving
node is a random event (such as the existence of an obstacle), rather than the distance from the transmitter.
We show that the aggregate traffic flow is a strong function of the channel distribution. In particular, we show that for certain distributions, the aggregate traffic flow scales at least as n/((log n)^v) for some fixed
v > 0, which is significantly larger than the
O(√n) results obtained for many geometric models.https://authors.library.caltech.edu/records/asfebaxw08State estimation utilizing multiple description coding over lossy networks
https://resolver.caltech.edu/CaltechAUTHORS:20110818111717130
Authors: Jin, Zhipu; Gupta, Vijay; Hassibi, Babak; Murray, Richard M.
Year: 2005
DOI: 10.1109/CDC.2005.1582267
For state estimation in networked control systems,
the impact of packet dropping over network links is an important problem. In this paper, we introduce multiple description (MD) source coding scheme to improve the statistical stability and performance of the estimation error covariance of Kalman filter with packet loss. We consider about two cases: when the packet loss over network links occurs in an i.i.d. fashion or in a bursty fashion. Compared with the traditional single description source coding, MD coding scheme can greatly improve the
performance of Kalman filter over a large set of packet loss
scenarios in both cases.https://authors.library.caltech.edu/records/nz65n12k40On Sensor Fusion in the Presence of Packetdropping Communication Channels
https://resolver.caltech.edu/CaltechAUTHORS:20110817110410615
Authors: Gupta, Vijay; Hassibi, Babak; Murray, Richard M.
Year: 2005
DOI: 10.1109/CDC.2005.1582712
In this paper we look at the problem of multisensor data fusion when data is being communicated over channels that drop packets randomly. We are motivated by the use of wireless links for communication among nodes in upcoming sensor networks. We wish to identify the information that should be communicated by each node to others given that some of the information it had transmitted earlier might have been lost. We solve the problem exactly for the case of two sensors and study the performance of the algorithm when more sensors are present. For the twosensor case, the performance of our algorithm is optimal in the sense that if a packet is received from the other sensor, it is equivalent to receiving all previous measurements, irrespective of the packet drop pattern.https://authors.library.caltech.edu/records/5hdpq54347RateDiversity Tradeoffs in Interference Channels with and without Cooperation
https://resolver.caltech.edu/CaltechAUTHORS:20110721161610004
Authors: Rao, Chaitanya; Hassibi, Babak
Year: 2006
DOI: 10.1109/ACSSC.2006.354834
We study the Gaussian interference channel in which independent and identically distributed Rayleigh fading
exists between any two nodes. Firstly we find the ratediversity relationship for three wellknown transmission/reception strategies. Out of these we see the best tradeoff is obtained when each receiver decodes both transmitter messages jointly, akin to two multiple access channels.
With a view to increasing diversity two more schemes are
then considered for the interference channel, now allowing
cooperation between nodes. One scheme is shown to increase
diversity up to a factor of three while reducing rate by four, while the other doubles diversity but cuts rate by a factor of three.https://authors.library.caltech.edu/records/1dqdxbcv15On the throughput of broadcast channels with imperfect CSI
https://resolver.caltech.edu/CaltechAUTHORS:20110728112210752
Authors: Vakili, Ali; Hassibi, Babak
Year: 2006
DOI: 10.1109/SPAWC.2006.346484
In a broadcast channel in which one transmitter provides
independent streams of data for n receivers, the amount of
channel state information (CSI) at the transmitter can affect
the capacity region dramatically. Having the accurate SNR
of all users at the transmitter, the opportunistic strategy can
be used to maximize the throughput (sumrate) of the system.
However, evaluating the SNR is basically an estimation problem
at the receiver which in practice is not error free. In this
paper, we analyze the effect of the noisy estimation of SNR on
the throughput of a broadcast channel. We propose a modified
opportunistic scheme in which the transmitter serves the user
with the highest estimated SNR, but backs off on the transmit
rate based on the variance of the estimation error. This
scheme can achieve the maximum achievable rate while the
CSI is limited to a noisy estimation of channel coefficients.
We obtain the optimum rate back off and find the asymptotic
behavior of the throughput under the scheduling scheme we
have proposed. We show that the effect of a nonzero estimation
error variance can be modeled as an SNR hit proportional
to this variance.https://authors.library.caltech.edu/records/vdfxy2f969On the capacity region of broadcast over wireless erasure networks
https://resolver.caltech.edu/CaltechAUTHORS:20150223072830545
Authors: Dana, Amir F.; Gowaikar, Radhika; Hassibi, Babak
Year: 2006
In this paper, we consider a special class of wireless networks, called wireless erasure networks. In these networks, each node is connected to a set of nodes by independent erasure channels. The network model incorporates the broadcast nature of the wireless environment in that each node sends out the same signal on its outgoing channels. However, we assume there is no interference in reception. In this paper we first look at the single source single destination unicast problem. We obtain the capacity under the assumption that erasure locations on all the links of the network are provided to the destination. It turns out that the capacity has a nice maxflow mincut interpretation. The definition of cutcapacity in these network is such that it incorporates the broadcast property of the wireless medium. In the second part of the paper, a timesharing scheme for broadcast problems over these networks is proposed and its achievable region is analyzed. We show that for some special cases, this timesharing scheme is optimal.https://authors.library.caltech.edu/records/p6k5kd4w64MultipleAntenna Systems and Wireless Networks
https://resolver.caltech.edu/CaltechAUTHORS:20110217115842248
Authors: Hassibi, Babak
Year: 2006
DOI: 10.1109/RWS.2006.1615095
The use of multiple antennas, at the transmitter, receiver or both, promises both higher data rates as well as greater reliability in pointtopoint communication channels. While this is now well understood, it is only recently that the impact of multiple antenna systems in a wireless network is being systematically assessed. In this paper, using the example of the downlink of a cellular system, we review some of the issues related to throughput, delay and scheduling that arise when multiple antennas are deployed at the base station or at the user terminals.https://authors.library.caltech.edu/records/bad8cejr59MIMO Receive Algorithms
https://resolver.caltech.edu/CaltechAUTHORS:20150223070309489
Authors: Kailath, Thomas; Vikalo, Haris; Hassibi, Babak
Year: 2006
DOI: 10.1017/CBO9780511616815.016
The optimal detection problem in multiantenna wireless communication systems often reduces to the problem of finding the leastsquares solution to a system of linear equations, where the unknown vector is comprised of integers, but the matrix coefficients and the given vector are realvalued. The problem is equivalent to finding the closest lattice point to a given point and is known to be NPhard. We review the most commonly used solution techniques, and discuss their computational complexity. Among heuristic algorithms, we focus on the nulling and cancelling techniques, and their fast implementations based on linear estimation theory. We then show that an exact method, the sphere decoding algorithm, often has expected complexity implementable in practical systems. We also describe extensions of sphere decoding techniques to receivers that make use of the socalled soft information.https://authors.library.caltech.edu/records/atcgz1jj84Further Results on Speeding up the Sphere Decoder
https://resolver.caltech.edu/CaltechAUTHORS:20110726144521891
Authors: Stojnic, M.; Vikalo, H.; Hassibi, B.
Year: 2006
DOI: 10.1109/ICASSP.2006.1661027
In many communication applications,maximumlikelihood
decoding reduces to solving an integer leastsquares problem
which is NP hard in the worstcase. On the other hand,
it has recently been shown that, over a wide range of dimensions
and SNR, the sphere decoder can be used to find the
exact solution with an expected complexity that is roughly
cubic in the dimension of the problem. However, the computational
complexity becomes prohibitive if the SNR is too
low and/or if the dimension of the problem is too large.
In earlier work, we targeted these two regimes attempting
to find faster algorithms by pruning the search tree beyond
what is done in the standard sphere decoder. The search tree
is pruned by computing lower bounds on the possible optimal
solution as we proceed to go down the tree. A tradeoff
between the computational complexity required to compute
the lower bound and the size of the pruned tree is readily
observed: the more effort we spend in computing a tight
lower bound, the more branches that can be eliminated in
the tree. Thus, even though it is possible to prune the search
tree (and hence the number of points visited) by several orders
of magnitude, this may be offset by the computations
required to perform the pruning. In this paper, we propose a
computationally efficient lower bound which requires solving
a single semidefinite program (SDP) at the top of the
search tree; the solution to the SDP is then used to deduce
the lower bounds on the optimal solution on all levels of
the search tree. Simulation results indicate significant improvement
in the computational complexity of the proposed
algorithm over the standard sphere decoding.https://authors.library.caltech.edu/records/dvxhave013Differentiated Rate Scheduling for MIMO Broadcast Channels with Estimation Errors
https://resolver.caltech.edu/CaltechAUTHORS:20110811132404970
Authors: Vakili, Ali; Dana, Amir F.; Hassibi, Babak
Year: 2006
DOI: 10.1109/ACSSC.2006.354859
We consider the throughput of a MIMO Gaussian broadcast channel with two generalizing assumptions: (1) differentiated quality of service, in the sense that the rates required by different users must satisfy certain rational constraints, and (2) imperfect channel side information (CSI), where we assume that the estimate of the underlying channel has some error with known distribution. Theoretically, with full and perfect CSI in hand, dirty paper coding (DPC) can achieve any point on the capacity region, including the nonsymmetrical boundary points. However, the full CSI requirement and the computational complexity of DPC motivates the development of simpler schemes which require little feedback from the users and can obtain a large portion of the capacity of the channel. In this paper we will look at the throughput and rate ratio achieved by schemes based on the idea of opportunistic beamforming and employing a rate backoff mechanism in order to maximize the throughput in the case of imperfect CSI. We will determine the optimal scheduling parameters and will show the order optimality of these schemes in the regime of large number of users.https://authors.library.caltech.edu/records/qrte1hst94Asymptotic Analysis of the Gaussian Broadcast Channel with Perturbation Preprocessing
https://resolver.caltech.edu/CaltechAUTHORS:20110727110357938
Authors: Stojnic, M.; Vikalo, H.; Hassibi, B.
Year: 2006
DOI: 10.1109/ICASSP.2006.1661084
The sum rate capacity of the multiantenna Gaussian broadcast channel has recently been computed. However, the search for computationally efficient practical schemes that achieve it is still in progress. When the channel state information is fully available at the transmitter, the dirty paper coding (DPC) technique is known to achieve the maximal throughput, but is computationally infeasible. In this paper, we analyze the asymptotic behavior of one of its alternatives – the recently suggested socalled vector perturbation technique. We show that for a square channel, where the number of users is large and equal to the number of transmit antennas, its sum rate approaches
that of the DPC technique. More precisely, we show
that at both low and high signaltonoise ratio (SNR), the
scheme under consideration is asymptotically optimal. Furthermore, we obtain similar results in the case where the number of users is much larger than the number of transmit antennas.https://authors.library.caltech.edu/records/2zv3h99z71An Algebraic Family of Distributed SpaceTime Codes for Wireless Relay Networks
https://resolver.caltech.edu/CaltechAUTHORS:20110614105704234
Authors: Oggier, Frédérique; Hassibi, Babak
Year: 2006
DOI: 10.1109/ISIT.2006.261774
This paper studies the design of distributed spacetime
codes for use in wireless relay networks. Earlier work
suggested that a suitable family of codes can be obtained by
using linear dispersion codes, provided the basis matrices were
unitary. In this paper we construct an explicit algebraic family
of such codes where full diversity is proved. The construction
uses cyclotomic field theory and yields basis matrices that are
indeed unitary. Simulation results show that the codes have better
performance than codes designed earlier by ad hoc and random
methods, and thus with less encoding complexity.https://authors.library.caltech.edu/records/g19z0qq782On the Performance of Sphere Decoding of Block Codes
https://resolver.caltech.edu/CaltechAUTHORS:20170508174050826
Authors: ElKhamy, Mostafa; Vikalo, Haris; Hassibi, Babak; McEliece, R. J.
Year: 2006
DOI: 10.1109/ISIT.2006.261824
The performance of sphere decoding of block codes over a variety of channels is investigated. We derive a tight bound on the performance of maximum likelihood decoding of linear codes on qary symmetric channels. We use this result to bound the performance of qary hard decision sphere decoders. We also derive a tight bound on the performance of soft decision sphere decoders on the AWGN channel for BPSK and MPSK modulated block codes. The performance of soft decision sphere decoding of arbitrary finite lattices or block codes is also analyzed.https://authors.library.caltech.edu/records/8jm324qt88On the Achievable Throughput in TwoScale Wireless Networks
https://resolver.caltech.edu/CaltechAUTHORS:20170511123623758
Authors: Gowaikar, Radhika; Hassibi, Babak
Year: 2006
DOI: 10.1109/ISIT.2006.261641
We propose a new model of wireless networks which we refer to as "twoscale networks". At a local scale, characterized by nodes being within a distance r, channel strengths are drawn independently and identically from a distanceindependent distribution. At a global scale, characterized by nodes being further apart from each other than a distance r, channel connections are governed by a Rayleigh distribution, with the power satisfying a distancebased decay law. Thus, at a local scale, channel strengths are determined primarily by random effects such as obstacles and scatterers whereas at the global scale channel strengths depend on distance.
For such networks, we propose a hybrid communications scheme, combining elements of P. Gupta et al. (2000) (for distancedependent networks) and R. Gowaikar et al. (2006) (for random networks). For a particular class of twoscale networks with N nodes, we show that an aggregate throughput of the form N^(1/(t1)) / (log^2 N) is achievable, where t > 2 is a parameter that depends on the distribution of the connection at the local scale and is independent of the decay law that operates at a global scale. For t < 3, this offers a significant improvement over the O(√N) results of P. Gupta et al. (2000).https://authors.library.caltech.edu/records/5h1c5fey14On the effect of quantization on performance at high rates
https://resolver.caltech.edu/CaltechAUTHORS:GUPacc06
Authors: Gupta, Vijay; Dana, Amir F.; Murray, Richard M.; Hassibi, Babak
Year: 2006
DOI: 10.1109/ACC.2006.1656407
We study the effect of quantization on the performance of a scalar dynamical system in the high rate regime. We evaluate the LQ cost for two commonly used quantizers: uniform and logarithmic and provide a lower bound on performance of any centroidbased quantizer based on entropy arguments. We also consider the case when the channel drops data packets stochastically.https://authors.library.caltech.edu/records/vwspgf5d57On Limits of Performance of DNA Microarrays
https://resolver.caltech.edu/CaltechAUTHORS:VIKicassp06
Authors: Vikalo, H.; Hassibi, B.; Hassibi, A.
Year: 2006
DOI: 10.1109/ICASSP.2006.1660517
DNA microarray technology relies on the hybridization process which is stochastic in nature. Probabilistic crosshybridization of nonspecific targets, as well as the shotnoise originating from specific targets binding, are among the many obstacles for achieving high accuracy in DNA microarray analysis. In this paper, we use statistical model of hybridization and crosshybridization processes to derive a lower bound (viz., the CramerRao bound) on the minimum meansquare error of the target concentrations estimation. A preliminary study of the CramerRao bound for estimating the target concentrations suggests that, in some regimes, crosshybridization may, in fact, be beneficial—a result with potential ramifications for probe design, which is currently focused on minimizing crosshybridization.https://authors.library.caltech.edu/records/9stgb72w76A Coding Strategy for Wireless Networks with no Channel Information
https://resolver.caltech.edu/CaltechAUTHORS:20150223072516823
Authors: Oggier, Frédérique; Hassibi, Babak
Year: 2006
In this paper, we present a coding strategy for wireless relay networks, where we assume no channel knowledge. More precisely, the relays operate without knowing the channel that affected their received signal, and the receiver decodes knowing none of the channel paths. The coding scheme is inspired by noncoherent differential spacetime coding, and is shown to yield a diversity linear in the number of relays. It is furthermore available for any number of relay nodes.https://authors.library.caltech.edu/records/s9g9srew74The Effect of Channel Estimation Error on the Throughput of Broadcast Channels
https://resolver.caltech.edu/CaltechAUTHORS:VAKicassp06
Authors: Vakili, Ali; Sharif, Masoud; Hassibi, Babak
Year: 2006
DOI: 10.1109/ICASSP.2006.1660897
In a broadcast channel in which one transmitter serves n receivers, the capacity region highly depends on the amount of channel state information (CSI) at the transmitter. Assuming that the transmitter knows the SNR of all the receivers, opportunistic strategy maximizes the throughput (sumrate) of the system. It is usually assumed that CSI is accurate, however, evaluating the SNR is basically an estimation problem in the receiver which cannot be done without error. In this paper, we analyze the effect of the noisy estimation of SNR on the throughput of a broadcast channel. We propose a generalization of the opportunistic transmission in which the transmitter still sends to the user with the highest estimated SNR, but backs off on the transmit rate based on the variance of the estimation error. We obtain the optimum amount of back off and compute the throughput for our scheduling scheme. Clearly, the estimation can be improved by using a longer training phase; however, longer training would deteriorate the throughput. In the final part of the paper, we address this trade off and obtain the optimum training strategy that maximizes the throughput of the system.https://authors.library.caltech.edu/records/hh73m5fp56On the Capacity Region of MultiAntenna Gaussian Broadcast Channels with Estimation Error
https://resolver.caltech.edu/CaltechAUTHORS:DANisit06
Authors: Dana, Amir F.; Sharif, Masoud; Hassibi, Babak
Year: 2006
DOI: 10.1109/ISIT.2006.261755
In this paper we consider the effect of channel estimation error on the capacity region of MIMO Gaussian broadcast channels. It is assumed that the receivers and the transmitter have (the same) estimates of the channel coefficients (i.e., the feedback channel is noiseless). We obtain an achievable rate region based on the dirty paper coding scheme. We show that this region is given by the capacity region of a dual multiaccess channel with a noise covariance that depends on the transmit power. We explore this duality to give the asymptotic behavior of the sumrate for a system with a large number of user, i.e., n rarr infin. It is shown that as long as the estimation error is of fixed (w.r.t n) variance, the sumcapacity is of order M log log n, where M is the number of antennas deployed at the transmitter. We further obtain the sumrate loss due to the estimation error. Finally, we consider a trainingbased scheme for block fading MISO Gaussian broadcast channels. We find the optimum length of the training interval as well as the optimum power used for training in order to maximize the achievable sumrate.https://authors.library.caltech.edu/records/d5dgv8yw27How Much Does Transmit Correlation Affect the SumRate of MIMO Downlink Channels?
https://resolver.caltech.edu/CaltechAUTHORS:NAFisit06
Authors: AlNaffouri, Tareq Y.; Sharif, Masoud; Hassibi, Babak
Year: 2006
DOI: 10.1109/ISIT.2006.261541
This paper considers the effect of spatial correlation between transmit antennas on the sumrate capacity of the MIMO broadcast channel (i.e., downlink of a cellular system). Specifically, for a system with a large number of users n, we analyze the scaling laws of the sumrate for the dirty paper coding and for different types of beamforming transmission schemes. When the channel is i.i.d., it has been shown that for large n, the sum rate is equal to M log log n + M log P/M + o(1) where M is the number of transmit antennas, P is the average signal to noise ratio, and o(1) refers to terms that go to zero as n rarr infin. When the channel exhibits some spatial correlation with a covariance matrix R (nonsingular with tr(R) = M), we prove that the sum rate of dirty paper coding is M log log n + M log P/M + log det(R) + o(1). We further show that the sumrate of various beamforming schemes achieves M log log n + M log P/M + M log c + o(1) where c les 1 depends on the type of beamforming. We can in fact compute c for random beamforming proposed in M. Sharif et al. (2005) and more generally, for random beamforming with preceding in which beams are premultiplied by a fixed matrix. Simulation results are presented at the end of the paper.https://authors.library.caltech.edu/records/w5k5bpwj77Unambiguous discrimination between two rank2 mixed quantum states
https://resolver.caltech.edu/CaltechAUTHORS:20150225072454293
Authors: Stojnic, Mihailo; Hassibi, Babak
Year: 2007
DOI: 10.1109/ISIT.2007.4557236
In this paper we consider the problem of the optimal quantum unambiguous detection between two mixed quantum states. More specifically, we consider two mixed quantum states of rank 2 which lie in a Hilbert space of dimension 4. Using duality theory we explicitly characterize the optimal measurement operators. Furthermore, as a byproduct of our framework we obtain a closed form solution of unambiguous discrimination between a pure and a mixed quantum state.https://authors.library.caltech.edu/records/7dn0z64r93Signal Processing for RealTime DNA Microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20150225071313817
Authors: Vikalo, Haris; Hassibi, Babak; Hassibi, Arjang
Year: 2007
DOI: 10.1109/ACSSC.2007.4487188
In conventional fluorescentbased microarrays, data is acquired after the completion of the hybridization phase. In this phase the target analytes (i.e., DNA fragments) bind to the capturing probes on the array and supposedly reach a steady state. Accordingly, microarray experiments essentially provide only a single, steadystate data point of the hybridization process. On the other hand, a novel technique (i.e., realtime microarrays) capable of recording the kinetics of hybridization in fluorescentbased microarrays has recently been proposed in [1]. The richness of the information obtained therein promises higher signaltonoise ratio, smaller estimation error, and broader assay detection dynamic range compared to the conventional microarrays. In the current paper, we model the kinetics of the hybridization process measured by the realtime microarrays, and develop techniques for estimating the amounts of analytes present therein.https://authors.library.caltech.edu/records/cb41ey1061PEP analysis of SDPbased noncoherent signal detection
https://resolver.caltech.edu/CaltechAUTHORS:20150225072044080
Authors: Stojnic, Mihailo; Hassibi, Babak; Vikalo, Haris
Year: 2007
DOI: 10.1109/ISIT.2007.4557661
In multiantenna communication systems, channel information is often not known at the receiver. To fully exploit the bandwidth resources of the system and ensure the practical feasibility of the receiver, the channel parameters are often estimated and then employed in the design of signal detection algorithms. However, sometimes communication can occur in the environment where learning the channel coefficients becomes infeasible. In this paper we consider the problem of maximumlikelihood (ML)detection in singleinput multipleoutput (SIMO) systems when the channel information is completely unavailable at the receiver and when employed signalling at the transmitter is qPSK. It is well known that finding the solution to this optimization requires solving an integer maximization of a quadratic form and is, in general, an NP hard problem. To solve it, we propose an approximate algorithm based on the semidefinite program (SDP) relaxation. We derive a bound on the pairwise probability of error (PEP) of the proposed algorithm and show that, the algorithm achieves the same diversity as the exact maximumlikelihood (ML) decoder. Furthermore, we prove that in the limit of large system dimension this bound differs from the corresponding one in the exact ML case by at most 3.92 dB if the transmitted symbols are from 2 or 4PSK constellations and by at most 2.55 dB if the transmitted symbols are from 8PSK constellation. This suggests that the proposed algorithm requires moderate increase in the signaltonoise ratio (SNR) in order to achieve performance comparable to that of the ML decoder but with often significantly lower complexity.https://authors.library.caltech.edu/records/q13nrs4849PEP Analysis of the SDP Based Joint Channel Estimation and Signal Detection
https://resolver.caltech.edu/CaltechAUTHORS:20150209071636394
Authors: Stojnic, M.; Hassibi, B.; Vikalo, H.
Year: 2007
DOI: 10.1109/ICASSP.2007.366474
In multiantenna communication systems, channel information is often not known at the receiver. To fully exploit bandwidth resources of the system and ensure practical feasibility of the receiver, channel parameters are often estimated blindly and then employed in the design of signal detection algorithms. Instead of separating channel estimation from signal detection, in this paper we focus on the joint channel estimation and signal detection problem in a singleinput multipleoutput (SIMO) system. It is well known that finding solution to this optimization requires solving an integer maximization of a quadratic form and is, in general, an NP hard problem. To solve it, we propose an approximate algorithm based on the semidefinite program (SDP) relaxation. We derive a bound on the pairwise probability of error (PEP) of the proposed algorithm and show that, the algorithm achieves the same diversity as the exact maximumlikelihood (ML) decoder. The computed PEP implies that, over a wide range of system parameters, the proposed algorithm requires moderate increase in the signaltonoise ratio (SNR) in order to achieve performance comparable to that of the ML decoder but with often significantly lower complexityhttps://authors.library.caltech.edu/records/tenvj24x66Outsphere decoder for noncoherent ML SIMO detection and its expected complexity
https://resolver.caltech.edu/CaltechAUTHORS:20101105103125338
Authors: Stojnic, M.; Hassibi, B.
Year: 2007
DOI: 10.1109/ACSSC.2007.4487494
In multiantenna communication systems, channel information
is often not known at the receiver. To fully exploit
the bandwidth resources of the system and ensure the practical
feasibility of the receiver, the channel parameters are
often estimated and then employed in the design of signal
detection algorithms. However, sometimes communication
can occur in an environment where learning the channel coefficients
becomes infeasible. In this paper we consider the
problem of maximumlikelihood (ML)detection in singleinput
multipleoutput (SIMO) systems when the channel information
is completely unavailable at the receiver and when
the employed signalling at the transmitter is qPSK. It is
well known that finding the solution to this optimization requires
solving an integer maximization of a quadratic form
and is, in general, an NP hard problem. To solve it, we propose
an exact algorithm based on the combination of branch
and bound tree search and semidefinite program (SDP) relaxation.
The algorithm resembles the standard sphere decoder
except that, since we are maximizing we need to construct
an upper bound at each level of the tree search. We
derive an analytical upper bound on the expected complexity
of the proposed algorithm.https://authors.library.caltech.edu/records/y9vntcfc02On the throughput of opportunistic beamforming with imperfect CSI
https://resolver.caltech.edu/CaltechAUTHORS:20150209072210248
Authors: Vakili, Ali; Dana, Amir F.; Hassibi, Babak
Year: 2007
DOI: 10.1145/1280940.1280945
The throughput of a multipleantenna broadcast channel
highly depends on the channel state information (CSI) at the
transmitter side. However, due to the time variant nature
of wireless channels, having perfect knowledge of the under
lying links appears to be a questionable assumption, especially when the number of users and/or antennas increases. Although it can become computationally prohibitive in practice, theoretically any point on the capacity region of a Gaussian broadcast channel is achievable using dirty paper coding (DPC) if full CSI is available.
The aforementioned drawbacks of DPC have motivated
the development of simpler transmission strategies that re
quire little CSI and yet can deliver a large portion of the capacity. One such scheme is opportunistic beamforming
that is shown to be able to achieve the same throughput scaling as that of DPC for the regime of large number of users. In this paper we investigate the performance of opportunistic beamforming when the perfect channel state information is not available; i.e., the channel estimation is erroneous. We will show that in order to maximize the throughput (sum rate capacity), the transmitter needs to back off the rate than what is suggested by the estimated channel state. We obtain the optimal back off and show that by using this modified opportunistic scheme, the same multiuser gain can be achieved.https://authors.library.caltech.edu/records/58hryz1w21On the Capacity Scalings of the Multiple Antenna GroupBroadcast Systems
https://resolver.caltech.edu/CaltechAUTHORS:20150205074459178
Authors: Dana, Amir F.; AlNaffouri, Tareq Y.; Hassibi, Babak
Year: 2007
DOI: 10.1109/ISIT.2007.4557318
In this paper, we consider a multiuser system called the groupbroadcast system. In this scenario the users are divided into different groups. Users in each group are interested in a common information independent from that of other groups. Such a situation occurs for example in digital audio and video broadcast systems where the users are divided into various groups according to the shows they are interested in. The paper first obtains upper and lower bounds for the sum rate capacity. Then it looks at system capacity for the large number of users regime and fixed number of antennas. Finally, the case when the number of users and antennas grow simultaneously is studied. It is shown that in order to achieve a constant rate per user the number of transmit antennas should scale at least logarithmically in the number of users.https://authors.library.caltech.edu/records/7rfcekv673On Recovery of Sparse Signals in Compressed DNA Microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20150225071034629
Authors: Vikalo, H.; Parvaresh, F.; Hassibi, B.
Year: 2007
DOI: 10.1109/ACSSC.2007.4487303
Currently, DNA micro arrays comprising tens of thousands of probe spots are employed to test entire genomes in a single experiment. Typically, each microarray spot contains a large number of copies of a single probe, and hence collects only a single data point. This is a wasteful use of the sensing resources in comparative DNA microarray experiments, where a test sample is measured relative to a reference sample. Since only a small fraction of the total number of genes represented by the two samples is differentially expressed, a large fraction of a microarray does not provide any useful information. To this end, in this paper we consider an alternative microarray design wherein each spot is a composite of several different probes, and the total number of spots is potentially much smaller than the number of genes being tested. Fewer spots directly translates to significantly lower costs due to cheaper array manufacturing, simpler image acquisition and processing, and smaller amount of genomic material needed for experiments. To recover signals from compressed microarray measurements, we leverage ideas from compressive sampling. Experimental verification of the proposed methodology is presented.https://authors.library.caltech.edu/records/9jakktyn59High Diversity Scheme for Wireless Networks based on Interference Cancellation
https://resolver.caltech.edu/CaltechAUTHORS:20150225071524441
Authors: Rao, Chaitanya; Hassibi, Babak
Year: 2007
DOI: 10.1109/ISIT.2007.4557395
Consider a wireless network with m transmitterreceiver pairs and an additional n relay nodes to assist communication. We are interested in the rate/diversity tradeoff of such a system. Since the presence of interference is known to reduce diversity significantly, we propose a transmission scheme based on interference cancellation by the relay nodes. This scheme achieves a diversity linear in the number of relay nodes (over all rates up to the maximum possible). Compared to a protocol where receivers decode all transmitted messages, the new scheme is seen to achieve higher diversity at higher rates.https://authors.library.caltech.edu/records/s609ypvz97Further Results on Performance Analysis for Compressive Sensing Using Expander Graphs
https://resolver.caltech.edu/CaltechAUTHORS:20100825134047546
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2007
DOI: 10.1109/ACSSC.2007.4487288
Compressive sensing is an emerging technology which can recover a sparse signal vector of dimension n via a much smaller number of measurements than n. In this paper, we will give further results on the performance bounds of compressive sensing. We consider the newly proposed expander graph based compressive sensing schemes and show that, similar to the l_1 minimization case, we can exactly recover any ksparse signal using only O(k log(n)) measurements, where k is the number of nonzero elements. The number of computational iterations is of order O(k log(n)), while each iteration involves very simple computational steps.https://authors.library.caltech.edu/records/f1wsay8m11Entropy Vectors, Network Information Theory and Wireless Networks
https://resolver.caltech.edu/CaltechAUTHORS:20150302164022192
Authors: Hassibi, Babak
Year: 2007
DOI: 10.1109/WIOPT.2007.4480016
Information theory is well poised to have an impact on the manner in which future networks are designed and maintained, both because wired networks are ripe for applications such as network coding and also because wireless networks cannot be satisfactorily dealt with using conventional networking tools. The challenge is that most network information theory problems are notoriously difficult and so the barriers that must be overcome are often quite high. In particular, there are only a limited number of tools available and so fresh approaches are quite welcome.
We describe an approach based on the definition of the space of "normalized" entropic vectors. In this framework, for a large class of acyclic memoryless networks, the capacity region for an arbitrary set of sources and destinations can be found by maximization of a linear function over the set of channelconstrained normalized entropic vectors and some linear constraints. The key point is that the closure of this set is convex and compact. While this may not necessarily make the problem simpler, it certainly circumvents the "infiniteletter characterization" issue, as well as the nonconvexity of earlier formulations. It also exposes the core of the problem as that of determining the space of normalized entropic vectors.
The approach has several interesting consequences: it allows
one to obtain the classical cutset bounds via a duality
argument; for wired networks, it shows one need only consider the space of unconstrained normalized entropic
vectors, thus separating channel and network codinga result very recently recognized in the community. Outer
bounds to the space of normalized entropic vectors are
known to be related to nonShannon inequalities. We
develop inner bounds on this space using latticegenerated
distributions and show how they can be used to
compute inner bounds on the capacity region of networks
using linear programming.https://authors.library.caltech.edu/records/0zxqw66e09Keynote Speaker
https://resolver.caltech.edu/CaltechAUTHORS:20170419164250781
Authors: Hassibi, Babak
Year: 2007
DOI: 10.1109/WIOPT.2007.4480016
Entropic Vectors, Convex Optimization and Wireless Networks Information theory is well poised to have an impact on the manner in which future networks are designed and maintained, both because wired networks are ripe for applications such as network coding and also because wireless networks cannot be satisfactorily dealt with using conventional networking tools. The challenge is that most network information theory problems are notoriously difficult and so the barriers that must be overcome are often quite high. In particular, there are only a limited number of tools available and so fresh approaches are quite welcome. We describe an approach based on the definition of the space of "normalized" entropic vectors. In this framework, for a large class of acyclic memoryless networks, the capacity region for an arbitrary set of sources and destinations can be found by maximization of a linear function over the set of channelconstrained normalized entropic vectors and some linear constraints. The key point is that the closure of this set is convex and compact. While this may not necessarily make the problem simpler, it certainly circumvents the "infiniteletter characterization" issue, as well as the nonconvexity of earlier formulations. It also exposes the core of the problem as that of determining the space of normalized entropic vectors.https://authors.library.caltech.edu/records/px7ppvz555On joint maximumlikelihood estimation of PCR efficiency and initial amount of target
https://resolver.caltech.edu/CaltechAUTHORS:VIKgensips06
Authors: Vikalo, H.; Hassibi, B.; Hassibi, A.
Year: 2007
DOI: 10.1109/GENSIPS.2006.353149
We consider the problem of estimating unknown parameters of the realtime polymerase chain reaction (RTPCR) from noisy observations. The joint ML estimator of the RTPCR efficiency and the initial number of DNA target molecules is derived. The meansquare error performance of the estimator is studied via simulations. The simulation results indicate that the proposed estimator significantly outperforms a competing technique.https://authors.library.caltech.edu/records/5ezgktx468On a Construction of Entropic Vectors Using LatticeGenerated Distributions
https://resolver.caltech.edu/CaltechAUTHORS:20150205074815340
Authors: Hassibi, Babak; Shadbakht, Sormeh
Year: 2007
DOI: 10.1109/ISIT.2007.4557096
The problem of determining the region of entropic vectors is a central one in information theory. Recently, there has been a great deal of interest in the development of nonShannon information inequalities, which provide outer bounds to the aforementioned region; however, there has been less recent work on developing inner bounds. This paper develops an inner bound that applies to any number of random variables and which is tight for 2 and 3 random variables (the only cases where the entropy region is known). The construction is based on probability distributions generated by a lattice. The region is shown to be a polytope generated by a set of linear inequalities. Study of the region for 4 and more random variables is currently under investigation.https://authors.library.caltech.edu/records/jf2jd7qm75ML Estimation of DNA Initial Copy Number in Polymerase Chain Reaction (PCR) Processes
https://resolver.caltech.edu/CaltechAUTHORS:VIKicassp07
Authors: Vikalo, H.; Hassibi, B.; Hassibi, A.
Year: 2007
DOI: 10.1109/ICASSP.2007.366705
Estimation of DNA copy number in a given biological sample is an extremely important problem in genomics. This problem is especially challenging when the number of the DNA strands is minuscule, which is often the case in applications such as pathogen and genetic mutation detection. A recently developed technique, realtime polymerase chain reaction (PCR), amplifies the number of initial target molecules by replicating them through a series of thermal cycles. Ideally, the number of target molecules doubles at the end of each cycle. However, in practice, due to biochemical noise the efficiency of the PCR reaction, defined as the fraction of target molecules which are successfully copied during a cycle, is always less than 1. In this paper, we formulate the problem of joint maximumlikelihood estimation of the PCR efficiency and the initial DNA copy number. As indicated by simulation studies, the performance of the proposed estimator is superior with respect to competing statistical approaches. Moreover, we compute the CramerRao lower bound on the meansquare estimation error.https://authors.library.caltech.edu/records/prz8r0bn25A coding scheme for wireless networks with multiple antenna nodes and no channel information
https://resolver.caltech.edu/CaltechAUTHORS:OGGicassp07
Authors: Oggier, Frédérique; Hassibi, Babak
Year: 2007
DOI: 10.1109/ICASSP.2007.366560
In this paper, we present a coding strategy for wireless relay networks where the relay nodes are small devices with few resources, while the source and sink are equipped with multiple antennas to increase the transmission rate. We assume no channel knowledge at all, and the receiver decodes knowing none of the channel paths. This coding scheme uses distributed spacetime coding techniques and is inspired by noncoherent differential spacetime coding. It is shown to yield a diversity linear in the minimum number of transmit/receive antennas times the number of relays.https://authors.library.caltech.edu/records/hvawd28838Estimation over Communication Networks: Performance Bounds and Achievability Results
https://resolver.caltech.edu/CaltechAUTHORS:DANacc07
Authors: Dana, A. F.; Gupta, V.; Hespanha, J. P.; Hassibi, B.; Murray, R. M.
Year: 2007
DOI: 10.1109/ACC.2007.4282933
This paper considers the problem of estimation over communication networks. Suppose a sensor is taking measurements of a dynamic process. However the process needs to be estimated at a remote location connected to the sensor through a network of communication links that drop packets stochastically. We provide a framework for computing the optimal performance in the sense of expected error covariance. Using this framework we characterize the dependency of the performance on the topology of the network and the packet dropping process. For independent and memoryless packet dropping processes we find the steadystate error for some classes of networks and obtain lower and upper bounds for the performance of a general network. Finally we find a necessary and sufficient condition for the stability of the estimate error covariance for general networks with spatially correlated and Markov type dropping process. This interesting condition has a maxcut interpretation.https://authors.library.caltech.edu/records/a7h63n0272On the Complexity of Exact MaximumLikelihood Decoding for Asymptotically Good Low Density Parity Check Codes: A New Perspective
https://resolver.caltech.edu/CaltechAUTHORS:XUWitw07b
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2007
DOI: 10.1109/ITW.2007.4313065
The problem of exact maximumlikelihood (ML) decoding of general linear codes is wellknown to be NPhard. In this paper, we show that exact ML decoding of a class of asymptotically good low density parity check codes — expander codes — over binary symmetric channels (BSCs) is possible with an averagecase polynomial complexity. This offers a new way of looking at the complexity issue of exact ML decoding for communication systems where the randomness in channel plays a fundamental central role. More precisely, for any bitflipping probability p in a nontrivial range, there exists a rate region of nonzero support and a family of asymptotically good codes which achieve error probability exponentially decaying in coding length n while admitting exact ML decoding in averagecase polynomial time. As p approaches zero, this rate region approaches the Shannon channel capacity region. Similar results can be extended to AWGN channels, suggesting it may be feasible to eliminate the error floor phenomenon associated with beliefpropagation decoding of LDPC codes in the high SNR regime. The derivations are based on a hierarchy of ML certificate decoding algorithms adaptive to the channel realization. In this process, we propose an efficient O(n^2) new ML certificate algorithm based on the maxflow algorithm. Moreover, exact ML decoding of the considered class of codes constructed from LDPC codes with regular left degree, of which the considered expander codes are a special case, remains NPhard; thus giving an interesting contrast between the worstcase and averagecase complexities.https://authors.library.caltech.edu/records/8gwt4d7281Normalized Entropy Vectors, Network Information Theory and Convex Optimization
https://resolver.caltech.edu/CaltechAUTHORS:HASitwitwn07
Authors: Hassibi, Babak; Shadbakht, Sormeh
Year: 2007
DOI: 10.1109/ITWITWN.2007.4318051
We introduce the notion of normalized entropic vectors  slightly different from the standard definition in the literature in that we normalize entropy by the logarithm of the alphabet size. We argue that this definition is more natural for determining the capacity region of networks and, in particular, that it smooths out the irregularities of the space of nonnormalized entropy vectors and renders the closure of the resulting space convex (and compact). Furthermore, the closure of the space remains convex even under constraints imposed by memoryless channels internal to the network. It therefore follows that, for a large class of acyclic memoryless networks, the capacity region for an arbitrary set of sources and destinations can be found by maximization of a linear function over the convex set of channelconstrained normalized entropic vectors and some linear constraints. While this may not necessarily make the problem simpler, it certainly circumvents the "infiniteletter characterization" issue, as well as the nonconvexity of earlier formulations, and exposes the core of the problem. We show that the approach allows one to obtain the classical cutset bounds via a duality argument. Furthermore, the approach readily shows that, for acyclic memoryless wired networks, one need only consider the space of unconstrained normalized entropic vectors, thus separating channel and network coding  a result very recently recognized in the literature.https://authors.library.caltech.edu/records/14sf8agk35Efficient Compressive Sensing with Deterministic Guarantees Using Expander Graphs
https://resolver.caltech.edu/CaltechAUTHORS:XUWitw07a
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2007
DOI: 10.1109/ITW.2007.4313110
Compressive sensing is an emerging technology which can recover a sparse signal vector of dimension n via a much smaller number of measurements than n. However, the existing compressive sensing methods may still suffer from relatively high recovery complexity, such as O(n^3), or can only work efficiently when the signal is super sparse, sometimes without deterministic performance guarantees. In this paper, we propose a compressive sensing scheme with deterministic performance guarantees using expandergraphsbased measurement matrices and show that the signal recovery can be achieved with complexity O(n) even if the number of nonzero elements k grows linearly with n. We also investigate compressive sensing for approximately sparse signals using this new method. Moreover, explicit constructions of the considered expander graphs exist. Simulation results are given to show the performance and complexity of the new method.https://authors.library.caltech.edu/records/kqnwhm1c78DiversityMultiplexing Gain Tradeoff of a MIMO System with Relays
https://resolver.caltech.edu/CaltechAUTHORS:RAOitwitwn07
Authors: Rao, Chaitanya; Hassibi, Babak
Year: 2007
DOI: 10.1109/ITWITWN.2007.4318029
We find the diversitymultiplexing gain tradeoff of a multipleantenna (MIMO) system with M transmit antennas, N receive antennas, R relay nodes, and with independent Rayleigh fading, in which the relays apply a distributed spacetime code. In this twostage scheme the tradeoff is shown to coincide with that of a MIMO system with R transmit and min{M, N} receive antennas.https://authors.library.caltech.edu/records/5ydnmfqn65Modeling the kinetics of hybridization in microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20101104111636482
Authors: Vikalo, H.; Hassibi, B.; Stojnic, M.; Hassibi, A.
Year: 2007
DOI: 10.1109/GENSIPS.2007.4365828
Conventional fluorescentbased microarrays acquire data
after the hybridization phase. In this phase the targets analytes
(i.e., DNA fragments) bind to the capturing probes
on the array and supposedly reach a steady state. Accordingly,
microarray experiments essentially provide only a
single, steadystate data point of the hybridization process.
On the other hand, a novel technique (i.e., realtime
microarrays) capable of recording the kinetics of hybridization
in fluorescentbased microarrays has recently
been proposed in [5]. The richness of the information obtained
therein promises higher signaltonoise ratio, smaller
estimation error, and broader assay detection dynamic range
compared to the conventional microarrays. In the current
paper, we develop a probabilistic model of the kinetics of
hybridization and describe a procedure for the estimation
of its parameters which include the binding rate and target
concentration. This probabilistic model is an important
step towards developing optimal detection algorithms for
the microarrays which measure the kinetics of hybridization,
and to understanding their fundamental limitations.https://authors.library.caltech.edu/records/d6zs905p94Signal Processing Aspects of RealTime DNA Microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20150225070303490
Authors: Vikalo, H.; Hassibi, B.; Hassibi, A.
Year: 2007
DOI: 10.1109/CAMSAP.2007.4497991
Data acquisition in conventional fluorescentbased microarrays takes place after the completion of a hybridization phase. During the hybridization phase, target analytes bind to their corresponding capturing probes on the array. The conventional microarrays attempt to detect presence and quantify amounts of the targets by collecting a single data point, supposedly taken after the hybridization process has reached its steadystate. Recently, socalled realtime microarrays capable of acquiring not only the steadystate data but the entire kinetics of hybridization have been proposed in [1]. The richness of the information obtained by the realtime microarrays promises higher signaltonoise ratio, smaller estimation error, and broader assay detection dynamic range compared to the conventional microarrays. In the current paper, we study the signal processing aspects of the realtime microarray data acquisition.https://authors.library.caltech.edu/records/bszsd7ed78A new exact closest lattice point search algorithm using linear constraints
https://resolver.caltech.edu/CaltechAUTHORS:20100507094336274
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2007
DOI: 10.1109/SPAWC.2007.4401291
The problem of finding the closest lattice point arises in several communications scenarios and is known to be NPhard. We propose a new closest lattice point search algorithm which utilizes a set of new linear inequality constraints to reduce the search of the closest lattice point to the intersection of a polyhedron and a sphere. This set of linear constraints efficiently leverage the geometric structure of the lattice to reduce considerably the number of points that must be visited. Simulation results verify that this algorithm offers substantial computational savings over standard sphere decoding when the dimension of the problem is large.https://authors.library.caltech.edu/records/ytataxfm14Sparse measurements, compressed sampling, and DNA microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20150225070500630
Authors: Vikalo, H.; Parvaresh, Fa.; Misra, S.; Hassibi, B.
Year: 2008
DOI: 10.1109/ICASSP.2008.4517676
DNA microarrays comprising tens of thousands of probe spots are currently being employed to test multitude of targets in a single experiment. Typically, each microarray spot contains a large number of copies of a single probe designed to capture a single target, and hence collects only a single data point. This is a wasteful use of the sensing resources in comparative DNA microarray experiments, where a test sample is measured relative to a reference sample. Since only a small fraction of the total number of genes represented by the two samples is differentially expressed, a vast number of probe spots will not provide any useful information. To this end we consider an alternative design, the socalled compressed microarrays, wherein each spot is a composite of several different probes and the total number of spots is potentially much smaller than the number of targets being tested. Fewer spots directly translates to significantly lower costs due to cheaper array manufacturing, simpler image acquisition and processing, and smaller amount of genomic material needed for experiments. To recover signals from compressed microarray measurements, we leverage ideas from compressive sampling. Moreover, we propose an algorithm which has far less computational complexity than the widelyused linearprogrammingbased methods, and can also recover signals with less sparsity.https://authors.library.caltech.edu/records/f30pf6gv61On exact maximumlikelihood detection for noncoherent MIMO wireless systems: A branchestimatebound optimization framework
https://resolver.caltech.edu/CaltechAUTHORS:20150224074838545
Authors: Xu, Weiyu; Stojnic, Mihailo; Hassibi, Babak
Year: 2008
DOI: 10.1109/ISIT.2008.4595343
Fast fading wireless environments pose a great challenge for achieving high spectral efficiency in next generation wireless systems. Joint maximumlikelihood (ML) channel estimation and signal detection is of great theoretical and practical interest, especially for multipleinput multipleoutput(MIMO) systems where the multiple channel coefficients need to be estimated. However, this is a hard combinatorial optimization problem, for which obtaining efficient exact algorithms has been elusive for the general MIMO systems. In this paper, we propose an efficient branchestimatebound noncoherent optimization framework which provably achieves the exact ML joint channel estimation and data detection for general MIMO systems. Numerical results indicate that the exact joint ML method can achieve substantial performance improvements over suboptimal methods including iterative channel estimation and signal detection. We also derive analytical bounds on the computational complexity of the new exact joint ML method and show that its average complexity approaches a constant times the length of the coherence time, as the SNR approaches infinity.https://authors.library.caltech.edu/records/wzrs0g1e58On estimation in realtime microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20150225070827146
Authors: Vikalo, H.; Hassibi, B.; Hassibi, A.
Year: 2008
DOI: 10.1109/ICASSP.2008.4517675
Conventional fluorescentbased microarrays acquire data after the hybridization phase. During this phase, the target analytes bind to the capturing probes on the array and, by the end of it, supposedly reach a steady state. Therefore, conventional microarrays attempt to detect and quantify the targets with a single data point taken in the steadystate. On the other hand, a novel technique, the socalled realtime microarray, capable of recording the kinetics of hybridization in fluorescentbased microarrays has recently been proposed in (Hassibi, 2007). The richness of the information obtained therein promises higher signaltonoise ratio, smaller estimation error, and broader assay detection dynamic range compared to conventional microarrays. In the current paper, we develop a probabilistic model for realtime microarrays and describe a procedure for the estimation of target amounts therein. Moreover, leveraging on system identification ideas, we propose a novel technique for the elimination of crosshybridization.https://authors.library.caltech.edu/records/fzw92fht94Compressed sensing of approximately sparse signals
https://resolver.caltech.edu/CaltechAUTHORS:20150224075138435
Authors: Stojnic, Mihailo; Xu, Weiyu; Hassibi, Babak
Year: 2008
DOI: 10.1109/ISIT.2008.4595377
It is well known that compressed sensing problems reduce to solving large underdetermined systems of equations. If we choose the compressed measurement matrix according to some appropriate distribution and the signal is sparse enough the l1 optimization can exactly recover the ideally sparse signal with overwhelming probability by Candes, E. and Tao, T., [2], [1]. In the current paper, we will consider the case of the socalled approximately sparse signals. These signals are a generalized version of the ideally sparse signals. Letting the zero valued components of the ideally sparse signals to take the values of certain small magnitude one can construct the approximately sparse signals. Using a different but simple proof technique we show that the claims similar to those of [2] and [1] related to the proportionality of the number of large components of the signals to the number of measurements, hold for approximately sparse signals as well. Furthermore, using the same technique we compute the explicit values of what this proportionality can be if the compressed measurement matrix A has a rotationally invariant distribution of the nullspace. We also give the quantitative tradeoff between the signal sparsity and the recovery robustness of the l_1 minimization. As it will turn out in an asymptotic case of the number of measurements the threshold result of [1] corresponds to a special case of our result.https://authors.library.caltech.edu/records/n8wcj7w690Lowcomplexity blind maximumlikelihood detection for SIMO systems with general constellations
https://resolver.caltech.edu/CaltechAUTHORS:20100729100229512
Authors: Xu, Weiyu; Stojnic, Mihailo; Hassibi, Babak
Year: 2008
DOI: 10.1109/ICASSP.2008.4518235
The demand for high data rate reliable communications poses great challenges to the next generation wireless systems in highly dynamic mobile environments. In this paper, we investigate the joint maximumlikelihood (ML) channel estimation and signal detection problem for singleinput multipleoutput (SIMO) wireless systems with general modulation constellations and propose an efficient sequential decoder for finding the exact joint ML solution. Unlike other known methods, the new decoder can even efficiently find the joint ML solution under high spectral efficiency nonconstant modulus modulation constellations. In particular, the new algorithm does not need such preprocessing steps as Cholesky or QR decomposition in the traditional sphere decoders for joint ML channel estimation and data detection. The elimination of such preprocessing not only reduces the number of floating point computations, but also will potentially lead to smaller size and power consumption in VLSI implementations while providing better numerical stability.https://authors.library.caltech.edu/records/v1nmfg1d97Explicit measurements with almost optimal thresholds for compressed sensing
https://resolver.caltech.edu/CaltechAUTHORS:20100721154038572
Authors: Parvaresh, Farzad; Hassibi, Babak
Year: 2008
DOI: 10.1109/ICASSP.2008.4518494
We consider the deterministic construction of a measurement
matrix and a recovery method for signals that are block
sparse. A signal that has dimension N = nd, which consists
of n blocks of size d, is called (s, d)block sparse if
only s blocks out of n are nonzero. We construct an explicit
linear mapping Φ that maps the (s, d)block sparse signal
to a measurement vector of dimension M, where s•dhttps://authors.library.caltech.edu/records/9xjqgq6m25Compressed sensing  probabilistic analysis of a nullspace characterization
https://resolver.caltech.edu/CaltechAUTHORS:20100707153301650
Authors: Stojnic, Mihailo; Xu, Weiyu; Hassibi, Babak
Year: 2008
DOI: 10.1109/ICASSP.2008.4518375
It is well known that compressed sensing problems reduce to solving large underdetermined systems of equations. To assure that the problem is well defined, i.e., that the solution is unique the vector of unknowns is of course assumed to be sparse. Nonetheless, even when the solution is unique, finding it in general may be computationally difficult. However, starting with the seminal work of Candes and Tao [2005], it has been shown that linear programming techniques, obtained from an l_1norm relaxation of the original nonconvex problem, can provably find the unknown vector in certain instances. In particular, using a certain restricted isometry property, Candes and Tao [2005] shows that for measurement matrices chosen from a random Gaussian ensemble, l_1 optimization can find the correct solution with overwhelming probability even when the number of nonzero entries of the unknown vector is proportional to the number of measurements (and the total number of unknowns). The subsequent paper [Donoho and Tanner, 2005] uses results on neighborly polytopes from [Vershik and Sporyshev, 1992] to give a "sharp" bound on what this proportionality should be in the Gaussian case. In the current paper, we observe that what matters is not so much the distribution from which the entries of the measurement matrix A are drawn, but rather the statistics of the nullspace of A. Using this observation, we provide an alternative proof of the main result of Candes and Tao [2005] by analyzing matrices whose nullspace is isotropic (of which i.i.d. Gaussian ensembles are a special case).https://authors.library.caltech.edu/records/bdgv1esf42The MIMO wiretap channel
https://resolver.caltech.edu/CaltechAUTHORS:OGGisccsp08
Authors: Oggier, Frédérique; Hassibi, Babak
Year: 2008
DOI: 10.1109/ISCCSP.2008.4537222
We study the MIMO wiretap channel, a MIMO broadcast channel where the transmitter sends some confidential information to one user which is a legitimate receiver, while the other user is an eavesdropper. Perfect secrecy is achieved when the transmitter and the legitimate receiver can communicate at some positive rate, while ensuring that the eavesdropper gets zero bits of information. In this paper, we compute the perfect secrecy capacity of the multiple antenna MIMO broadcast channel, where the number of antennas is arbitrary for both the transmitter and the two receivers. Our technique involves a careful study of a Satolike upper bound via the solution of a certain algebraic Riccati equation.https://authors.library.caltech.edu/records/d1g6pxen58Wireless erasure networks with feedback
https://resolver.caltech.edu/CaltechAUTHORS:20170403172107989
Authors: Smith, Brian; Hassibi, Babak
Year: 2008
DOI: 10.1109/ISIT.2008.4595004
Consider a lossy packet network of queues, communicating over a wireless medium. This paper presents a throughputoptimal transmission strategy for a unicast network when feedback is available, which has the following advantages: It requires a very limited form of acknowledgment feedback. It is completely distributed, and independent of the network topology. Finally, communication at the information theoretic cutset rate requires no network coding and no rateless coding on the packets. This simple strategy consists of each node randomly choosing a packet from its buffer to transmit at each opportunity. However, the packet is only deleted from a node's buffer once it has been successfully received by the final destination.https://authors.library.caltech.edu/records/aqcr0s8k94The secrecy capacity of the MIMO wiretap channel
https://resolver.caltech.edu/CaltechAUTHORS:OGGisit08
Authors: Oggier, Frédérique; Hassibi, Babak
Year: 2008
DOI: 10.1109/ISIT.2008.4595041
We consider the MIMO wiretap channel, that is a MIMO broadcast channel where the transmitter sends some confidential information to one user which is a legitimate receiver, while the other user is an eavesdropper. Perfect secrecy is achieved when the transmitter and the legitimate receiver can communicate at some positive rate, while insuring that the eavesdropper gets zero bits of information. In this paper, we compute the perfect secrecy capacity of the multiple antenna MIMO broadcast channel, where the number of antennas is arbitrary for both the transmitter and the two receivers. Our technique involves a careful study of a Satolike upper bound via the solution of a certain algebraic Riccati equation.https://authors.library.caltech.edu/records/dxhbf6fr57The entropy region for three Gaussian random variables
https://resolver.caltech.edu/CaltechAUTHORS:20150127072401829
Authors: Hassibi, Babak; Shadbakht, Sormeh
Year: 2008
DOI: 10.1109/ISIT.2008.4595469
Given n (discrete or continuous) random variables X_i, the (2^n – 1)dimensional vector obtained by evaluating the joint entropy of all nonempty subsets of {X_(1,hellip), X_n} is called an entropic vector. Determining the region of entropic vectors is an important open problem in information theory. Recently, Chan has shown that the entropy regions for discrete and continuous random variables, though different, can be determined from one another. An important class of continuous random variables are those that are vectorvalued and jointly Gaussian. It is known that Gaussian random variables violate the Ingleton bound, which many random variables such as those obtained from linear codes over finite fields do satisfy, and they also achieve certain nonShannon inequalities. In this paper we give a full characterization of the entropy region for three jointlyGaussian vectorvalued random variables and, rather surprisingly, show that the region is strictly smaller than the entropy region for three arbitrary random variables. However, we also show the following result. For any given entropic vector h isin R^7, there exists a thetas* > 0, such that for all thetas ges thetas*, the vector 1/thetas h can be generated by three vectorvalued jointly Gaussian random variables. This implies that for three random variables the region of entropic vectors can be obtained by considering the cone generated by the space of Gaussian entropic vectors. It also suggests that studying Gaussian random variables for n ges 4 may be a fruitful approach to studying the space of entropic vectors for arbitrary n.https://authors.library.caltech.edu/records/vj52wa1e22On exact maximumlikelihood detection for noncoherent MIMO wireless systems: A branchestimatebound optimization framework
https://resolver.caltech.edu/CaltechAUTHORS:20190304085002366
Authors: Xu, Weiyu; Stojnic, Mihailo; Hassibi, Babak
Year: 2008
DOI: 10.1109/ISIT.2008.4595343
Fast fading wireless environments pose a great challenge for achieving high spectral efficiency in next generation wireless systems. Joint maximumlikelihood (ML) channel estimation and signal detection is of great theoretical and practical interest, especially for multipleinput multipleoutput(MIMO) systems where the multiple channel coefficients need to be estimated. However, this is a hard combinatorial optimization problem, for which obtaining efficient exact algorithms has been elusive for the general MIMO systems. In this paper, we propose an efficient branchestimatebound noncoherent optimization framework which provably achieves the exact ML joint channel estimation and data detection for general MIMO systems. Numerical results indicate that the exact joint ML method can achieve substantial performance improvements over suboptimal methods including iterative channel estimation and signal detection. We also derive analytical bounds on the computational complexity of the new exact joint ML method and show that its average complexity approaches a constant times the length of the coherence time, as the SNR approaches infinity.https://authors.library.caltech.edu/records/5pr3mvvn83Multicast in wireless erasure networks with feedback
https://resolver.caltech.edu/CaltechAUTHORS:20190301141453792
Authors: Jiang, Chong; Smith, Brian; Hassibi, Babak; Vishwanath, Sriram
Year: 2008
DOI: 10.1109/ISCC.2008.4625774
This paper studies the lossy, wireless packet network of [1], in the case of a multicast requirement and the availability of feedback. In the unicast case, feedback is sufficient to allow a strategy which achieves the throughputoptimal cutset capacity without requiring network coding [3]. We provide a counterexample to show that source coding and feedback, without network coding, is insufficient to achieve the cutset capacity for the multicast wireless erasure network. In particular, we examine a network with one source, one relay, and two destinations. We show that even with the highly optimistic assumption of feedback which provides global packet state awareness, this network still fails to reach capacity. This bridges the gap between two previously known results; one, that network coding can achieve the capacity of the wireless erasure network, and two, that feedback allows a capacity achieving scheme which does not require network coding in the unicast wireless erasure network.https://authors.library.caltech.edu/records/2cenkdgs65Necessary and Sufficient Conditions for Success of the Nuclear Norm Heuristic for Rank Minimization
https://resolver.caltech.edu/CaltechAUTHORS:20150130074855649
Authors: Recht, Benjamin; Xu, Weiyu; Hassibi, Babak
Year: 2008
DOI: 10.1109/CDC.2008.4739332
Minimizing the rank of a matrix subject to constraints is a challenging is a challenging problem that arises
in many control applications including controller design, realization theory and model reduction. This class of optimization problems, known as rank minimization, is NPHARD, and for most practical problems there are no efficient algorithms that yield exact solutions. A popular heuristic algorithm replaces the rank function with the nuclear norm—equal to the sum of the singular values—of the decision variable. In this paper, we provide a necessary and sufficient condition that quantifies when this heuristic successfully finds the minimum rank
solution of a linear constraint set. We further show that most of the problems of interest in control can be formulated as rank minimization subject to such linear constraints. We additionally provide a probability distribution over instances of the affine rank minimization problem such that instances sampled from this distribution satisfy our conditions for success with
overwhelming probability provided the number of constraints
is appropriately large. Finally, we give empirical evidence that these probabilistic bounds provide accurate predictions of the heuristic's performance in nonasymptotic scenarios.https://authors.library.caltech.edu/records/ec7hj4et47A Stieltjes transform approach for studying the steadystate behavior of random Lyapunov and Riccati recursions
https://resolver.caltech.edu/CaltechAUTHORS:20150204075604396
Authors: Vakili, Ali; Hassibi, Babak
Year: 2008
DOI: 10.1109/CDC.2008.4739258
In this paper we study the asymptotic eigenvalue distribution of certain random Lyapunov and Riccati recursions that arise in signal processing and control. The analysis of such recursions has remained elusive when the system and/or covariance matrices are random. Here we use transform techniques (such as the Stieltjes transform and free probability) that have gained popularity in the study of large random matrices. While we have not yet developed a full theory, we do obtain explicit formula for the asymptotic eigendistribution of certain classes of Lyapunov and Riccati recursions, which well match simulation results. Generalizing the results to arbitrary classes of such recursions is currently under investigation.https://authors.library.caltech.edu/records/qpvwy2d519Particle filtering for Quantized Innovations
https://resolver.caltech.edu/CaltechAUTHORS:20100511134410043
Authors: Sukhavasi, Ravi Teja; Hassibi, Babak
Year: 2009
DOI: 10.1109/ICASSP.2009.4960062
In this paper, we reexamine the recently proposed distributed state estimators based on quantized innovations. It is widely believed that the error covariance of the Quantized Innovation Kalman filter follows a modified Riccati recursion. We present stable linear dynamical systems for which this is violated and the filter diverges. We propose a Particle Filter that approximates the optimal nonlinear filter and observe that the error covariance of the Particle Filter follows the modified Riccati recursion. We also simulate a Posterior CramerRao bound (PCRB) for this filtering problem.https://authors.library.caltech.edu/records/17jx0bxe42On the recovery of nonnegative sparse vectors from sparse measurements inspired by expanders
https://resolver.caltech.edu/CaltechAUTHORS:20100510094616369
Authors: Khajehnejad, M. Amin; Hassibi, Babak
Year: 2009
DOI: 10.1109/ICASSP.2009.4960227
This paper studies compressed sensing for the recovery of nonnegative sparse vectors from a smaller number of measurements than the ambient dimension of the unknown vector. We focus on measurement matrices that are sparse, i.e., have only a constant number of nonzero (and nonnegative) entries in each column. For such measurement matrices we give a simple necessary and sufficient condition for l1 optimization to successfully recover the unknown vector. Using a simple ldquoperturbationrdquo to the adjacency matrix of an unbalanced expander, we obtain simple closed form expressions for the threshold relating the ambient dimension n, number of measurements m and sparsity level k, for which l1 optimization is successful with overwhelming probability. Simulation results suggest that the theoretical thresholds are fairly tight and demonstrate that the ldquoperturbationsrdquo significantly improve the performance over a direct use of the adjacency matrix of an expander graph.https://authors.library.caltech.edu/records/jrd4exkk65On the eigendistribution of the steadystate error covariance matrix for the extended RLS algorithm
https://resolver.caltech.edu/CaltechAUTHORS:20100511151933460
Authors: Vakili, Ali; Familier, Eythan; Hassibi, Babak
Year: 2009
DOI: 10.1109/ICASSP.2009.4960212
In an earlier work, we used transform methods from the theory of random matrices to analytically compute the asymptotic eigendistribution of the error covariance matrix of the singlemeasurement RLS filter. When we have a multiplicity of measurements, as happens in extended RLS filtering, the analysis is much more complicated. In this paper we study the multiple measurement case and obtain a system of two coupled equations for the Stieltjes transform of the asymptotic eigendistribution. Numerical solutions of this system very well predict the actual asymptotic eigendistribution for systems with as low as n = 10  20 state dimensions.https://authors.library.caltech.edu/records/cqfbxhkt63On the distribution of indefinite quadratic forms in Gaussian random variables
https://resolver.caltech.edu/CaltechAUTHORS:20150204073234311
Authors: AlNaffouri, Tareq Y.; Hassibi, Babak
Year: 2009
DOI: 10.1109/ISIT.2009.5205261
In this work, we propose a transparent approach to evaluating the CDF of indefinite quadratic forms in Gaussian random variables and ratios of such forms. This quantity appears in the analysis of different receivers in communication systems and in various applications in signal processing. Instead of attempting to find the pdf of this quantity as is the case in many papers in literature, we focus on finding the CDF. The basic trick that we implement is to replace inequalities that appear in the CDF calculations with the unit step function and replace the latter with its Fourier transform. This produces a multidimensional integral that can be evaluated using complex integration. We show how our approach extends to nonzero mean Gaussian real/complex vectors and to the joint distribution of indefinite quadratic forms.https://authors.library.caltech.edu/records/m3cnht5067Nonnegative Compressed Sensing with Minimal Perturbed Expanders
https://resolver.caltech.edu/CaltechAUTHORS:20100507152555873
Authors: Khajehnejad, M. Amin; Dimakis, Alexandros G.; Hassibi, Babak
Year: 2009
DOI: 10.1109/DSP.2009.4786012
This paper studies compressed sensing for the recovery of nonnegative sparse vectors from a smaller number of measurements than the ambient dimension of the unknown vector. We construct sparse measurement matrices for the recovery of nonnegative vectors, using perturbations of adjacency matrices of expander graphs with much smaller expansion coefficients than previously suggested schemes. These constructions are crucial in applications, such as DNA microarrays and sensor networks, where dense measurements are not practically feasible. We present a necessary and sufficient condition for ℓ_1 optimization to successfully recover the unknown vector and obtain closed form expressions for the recovery threshold. We finally present a novel recovery algorithm that exploits expansion and is faster than ℓ_1 optimization.https://authors.library.caltech.edu/records/46h3wqgb72Limits of performance of realtime DNA microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20150224074532053
Authors: Vikalo, Haris; Hassibi, Babak
Year: 2009
DOI: 10.1109/ALLERTON.2009.5394924
DNA microarrays rely on chemical attraction between the nucleic acid sequences of interest (mRNA and DNA sequences, referred to as targets) and their molecular complements which serve as biological sensing elements (probes). The attraction between the complementary sequences leads to binding, in which probes capture target molecules. Molecular binding is a stochastic process and hence the number of captured analytes at any time is a random variable. Today, majority of DNA microarrays acquire only a single measurement of the binding process, essentially taking one sample from the steadystate distribution of the binding process. Realtime DNA microarrays provide much more: they can take multiple temporal measurements which not only allow more precise characterization of the steadystate but also enable faster detection based on the early kinetics of the binding process. In this paper, we derive the CramerRao lower bound on the meansquare error of estimating the target amounts in realtime DNA microarrays, and compare it to that of conventional microarrays. The results suggest that a few temporal samples collected in the early phase of the binding process are often sufficient to enable significant performance improvement of the realtime microarrays over the conventional ones.https://authors.library.caltech.edu/records/468g0vt674Breaking the ℓ_1 recovery thresholds with reweighted ℓ_1 optimization
https://resolver.caltech.edu/CaltechAUTHORS:20150224074243785
Authors: Xu, Weiyu; Khajehnejad, M. Amin; Avestimehr, A. Salman; Hassibi, Babak
Year: 2009
DOI: 10.1109/ALLERTON.2009.5394882
It is now well understood that ℓ_1 minimization algorithm is able to recover sparse signals from incomplete measurements and sharp recoverable sparsity thresholds have also been obtained for the l1 minimization algorithm. In this paper, we investigate a new iterative reweighted ℓ_1 minimization algorithm and showed that the new algorithm can increase the sparsity recovery threshold of ℓ_1 minimization when decoding signals from relevant distributions. Interestingly, we observed that the recovery threshold performance of the new algorithm depends on the behavior, more specifically the derivatives, of the signal amplitude probability distribution at the origin.https://authors.library.caltech.edu/records/xz37n5pw80Approximate capacity region of the twopair bidirectional Gaussian relay network
https://resolver.caltech.edu/CaltechAUTHORS:20150204073559541
Authors: Sezgin, Aydin; Khajehnejad, M. Amin; Avestimehr, A. Salman; Hassibi, Babak
Year: 2009
DOI: 10.1109/ISIT.2009.5205584
We study the capacity of the Gaussian twopair fullduplex directional (or twoway) relay network with a singlerelay supporting the communication of the pairs. This network is a generalization of the well known bidirectional relay channel, where we have only one pair of users. We propose a novel transmission technique which is based on a specific superposition of lattice codes and random Gaussian codes at the source nodes. The relay attempts to decode the Gaussian codewords and the superposition of the lattice codewords of each pair. Then it forwards this information to all users. We analyze the achievable rate of this scheme and show that for all channel gains it achieves to within 2 bits/sec/Hz per user of the cutset upper bound on the capacity region of the twopair bidirectional relay network.https://authors.library.caltech.edu/records/1hshktag15Compressed Sensing over the Grassmann Manifold: A Unified Analytical Framework
https://resolver.caltech.edu/CaltechAUTHORS:20100729100253548
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2009
DOI: 10.1109/ALLERTON.2008.4797608
It is well known that compressed sensing problems reduce to finding the sparse solutions for large underdetermined systems of equations. Although finding the sparse solutions in general may be computationally difficult, starting with the seminal work of [2], it has been shown that linear programming techniques, obtained from an l_(1)norm relaxation of the original nonconvex problem, can provably find the unknown vector in certain instances. In particular, using a certain restricted isometry property, [2] shows that for measurement matrices chosen from a random Gaussian ensemble, l_1 optimization can find the correct solution with overwhelming probability even when the support size of the unknown vector is proportional to its dimension. The paper [1] uses results on neighborly polytopes from [6] to give a ldquosharprdquo bound on what this proportionality should be in the Gaussian measurement ensemble. In this paper we shall focus on finding sharp bounds on the recovery of ldquoapproximately sparserdquo signals (also possibly under noisy measurements). While the restricted isometry property can be used to study the recovery of approximately sparse signals (and also in the presence of noisy measurements), the obtained bounds can be quite loose. On the other hand, the neighborly polytopes technique which yields sharp bounds for ideally sparse signals cannot be generalized to approximately sparse signals. In this paper, starting from a necessary and sufficient condition for achieving a certain signal recovery accuracy, using highdimensional geometry, we give a unified nullspace Grassmannian anglebased analytical framework for compressive sensing. This new framework gives sharp quantitative tradeoffs between the signal sparsity and the recovery accuracy of the l_1 optimization for approximately sparse signals. As it will turn out, the neighborly polytopes result of [1] for ideally sparse signals can be viewed as a special case of ours. Our result concerns fundamental properties of linear subspaces and so may be of independent mathematical interest.https://authors.library.caltech.edu/records/jh0hmh6d03Cayley's hyperdeterminant, the principal minors of a symmetric matrix and the entropy region of 4 Gaussian random variables
https://resolver.caltech.edu/CaltechAUTHORS:20100722101032699
Authors: Shadbakht, Sormeh; Hassibi, Babak
Year: 2009
DOI: 10.1109/ALLERTON.2008.4797553
It has recently been shown that there is a connection between Cayley's hypdeterminant and the principal minors of a symmetric matrix. With an eye towards characterizing the entropy region of jointly Gaussian random variables, we obtain three new results on the relationship between Gaussian random variables and the hyperdeterminant. The first is a new (determinant) formula for the 2×2×2 hyperdeterminant. The second is a new (transparent) proof of the fact that the principal minors of an ntimesn symmetric matrix satisfy the 2 × 2 × .... × 2 (n times) hyperdeterminant relations. The third is a minimal set of 5 equations that 15 real numbers must satisfy to be the principal minors of a 4×4 symmetric matrix.https://authors.library.caltech.edu/records/sp19vya241A Stieltjes transform approach for analyzing the RLS adaptive Filter
https://resolver.caltech.edu/CaltechAUTHORS:20100723095254413
Authors: Vakili, Ali; Hassibi, Babak
Year: 2009
DOI: 10.1109/ALLERTON.2008.4797590
Although the RLS filter is wellknown and various algorithms have been developed for its implementation, analyzing its performance when the regressors are random, as is often the case, has proven to be a formidable task. The reason is that the Riccati recursion, which propagates the error covariance matrix, becomes a random recursion. The existing results are approximations based on assumptions that are often not very realistic. In this paper we use ideas from the theory of large random matrices to find the asymptotic (in time) eigendistribution of the error covariance matrix of the RLS filter. Under the assumption of a large dimensional state vector (in most cases n = 1020 is large enough to get quite accurate predictions) we find the asymptotic eigendistribution of the error covariance for temporally white regressors, shift structured regressors, and for the RLS filter with intermittent observations.https://authors.library.caltech.edu/records/jkprra6r61Capacity region of the deterministic multipair bidirectional relay network
https://resolver.caltech.edu/CaltechAUTHORS:20100510105820163
Authors: Avestimehr, A. Salman; Khajehnejad, M. Amin; Sezgin, Aydin; Hassibi, Babak
Year: 2009
DOI: 10.1109/ITWNIT.2009.5158541
In this paper we study the capacity region of the multipair bidirectional (or twoway) wireless relay network, in which a relay node facilitates the communication between multiple pairs of users. This network is a generalization of the well known bidirectional relay channel, where we have only one pair of users. We examine this problem in the context of the deterministic channel interaction model, which eliminates the channel noise and allows us to focus on the interaction between signals. We characterize the capacity region of this network when the relay is operating at either fullduplex mode or halfduplex mode (with non adaptive listentransmit scheduling). In both cases we show that the cutset upper bound is tight and, quite interestingly, the capacity region is achieved by a simple equationforwarding strategy.https://authors.library.caltech.edu/records/j34ddabp85On the Entropy Region of Discrete and Continuous Random Variables and Network Information Theory
https://resolver.caltech.edu/CaltechAUTHORS:20100726103900559
Authors: Shadbakht, Sormeh; Hassibi, Babak
Year: 2009
DOI: 10.1109/ACSSC.2008.5074810
We show that a large class of network information theory problems can be cast as convex optimization over the convex space of entropy vectors. A vector in 2^(n)  1 dimensional space is called entropic if each of its entries can be regarded as the joint entropy of a particular subset of n random variables (note that any set of size n has 2^(n)  1 nonempty subsets.) While an explicit characterization of the space of entropy vectors is wellknown for n = 2, 3 random variables, it is unknown for n > 3 (which is why most network information theory problems are open.) We will construct inner bounds to the space of entropic vectors using tools such as quasiuniform distributions, lattices, and Cayley's hyperdeterminant.https://authors.library.caltech.edu/records/6h00rm2m06Weighted ℓ_1 minimization for sparse recovery with prior information
https://resolver.caltech.edu/CaltechAUTHORS:20100816133504795
Authors: Khajehnejad, M. Amin; Xu, Weiyu; Avestimehr, A. Salman; Hassibi, Babak
Year: 2009
DOI: 10.1109/ISIT.2009.5205716
In this paper we study the compressed sensing problem of recovering a sparse signal from a system of underdetermined linear equations when we have prior information about the probability of each entry of the unknown signal being nonzero. In particular, we focus on a model where the entries of the unknown vector fall into two sets, each with a different probability of being nonzero. We propose a weighted ℓ_1 minimization recovery algorithm and analyze its performance using a Grassman angle approach. We compute explicitly the relationship between the system parameters (the weights, the number of measurements, the size of the two sets, the probabilities of being nonzero) so that an iid random Gaussian measurement matrix along with weighted ℓ_1 minimization recovers almost all such sparse signals with overwhelming probability as the problem dimension increases. This allows us to compute the optimal weights. We also provide simulations to demonstrate the advantages of the method over conventional ℓ_1 optimization.https://authors.library.caltech.edu/records/w2tdbhzr97On sharp performance bounds for robust sparse signal recoveries
https://resolver.caltech.edu/CaltechAUTHORS:20100816144533590
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2009
DOI: 10.1109/ISIT.2009.5205718
It is well known in compressive sensing that l_1 minimization can recover the sparsest solution for a large class of underdetermined systems of linear equations, provided the signal is sufficiently sparse. In this paper, we compute sharp performance bounds for several different notions of robustness in sparse signal recovery via l_1 minimization. In particular, we determine necessary and sufficient conditions for the measurement matrix A under which l_1 minimization guarantees the robustness of sparse signal recovery in the "weak", "sectional" and "strong" (e.g., robustness for "almost all" approximately sparse signals, or instead for "all" approximately sparse signals). Based on these characterizations, we are able to compute sharp performance bounds on the tradeoff between signal sparsity and signal recovery robustness in these various senses. Our results are based on a highdimensional geometrical analysis of the nullspace of the measurement matrix A. These results generalize the thresholds results for purely sparse signals and also present generalized insights on l_1 minimization for recovering purely sparse signals from a nullspace perspective.https://authors.library.caltech.edu/records/1vvapqpe23Violating the Ingleton inequality with finite groups
https://resolver.caltech.edu/CaltechAUTHORS:20100804152510742
Authors: Mao, Wei; Hassibi, Babak
Year: 2009
DOI: 10.1109/ALLERTON.2009.5394878
It is well known that there is a onetoone correspondence
between the entropy vector of a collection of n random variables and a certain groupcharacterizable vector obtained from a finite group and n of its subgroups [1]. However, if one restricts attention to abelian groups then not all entropy vectors can be obtained. This is an explanation for the fact shown by Dougherty et al [2] that linear network codes cannot achieve capacity in general network coding problems (since linear network codes form an abelian group). All abelian groupcharacterizable vectors, and by fiat all entropy vectors generated by linear network codes, satisfy a linear inequality called the Ingleton inequality. In this paper, we study the problem of finding nonabelian finite groups that yield characterizable vectors which violate the Ingleton inequality. Using a refined computer search, we find the symmetric group S_5 to be the smallest group that violates the Ingleton inequality. Careful study of the structure of this group, and its subgroups, reveals that it belongs to the Ingletonviolating family PGL(2, p) with primes p ≥ 5, i.e., the projective group of 2 × 2 nonsingular matrices with entries in F_p. This family of groups is therefore a good candidate for constructing network codes more powerful than linear network codes.https://authors.library.caltech.edu/records/e8vdfknx78Generalizing Kronecker graphs in order to model searchable networks
https://resolver.caltech.edu/CaltechAUTHORS:20100805080815717
Authors: Bodine, Elizabeth; Hassibi, Babak; Wierman, Adam
Year: 2009
DOI: 10.1109/ALLERTON.2009.5394816
This paper describes an extension to stochastic
Kronecker graphs that provides the special structure required
for searchability, by defining a "distance"dependent Kronecker
operator. We show how this extension of Kronecker graphs
can generate several existing social network models, such as
the WattsStrogatz smallworld model and Kleinberg's latticebased
model. We focus on a specific example of an expanding
hypercube, reminiscent of recently proposed social network
models based on a hidden hyperbolic metric space, and prove
that a greedy forwarding algorithm can find very short paths
of length O((log log n)^2) for graphs with n nodes.https://authors.library.caltech.edu/records/byfkbqck76The Kalman Like Particle Filter: Optimal Estimation With Quantized Innovations/Measurements
https://resolver.caltech.edu/CaltechAUTHORS:20150204072431118
Authors: Sukhavasi, Ravi Teja; Hassibi, Babak
Year: 2009
DOI: 10.1109/CDC.2009.5400517
We study the problem of optimal estimation using quantized innovations, with application to distributed estimation over sensor networks. We show that the state probability density conditioned on the quantized innovations can be expressed as the sum of a Gaussian random vector and a certain truncated Gaussian vector. This structure bears close resemblance to the full information Kalman filter and so allows us to effectively combine the Kalman structure with a particle filter to recursively compute the state estimate. We call the resulting filter the Kalman like particle filter (KLPF) and observe that it delivers close to optimal performance using far fewer particles than that of a particle filter directly applied to the original problem. We also note that the conditional state density follows a, so called, generalized closed skewnormal (GCSN) distribution.https://authors.library.caltech.edu/records/j42wyksj63On the steadystate performance of Kalman filtering with intermittent observations for stable systems
https://resolver.caltech.edu/CaltechAUTHORS:20190226095328309
Authors: Vakili, Ali; Hassibi, Babak
Year: 2009
DOI: 10.1109/CDC.2009.5400386
Many recent problems in distributed estimation and control reduce to estimating the state of a dynamical system using sensor measurements that are transmitted across a lossy network. A framework for analyzing such systems was proposed in and called Kalman filtering with intermittent observations. The performance of such a system, i.e., the error covariance matrix, is governed by the solution of a matrixvalued random Riccati recursion. Unfortunately, to date, the tools for analyzing such recursions are woefully lacking, ostensibly because the recursions are both nonlinear and random, and hence intractable if one wants to analyze them exactly. In this paper, we extend some of the large random matrix techniques first introduced in to Kalman filtering with intermittent observations. For systems with a stable system matrix and i.i.d. timevarying measurement matrices, we obtain explicit equations that allow one to compute the asymptotic eigendistribution of the error covariance matrix. Simulations show excellent agreement between the theoretical and empirical results for systems with as low as n = 10, 20 states. Extending the results to unstable system matrices and timeinvariant measurement matrices is currently under investigation.https://authors.library.caltech.edu/records/06xf2ecx83NearOptimal Detection in MIMO Systems using Gibbs Sampling
https://resolver.caltech.edu/CaltechAUTHORS:20150130070829687
Authors: Hansen, Morten; Hassibi, Babak; Dimakis, Alexandros G.; Xu, Weiyu
Year: 2009
DOI: 10.1109/GLOCOM.2009.5425927
In this paper we study a Markov Chain Monte Carlo (MCMC) Gibbs sampler for solving the integer leastsquares problem. In digital communication the problem is equivalent to performing Maximum Likelihood (ML) detection in MultipleInput MultipleOutput (MIMO) systems. While the use of MCMC methods for such problems has already been proposed, our method is novel in that we optimize the "temperature" parameter so that in steady state, i.e. after the Markov chain has mixed, there is only polynomially (rather than exponentially) small probability of encountering the optimal solution. More precisely, we obtain the largest value of the temperature parameter for this to occur, since the higher the temperature, the faster the mixing. This is in contrast to simulated annealing techniques where, rather than being held fixed, the temperature parameter is tended to zero. Simulations suggest that the resulting Gibbs sampler provides a computationally efficient way of achieving
approximative ML detection in MIMO systems having a huge number of transmit and receive dimensions. In fact, they further suggest that the Markov chain is rapidly mixing. Thus, it has been observed that even in cases were ML detection using, e.g. sphere decoding becomes infeasible, the Gibbs sampler can still offer a nearoptimal solution using much less computations.https://authors.library.caltech.edu/records/mrsxxznb71On Group Network Codes: IngletonBound
Violations and Independent Sources
https://resolver.caltech.edu/CaltechAUTHORS:20110329114452647
Authors: Mao, Wei; Thill, Matthew; Hassibi, Babak
Year: 2010
DOI: 10.1109/ISIT.2010.5513749
In principle, network codes derived from nonAbelian groups can be used to attain every point in the capacity region of wired acyclic networks. However, group codes derived from a particular group, and its subgroups, is useful only if it can model independent sources, as well as violate the Ingleton bound which restricts the capacity region obtainable by linear network codes. We study both the independent source and the Ingletonviolating requirement for subgroups of the groups PGL(2,p) and GL(2,p) with primes p ≥ 5. For both these groups we demonstrate that the requirements can be met, which suggests that PGL(2,p) and GL(2,p) are rich enough groups to construct network codes superior to linear ones. We also construct a model for independent sources using the direct product of the aforementioned groups.https://authors.library.caltech.edu/records/mxm0f3e867MCMC Methods for Entropy Optimization and
Nonlinear Network Coding
https://resolver.caltech.edu/CaltechAUTHORS:20110330091945765
Authors: Shadbakht, Sormeh; Hassibi, Babak
Year: 2010
DOI: 10.1109/ISIT.2010.5513737
Although determining the space of entropic vectors for n random variables, denoted by Γ^*_n, is crucial for solving a large class of network information theory problems, there has been scant progress in explicitly characterizing Γ^*_n for n ≥ 4. In this paper, we present a certain characterization of quasiuniform distributions that allows one to numerically stake out the entropic region via a random walk to any desired accuracy. When coupled with Monte Carlo Markov Chain (MCMC) methods, one may "bias" the random walk so as to maximize certain functions of the entropy vector. As an example, we look at maximizing the violation of the Ingleton inequality for four random variables and report a violation well in excess of what has been previously available in the literature. Inspired by the MCMC method, we also propose a framework for designing optimal nonlinear network codes via performing a random walk over certain truth tables. We show that the method can be decentralized and demonstrate its efficacy by applying it to the Vamos network and a certain storage problem from [1].https://authors.library.caltech.edu/records/t9whrjyb50Iterative weighted ℓ_1 Optimization for compressed sensing and coding
https://resolver.caltech.edu/CaltechAUTHORS:20150224074009355
Authors: Khajehnejad, M. Amin; Dimakis, Alexandros G.; Hassibi, Babak
Year: 2010
DOI: 10.1109/ACSSC.2010.5757668
We introduce a novel algorithm for decoding binary linear codes by linear programming. We build on the LP decoding algorithm of Feldman et al. and introduce a postprocessing step that solves a second linear program that reweights the objective function based on the outcome of the original LP decoder output. Our analysis shows that for some LDPC ensembles we can improve the provable threshold guarantees compared to standard LP decoding. We also show significant empirical performance gains for the reweighted LP decoding algorithm with very small additional computational complexity.https://authors.library.caltech.edu/records/24trr34112A symmetric adaptive algorithm for speedingup consensus
https://resolver.caltech.edu/CaltechAUTHORS:20110413133402007
Authors: Thai, Daniel; BodineBaron, Elizabeth; Hassibi, Babak
Year: 2010
DOI: 10.1109/ICASSP.2010.5496237
Performing distributed consensus in a network has been an
important research problem for several years, and is directly
applicable to sensor networks, autonomous vehicle formation,
etc. While there exists a wide variety of algorithms that can be
proven to asymptotically reach consensus, in applications involving
timevarying parameters and tracking, it is often crucial
to reach consensus "as quickly as possible". In [?] it has
been shown that, with global knowledge of the network topology,
it is possible to optimize the convergence time in distributed
averaging algorithms via solving a semidefinite program
(SDP) to obtain the optimal averaging weights. Unfortunately,
in most applications, nodes do not have knowledge
of the full network topology and cannot implement the required
SDP in a distributed fashion. In this paper, we present a
symmetric adaptive weight algorithm for distributed consensus
averaging on bidirectional noiseless networks. The algorithm
uses an LMS (Least Mean Squares) approach to adaptively
update the edge weights used to calculate each node's
values. The derivation shows that global error can be minimized
in a distributed fashion and that the resulting adaptive
weights are symmetric—symmetry being critical for convergence
to the true average. Simulations show that convergence
time is nearly equal to that of a nonsymmetric adaptive algorithm
developed in [?], and significantly better than that of
the nonadaptive MetropolisHastings algorithm. Most importantly,
our symmetric adaptive algorithm converges to the
sample mean, whereas the method of [?] converges to an arbitrary
value and results in significant error.https://authors.library.caltech.edu/records/dv0a927e32Breaking through the Thresholds: an Analysis for Iterative Reweighted ℓ_1 Minimization via the Grassmann Angle Framework
https://resolver.caltech.edu/CaltechAUTHORS:20150130073717820
Authors: Xu, Weiyu; Khajehnejad, M. Amin; Avestimehr, A. Salman; Hassibi, Babak
Year: 2010
DOI: 10.1109/ICASSP.2010.5495210
It is now well understood that the ℓ_1 minimization algorithm is able to recover sparse signals from incomplete measurements [2], [1], [3] and sharp recoverable sparsity thresholds have also been obtained for the ℓ_1 minimization algorithm. However, even though iterative reweighted ℓ_1 minimization algorithms or related algorithms have been empirically observed to boost the recoverable sparsity thresholds for certain types of signals, no rigorous theoretical results have been established to prove this fact. In this paper, we try to provide a theoretical foundation for analyzing the iterative reweighted ℓ1 algorithms. In particular, we show that for a nontrivial class of signals, the iterative reweighted ℓ_1 minimization can indeed deliver recoverable sparsity thresholds larger than that given in [1], [3]. Our results are based on a highdimensional geometrical analysis (Grassmann angle analysis) of the nullspace characterization for ℓ_1 minimization and weighted ℓ1 minimization algorithms.https://authors.library.caltech.edu/records/ahbd6m6a02Scalar Linear Network Coding for Networks with Two Sources
https://resolver.caltech.edu/CaltechAUTHORS:20190226084142523
Authors: Shadbakht, Sormeh; Jafarian, Amin; Hassibi, Babak
Year: 2010
DOI: 10.1109/ICC.2010.5501824
Determining the capacity of networks has been a longstanding issue of interest in the literature. Although for multisource multisink networks it is known that using network coding is advantageous over traditional routing, finding the best coding strategy is not trivial in general. Among different classes of codes that could be potentially used in a network, linear codes due to their simplicity are of particular interest. Although linear codes are proven to be suboptimal in general, in some cases such as the multicast scenario they achieve the cutset bound. Since determining the capacity of a network is closely related to the characterization of the entropy region of all its random variables, if one is interested in finding the best linear solution for a network, one should find the region of all linear representable entropy vectors of that network. With this approach, we study the scalar linear solutions over arbitrary network problems with two sources. We explicitly calculate this region for small number of variables and suggest a method for larger networks through finding the best scalar linear solution to a storage problem as an example of practical interest.https://authors.library.caltech.edu/records/hrx6f3zf17Improved sparse recovery thresholds with twostep reweighted ℓ_1 minimization
https://resolver.caltech.edu/CaltechAUTHORS:20110329105653717
Authors: Khajehnejad, M. Amin; Xu, Weiyu; Avestimehr, A. Salman; Hassibi, Babak
Year: 2010
DOI: 10.1109/ISIT.2010.5513417
It is well known that ℓ_1 minimization can be used to recover sufficiently sparse unknown signals from compressed linear measurements. In fact, exact thresholds on the sparsity, as a function of the ratio between the system dimensions, so that with high probability almost all sparse signals can be recovered from iid Gaussian measurements, have been computed and are referred to as "weak thresholds" [4]. In this paper, we introduce a reweighted ℓ_1 recovery algorithm composed of two steps: a standard ℓ_1 minimization step to identify a set of entries where the signal is likely to reside, and a weighted ℓ_1 minimization step where entries outside this set are penalized. For signals where the nonsparse component has iid Gaussian entries, we prove a "strict" improvement in the weak recovery threshold. Simulations suggest that the improvement can be quite impressive—over 20% in the example we consider.https://authors.library.caltech.edu/records/1bh73jpv85Reweighted LP decoding for LDPC codes
https://resolver.caltech.edu/CaltechAUTHORS:20170314164934408
Authors: Khajehnejad, M. Amin; Dimakis, Alexandros G.; Hassibi, Babak; Bradley, William
Year: 2010
DOI: 10.1109/ALLERTON.2010.5706905
We introduce a novel algorithm for decoding binary linear codes by linear programming. We build on the LP decoding algorithm of Feldman et al. and introduce a postprocessing step that solves a second linear program that reweights the objective function based on the outcome of the original LP decoder output. Our analysis shows that for some LDPC ensembles we can improve the provable threshold guarantees compared to standard LP decoding. We also show significant empirical performance gains for the reweighted LP decoding algorithm with very small additional computational complexity.https://authors.library.caltech.edu/records/vv52tahx61MCMC methods for integer leastsquares problems
https://resolver.caltech.edu/CaltechAUTHORS:20150204072014452
Authors: Hassibi, Babak; Dimakis, Alexandros G.; Papailiopoulos, Dimitris
Year: 2010
DOI: 10.1109/ALLERTON.2010.5706947
We consider the problem of finding the leastsquares solution to a system of linear equations where the unknown vector has integer entries (or, more precisely, has entries belonging to a subset of the integers), yet where the coefficient matrix and given vector are comprised of real numbers. Geometrically, this problem is equivalent to finding the closest lattice point to a given point and is known to be NP hard. In communication applications, however, the given vector is not arbitrary, but is a lattice point perturbed by some noise vector. Therefore it is of interest to study the computational complexity of various algorithms as a function of the noise variance or, often more appropriately, the SNR.
In this paper, we apply a particular version of the Monte Carlo Markov chain (MCMC) approach to solving this problem, which is called a "heat bath". We show that there is a tradeoff between the mixing time of the Markov chain (how long it takes until the chain reaches its stationary distribution) and how long it takes for the algorithm to find the optimal solution once the chain has mixed. The complexity of the algorithm is essentially the sum of these two times. More specifically, the higher the temperature, the faster the mixing, yet the slower the discovery of the optimal solution in steady state. Conversely, the lower the temperature, the slower the mixing, yet the faster the discovery of the optimal solution once the chain is mixed.
We first show that for the probability of error of the maximumlikelihood (ML) solution to go to zero the SNR must scale at least as 2 ln N + α(N), where N is the ambient problem dimension and α(N) is any sequence that tends to positive infinity. We further obtain the optimal value of the temperature such that the average time required to encounter the optimal solution in steady state is polynomial. Simulations show that, with this choice of the temperature parameter, the optimal solution can be found in reasonable time

. This suggests that the Markov chain mixes in polynomialtime, though we have not been able to prove this. It seems reasonable to conjecture that for SNR scaling as O((ln(N))1+∈), and for appropriate choice of the temperature parameter, the heat bath algorithm finds the optimal solution in polynomialtime.https://authors.library.caltech.edu/records/5fh3xd7p70Peer Effects and Stability in Matching Markets
https://resolver.caltech.edu/CaltechAUTHORS:20150127074255645
Authors: BodineBaron, Elizabeth; Lee, Christina; Chong, Anthony; Hassibi, Babak; Wierman, Adam
Year: 2011
DOI: 10.1007/9783642248290_12
Manytoone matching markets exist in numerous different forms, such as college admissions, matching medical interns to hospitals for residencies, assigning housing to college students, and the classic firms and workers market. In all these markets, externalities such as complementarities and peer effects severely complicate the preference ordering of each agent. Further, research has shown that externalities lead to serious problems for market stability and for developing efficient algorithms to find stable matchings. In this paper we make the observation that peer effects are often the result of underlying social connections, and we explore a formulation of the manytoone matching market where peer effects are derived from an underlying social network. The key feature of our model is that it captures peer effects and complementarities using utility functions, rather than traditional preference ordering. With this model and considering a weaker notion of stability, namely
twosided exchange stability, we prove that stable matchings always exist and characterize the set of stable matchings in terms of social welfare. We also give distributed algorithms that are guaranteed to converge to a twosided exchange stable matching. To assess the competitive ratio of these algorithms and to more generally characterize the efficiency of matching markets with externalities, we provide general bounds on how far the welfare of the worstcase stable matching can be from the welfare of the optimal matching, and find that the structure of the social network (e.g. how well clustered the network is) plays a large role.https://authors.library.caltech.edu/records/7gx5tf8h88Improved thresholds for rank minimization
https://resolver.caltech.edu/CaltechAUTHORS:20150204070559629
Authors: Oymak, Samet; Khajehnejad, M. Amin; Hassibi, Babak
Year: 2011
DOI: 10.1109/ICASSP.2011.5947726
Nuclear norm minimization (NNM) has recently gained attention for its use in rank minimization problems. In this paper, we define weak, sectional and strong recovery for NNM to succeed at finding the low rank solution. We find tight conditions for these and analyze them for the case where the linear measurement operator consists of i.i.d. Gaussian entries. Finally we calculate the so called weak, sectional and strong thresholds for the success of nuclear norm minimization. To obtain our results, we generalize the notion of sign and support from sparse vectors to low rank matrices, and achieve a weak threshold which is much closer to the empirical phase transition curve of nuclear norm minimization than the existing bounds available in the literature.https://authors.library.caltech.edu/records/c5cm62kn55Compressed Network Tomography for Probabilistic Tree Mixture Models
https://resolver.caltech.edu/CaltechAUTHORS:20150224073614235
Authors: Khajehnejad, M. Amin; Khojastepour, Amir; Hassibi, Babak
Year: 2011
DOI: 10.1109/GLOCOM.2011.6133853
We consider the problem of network tomography
in probabilistic tree mixture models. We invoke the theory of compressed sensing and prove that the distribution of a random communication network model with n nodes represented by a probabilistic mixture of k trees can be identified using low order routing summaries pertinent to groups of small sizes d << n in the network. We prove that, if the number of collected statistics m is at least O(n^(log k)), then certain classes of inference algorithms can successfully determine the unknown model, i.e. the topologies of mixing trees and their corresponding
probabilities. We show that a variation of ℓ_1
minimization over the space of all possible trees of
n nodes can be used for this purpose. In addition, we propose a novel inference algorithm
with a complexity polynomial in
n^(log k), with the same provable
guarantee. The proposed model is applicable to practical situations such as adhoc and PeertoPeer(P2P) networks, and the presented inference method can lead to distributed protocols for network monitoring and tomography. In particular, we provide preliminary insight and numerical results on how the ideas are amenable to wireless sensor networks.https://authors.library.caltech.edu/records/36qdtbvq57Anytime reliable codes for stabilizing plants over erasure channels
https://resolver.caltech.edu/CaltechAUTHORS:20150204071707257
Authors: Sukhavasi, Ravi Teja; Hassibi, Babak
Year: 2011
DOI: 10.1109/CDC.2011.6161478
The problem of stabilizing an unstable plant over a noisy communication link is an increasingly important one that arises in problems of distributed control and networked control systems. Although the work of Schulman and Sahai over the past two decades, and their development of the notions of "tree codes" and "anytime capacity", provides the theoretical framework for studying such problems, there has been scant practical progress in this area because explicit constructions of tree codes with efficient encoding and decoding did not exist. To stabilize an unstable plant driven by bounded noise over a noisy channel one needs realtime encoding and realtime decoding and a reliability which increases exponentially with delay, which is what tree codes guarantee. We prove the existence of linear tree codes with high probability and, for erasure channels, give a construction with an expected decoding complexity that is constant per time instant. We give sufficient conditions on the rate and reliability required of the tree codes to stabilize vector plants and argue that they are asymptotically tight. This work takes an important step towards controlling plants over noisy channels, and we demonstrate the efficacy of the method through a simulation.https://authors.library.caltech.edu/records/v5891wdc46Weighted compressed sensing and rank minimization
https://resolver.caltech.edu/CaltechAUTHORS:20150204071344814
Authors: Oymak, Samet; Khajehnejad, M. Amin; Hassibi, Babak
Year: 2011
DOI: 10.1109/ICASSP.2011.5947163
We present an alternative analysis of weighted ℓ_1 minimization for sparse signals with a nonuniform sparsity model, and extend our results to nuclear norm minimization for matrices with nonuniform singular vector distribution. In the case of vectors, we find explicit upper bounds for the successful recovery thresholds, and give a simple suboptimal weighting rule. For matrices, the thresholds we find are only implicit, and the optimal weight selection requires an exhaustive search. For the special case of very wide matrices, the relationship is made explicit and the optimal weight assignment is the same as the vector case. We demonstrate through simulations that for vectors, the suggested weighting scheme improves the recovery performance over that of regular ℓ_1 minimization.https://authors.library.caltech.edu/records/qrm01jmr52Tight Recovery Thresholds and Robustness Analysis for Nuclear Norm Minimization
https://resolver.caltech.edu/CaltechAUTHORS:20120406113136818
Authors: Oymak, Samet; Hassibi, Babak
Year: 2011
DOI: 10.1109/ISIT.2011.6033977
Nuclear norm minimization (NNM) has recently gained significant attention for its use in rank minimization problems. Using null space characterizations, recovery thresholds for NNM have been previously studied for the case of Gaussian measurements as matrix dimensions tend to infinity. However simulations show that the thresholds are far from optimal, especially in the low rank region. In this paper we apply the recent analysis of Stojnic for ℓ_1minimization to the null space conditions of NNM. The results are significantly better and in particular our weak threshold appears to match with simulation results. Further, our closed form bounds suggest for any rank growing linearly with matrix size n one needs only three times of oversampling (the model complexity) for weak recovery and eight times for strong recovery. Additionally, the results for robustness analysis are given which indicate with slightly more measurements recovery guarantees for approximately low rank matrices can be given.https://authors.library.caltech.edu/records/8t05krzb72Summary Based Structures with Improved Sublinear Recovery for Compressed Sensing
https://resolver.caltech.edu/CaltechAUTHORS:20120406072754339
Authors: Khajehnejad, M. Amin; Yoo, Juhwan; Anandkumar, Animashree; Hassibi, Babak
Year: 2011
DOI: 10.1109/ISIT.2011.6033775
We introduce a new class of measurement matrices for compressed sensing, using low order summaries over binary sequences of a given length. We prove recovery guarantees for three reconstruction algorithms using the proposed measurements, including ℓ_1 minimization and two combinatorial methods. In particular, one of the algorithms recovers ksparse vectors of length N in sublinear time poly(k log N), and requires at most O(k log N log log N) measurements. The empirical oversampling constant of the algorithm is significantly better than existing sublinear recovery algorithms such as Chaining Pursuit and Sudocodes. In particular, for 10^3 ≤ N ≤ 10^(12) and k = 100, the oversampling factor is between 5 to 25. We provide preliminary insight into how the proposed constructions, and the fast recovery scheme can be used in a number of practical applications such as market basket analysis, and real time compressed sensing implementation.https://authors.library.caltech.edu/records/cqdbkea648Subspace Expanders and Matrix Rank Minimization
https://resolver.caltech.edu/CaltechAUTHORS:20120406111241824
Authors: Oymak, Samet; Khajehnejad, Amin; Hassibi, Babak
Year: 2011
DOI: 10.1109/ISIT.2011.6033974
Matrix rank minimization (RM) problems recently gained extensive attention due to numerous applications in machine learning, system identification and graphical models. In RM problem, one aims to find the matrix with the lowest rank that satisfies a set of linear constraints. The existing algorithms include nuclear norm minimization (NNM) and singular value thresholding. Thus far, most of the attention has been on i.i.d. Gaussian or Bernoulli measurement operators. In this work, we introduce a new class of measurement operators, and a novel recovery algorithm, which is notably faster than NNM. The proposed operators are based on what we refer to as subspace expanders, which are inspired by the well known expander graphs based measurement matrices in compressed sensing. We show that given an n×n PSD matrix of rank r, it can be uniquely recovered from a minimal sampling of O(nr) measurements using the proposed structures, and the recovery algorithm can be cast as matrix inversion after a few initial processing steps.https://authors.library.caltech.edu/records/cj3821bv14Explicit Matrices for Sparse Approximation
https://resolver.caltech.edu/CaltechAUTHORS:20120405093333989
Authors: Khajehnejad, Amin; Tehrani, Arash Saber; Dimakis, Alexandros G.; Hassibi, Babak
Year: 2011
DOI: 10.1109/ISIT.2011.6034170
We show that girth can be used to certify that sparse compressed sensing matrices have good sparse approximation guarantees. This allows us to present the first deterministic measurement matrix constructions that have an optimal number of measurements for ℓ_1/ℓ_1 approximation. Our techniques are coding theoretic and rely on a recent connection of compressed sensing to LP relaxations for channel decoding.https://authors.library.caltech.edu/records/embvdxj002A Simplified Approach to Recovery Conditions for
Low Rank Matrices
https://resolver.caltech.edu/CaltechAUTHORS:20120406112117384
Authors: Oymak, Samet; Mohan, Karthik; Fazel, Maryam; Hassibi, Babak
Year: 2011
DOI: 10.1109/ISIT.2011.6033976
Recovering sparse vectors and lowrank matrices from noisy linear measurements has been the focus of much recent research. Various reconstruction algorithms have been studied, including ℓ_1 and nuclear norm minimization as well as ℓ_p minimization with p < 1. These algorithms are known to succeed if certain conditions on the measurement map are satisfied. Proofs for the recovery of matrices have so far been much more involved than in the vector case. In this paper, we show how several classes of recovery conditions can be extended from vectors to matrices in a simple and transparent way, leading to the best known restricted isometry and nullspace conditions for matrix recovery. Our results rely on the ability to "vectorize" matrices through the use of a key singular value inequality.https://authors.library.caltech.edu/records/0rwpnyye61Linear error correcting codes with anytime reliability
https://resolver.caltech.edu/CaltechAUTHORS:20150204071048761
Authors: Sukhavasi, Ravi Teja; Hassibi, Babak
Year: 2011
DOI: 10.1109/ISIT.2011.6033847
We consider rate R = k/n causal linear codes that map a sequence of kdimensional binary vectors {b_t}_(t=0)^∞ to a sequence of ndimensional binary vectors {ct}_(t=0)^∞, such that each ct is a function of {b_τ}_(τ=0)^t. Such a code is called anytime reliable, for a particular binaryinput memoryless channel, if at each time instant t, and for all delays d ≥ d_0, the probability of error P(b_(tdt) ≠ b_(td)) decays exponentially in d, i.e., P(b_(tdt) ≠ b_(td)) ≤ η2^(βnd), for some β >; 0. Anytime reliable codes are useful in interactive communication problems and, in particular, can be used to stabilize unstable plants across noisy channels. Schulman proved the existence of such codes which, due to their structure, he called tree codes in [1]; however, to date, no explicit constructions and tractable decoding algorithms have been devised. In this paper, we show the existence of anytime reliable "linear" codes with "high probability", i.e., suitably chosen random linear causal codes are anytime reliable with high probability. The key is to consider timeinvariant codes (i.e., ones with Toeplitz generator and parity check matrices) which obviates the need to union bound over all times. For the binary erasure channel we give a simple ML decoding algorithm whose average complexity is constant per time instant and for which the probability that complexity at a given time t exceeds KC^3 decays exponentially in C. We show the efficacy of the method by simulating the stabilization of an unstable plant across a BEC, and remark on the tradeoffs between the utilization of the communication resources and the control performance.https://authors.library.caltech.edu/records/4jyv264h44FRETBased RealTime DNA Microarrays
https://resolver.caltech.edu/CaltechAUTHORS:20181010162103240
Authors: Hassibi, Arjang; Vikalo, Haris; Riechmann, José Luis; Hassibi, Babak
Year: 2011
DOI: 10.1007/9781617794247_12
We present a quantification method for affinitybased DNA microarrays which is based on the realtime measurements of hybridization kinetics. This method, i.e., realtime DNA microarrays, enhances the detection dynamic range of conventional systems by being impervious to probe saturation, washing artifacts, microarray spottospot variations, and other intensityaffecting impediments. We demonstrate in both theory and practice that the timeconstant of target capturing is inversely proportional to the concentration of the target analyte, which we take advantage of as the fundamental parameter to estimate the concentration of the analytes. Furthermore, to experimentally validate the capabilities of this method in practical applications, we present a FRETbased assay which enables the realtime detection in gene expression DNA microarrays.https://authors.library.caltech.edu/records/pb824njx81On the Mixing Time of Markov Chain Monte Carlo for Integer LeastSquare Problems
https://resolver.caltech.edu/CaltechAUTHORS:20150126074841944
Authors: Xu, Weiyu; Dimakis, Alexandros G.; Hassibi, Babak
Year: 2012
DOI: 10.1109/CDC.2012.6425890
In this paper, we study the mixing time of Markov Chain Monte Carlo (MCMC) for integer leastsquare (LS) optimization problems. It is found that the mixing time of
MCMC for integer LS problems depends on the structure of the underlying lattice. More specifically, the mixing time of MCMC is closely related to whether there is a local minimum in the lattice structure. For some lattices, the mixing time
of the Markov chain is independent of the signaltonoise ratio (SNR) and grows polynomially in the problem dimension; while for some lattices, the mixing time grows unboundedly as SNR grows. Both theoretical and empirical results suggest
that to ensure fast mixing, the temperature for MCMC should often grow positively as the
SNR increases. We also derive the probability that there exist local minima in an integer leastsquare problem, which can be as high as 1/3  1/√5 + (2 arctan(√(5/3))/(√5Π).https://authors.library.caltech.edu/records/4cgfkf2e43Fundamental thresholds in compressed sensing: a highdimensional geometry approach
https://resolver.caltech.edu/CaltechAUTHORS:20121107095923799
Authors: Xu, Weiyu; Hassibi, Babak
Year: 2012
DOI: 10.1017/CBO9780511794308.008
In this chapter, we introduce a unified highdimensional geometric framework for analyzing the phase transition phenomenon of ℓ_1 minimization in compressive sensing. This framework connects studying the phase transitions of ℓ_1 minimization with computing the Grassmann angles in highdimensional convex geometry. We demonstrate the broad applications of this Grassmann angle framework by giving sharp phase transitions for ℓ_1 minimization recovery robustness, weighted ℓ_1 minimization algorithms, and iterative reweighted ℓ_1 minimization algorithms.https://authors.library.caltech.edu/records/1pmdtkez63Deterministic phase guarantees for robust recovery in incoherent dictionaries
https://resolver.caltech.edu/CaltechAUTHORS:20150224072653860
Authors: Li, Cheuk Ting; Oymak, Samet; Hassibi, Babak
Year: 2012
DOI: 10.1109/ICASSP.2012.6288749
This paper presents a relaxation of an assumption usually imposed in the recovery of sparse vectors with random support in pairs of orthonormal bases or incoherent dictionaries by basis pursuit. The assumption requires the phases of the entries of the sparse vector to be chosen randomly in [0, 2π). This paper provides probabilistic recovery guarantees for deterministic phases. We prove that, if a phase pattern is fixed, then a sparse vector with random support and corresponding phases can be recovered with high probability. As a result, the phases can take any distribution and can be dependent, as long as they are independent of the support. Furthermore, this improvement does not come at the expense of the maximum recoverable sparsity.https://authors.library.caltech.edu/records/xtkd8t1a81A simpler approach to weighted ℓ_1 minimization
https://resolver.caltech.edu/CaltechAUTHORS:20150224072919592
Authors: Krishnaswamy, Anilesh K.; Oymak, Samet; Hassibi, Babak
Year: 2012
DOI: 10.1109/ICASSP.2012.6288700
In this paper, we analyze the performance of weighted ℓ_1 minimization over a nonuniform sparse signal model by extending the "Gaussian width" analysis proposed in [1]. Our results are consistent with those of [7] which are currently the best known ones. However, our methods are less computationally intensive and can be easily extended to signals which have more than two sparsity classes. Finally, we also provide a heuristic for estimating the optimal weights, building on a more general model presented in [11]. Our results reinforce the fact that weighted ℓ_1 minimization is substantially better than regular ℓ_1 minimization and provide an easy way to calculate the optimal weights.https://authors.library.caltech.edu/records/bb66e1gg56Projected ℓ_1Minimization for Compressed Sensing
https://resolver.caltech.edu/CaltechAUTHORS:20120907082737118
Authors: Khajehnejad, Amin; Thill, Matthew; Hassibi, Babak
Year: 2012
DOI: 10.1109/ICASSP.2012.6288699
We propose a new algorithm to recover a sparse signal from a system of linear measurements. By projecting the measured signal onto a properly chosen subspace, we can use the projection to zero in on a lowsparsity portion of our original signal, which we can recover using ℓ_1minimization. We can then recover the remaining portion of our signal from an overdetermined system of linear equations. We prove that our scheme improves the threshold of ℓ_1minimization, and we derive an upper bound for this new threshold. We support our theoretical results with numerical simulations which demonstrate that certain classes of signals come close to achieving this upper bound.https://authors.library.caltech.edu/records/7t8hyhx466Phase retrieval for sparse signals using rank minimization
https://resolver.caltech.edu/CaltechAUTHORS:20130208133900478
Authors: Jaganathan, Kishore; Oymak, Samet; Hassibi, Babak
Year: 2012
DOI: 10.1109/ICASSP.2012.6288658
Signal recovery from the amplitudes of the Fourier transform, or equivalently from the autocorrelation function is a classical problem. Due to the absence of phase information, signal recovery requires some form of additional prior information. In this paper, the prior information we assume is sparsity. We develop a convex optimization based framework to retrieve the signal support from the support of the autocorrelation, and propose an iterative algorithm which terminates in a signal with the least sparsity satisfying the autocorrelation constraints. Numerical results suggest that unique recovery up to a global sign change, time shift and/or time reversal is possible with a very high probability for sufficiently sparse signals.https://authors.library.caltech.edu/records/58x4cj9q58An analog sublinear time sparse signal acquisition framework based on structured matrices
https://resolver.caltech.edu/CaltechAUTHORS:20120907092240681
Authors: Yoo, Juhwan; Khajehnejad, Amin; Hassibi, Babak; EmamiNeyestanak, Azita
Year: 2012
DOI: 10.1109/ICASSP.2012.6289122
Advances in compressedsensing (CS) have sparked interest in designing information acquisition systems that process data at close to the information rate. Initial proposals for CS signal acquisition systems utilized random matrix ensembles in conjunction with convex relaxation based signal reconstruction algorithms. While providing universal performance bounds, random matrix based formulations present several practical problems due to: the difficulty in physically implementing key mathematical operations, and their dense representation. In this paper, we present a CS architecture which is based on a sublinear time recovery algorithm (with minimum memory requirement) that exploits a novel structured matrix. This formulation allows the use of a reconstruction algorithm based on relatively simple computational primitives making it more amenable to implementation in a fullyintegrated form. Theoretical recovery guarantees are discussed and a hypothetical physical CS decoder is described.https://authors.library.caltech.edu/records/w0gmwfr543Minimizing the social cost of an epidemic
https://resolver.caltech.edu/CaltechAUTHORS:20150223074754304
Authors: BodineBaron, Elizabeth; Bose, Subhonmesh; Hassibi, Babak; Wierman, Adam
Year: 2012
DOI: 10.1007/9783642303739_41
In this paper we quantify the total cost of an epidemic spreading through a social network, accounting for both the immunization and disease costs. Previous research has typically focused on determining the optimal strategy to limit the lifetime of a disease, without considering the cost of such strategies. In the large graph limit, we calculate the exact expected disease cost for a general random graph, and we illustrate it for the specific example of an ErdösRényi network. We also give an upper bound on the expected disease cost for finitesize graphs, and show through simulation that the upper bound is tight for ErdösRényi networks and graphs with exponential degree distributions. Finally, we study how to optimally perform a oneshot immunization to minimize the social cost of a disease, including both the cost of the disease and the cost of immunization.https://authors.library.caltech.edu/records/4tkj3wjj20Recovery of sparse 1D signals from the magnitudes of their Fourier transform
https://resolver.caltech.edu/CaltechAUTHORS:20130204114142719
Authors: Jaganathan, Kishore; Oymak, Samet; Hassibi, Babak
Year: 2012
DOI: 10.1109/ISIT.2012.6283508
The problem of signal recovery from the autocorrelation, or equivalently, the magnitudes of the Fourier transform, is of paramount importance in various fields of engineering. In this work, for onedimensional signals, we give conditions, which when satisfied, allow unique recovery from the autocorrelation with very high probability. In particular, for sparse signals, we develop two noniterative recovery algorithms. One of them is based on combinatorial analysis, which we prove can recover signals up to sparsity o(n^(1/3)) with very high probability, and the other is developed using a convex optimization based framework, which numerical simulations suggest can recover signals upto sparsity o(n^(1/2)) with very high probability.https://authors.library.caltech.edu/records/68asndqv24Recovery Threshold for Optimal Weight ℓ_1 Minimization
https://resolver.caltech.edu/CaltechAUTHORS:20120828151151611
Authors: Oymak, Samet; Khajehnejad, M. Amin; Hassibi, Babak
Year: 2012
DOI: 10.1109/ISIT.2012.6283717
We consider the problem of recovering a sparse signal from underdetermined measurements when we have prior information about the sparsity structure of the signal. In particular, we assume that the entries of the signal can be partitioned into two known sets S_1, S_2 where the relative sparsities over the two sets are different. In this situation it is advantageous to replace classical ℓ_1 minimization with weighted ℓ_1 minimization, where the sparser set is given a larger weight. In this paper we give a simple closed form expression for the minimum number of measurements required for successful recovery when the optimal weights are chosen. The formula shows that the number of measurements is upper bounded by the sum of the minimum number of measurements needed had we measured the S_1 and S_2 components of the signal separately. In fact, our results indicate that this upper bound is tight and we actually have equality. Our proof technique uses the "escape through a mesh" framework and connects to the Minimax MSE of a certain basis pursuit denisoing problem.https://authors.library.caltech.edu/records/aqwepkm077On a Relation between the Minimax Risk and the Phase Transitions of Compressed Recovery
https://resolver.caltech.edu/CaltechAUTHORS:20130731085137179
Authors: Oymak, Samet; Hassibi, Babak
Year: 2012
DOI: 10.1109/Allerton.2012.6483330
This paper provides a sharp analysis of the optimally tuned denoising problem and establishes a relation between the estimation error (minimax risk) and phase transition for compressed sensing recovery using convex and continuous functions. Phase transitions deal with recovering a signal xo from compressed linear observations Ax_0 by minimizing a certain convex function f(·). On the other hand, denoising is the problem of estimating a signal x_0 from noisy observations y = x_0+z using the regularization min_x λ/f(x) + 1/2∥yx∥_2^2. In general, these problems are more meaningful and useful when the signal x_0 has a certain structure and the function f(·) is chosen to exploit this structure. Examples include, l_1 and l_1  l_2 norms for sparse and block sparse vectors and nuclear norm for low rank matrices. In this work, we carefully analyze the minimax denoising problem and relate our results to the phase transition performance under a considerably general setting where the measurement A in compressed recovery and the noise z in the denoising problem are iid Gaussian random variables. Our results suggest that the required number of observations to recover a compressed signal is closely related to the asymptotic variance of the optimal estimation error. This relation was first empirically noted in [9]. Here we provide a rigorous foundation.https://authors.library.caltech.edu/records/5fkkpx8q23On Robust Phase Retrieval for Sparse Signals
https://resolver.caltech.edu/CaltechAUTHORS:20130730135255859
Authors: Jaganathan, Kishore; Oymak, Samet; Hassibi, Babak
Year: 2012
DOI: 10.1109/Allerton.2012.6483299
Recovering signals from their Fourier transform magnitudes is a classical problem referred to as phase retrieval and has been around for decades. In general, the Fourier transform magnitudes do not carry enough information to uniquely identify the signal and therefore additional prior information is required. In this paper, we shall assume that the underlying signal is sparse, which is true in many applications such as Xray crystallography, astronomical imaging, etc. Recently, several techniques involving semidefinite relaxations have been proposed for this problem, however very little analysis has been performed. The phase retrieval problem can be decomposed into two tasks  (i) identifying the support of the sparse signal from the Fourier transform magnitudes, and (ii) recovering the signal using the support information. In earlier work [13], we developed algorithms for (i) which provably recovered the support for sparsities upto O(n^(1/3ϵ)). Simulations suggest that support recovery is possible upto sparsity O(n^(1/2ϵ)). In this paper, we focus on (ii) and propose an algorithm based on semidefinite relaxation, which provably recovers the signal from its Fourier transform magnitude and support knowledge with high probability if the support size is O(n^(1/2ϵ)).https://authors.library.caltech.edu/records/075skq8p10Frames, Group Codes, and Subgroups of (Z/pZ)×
https://resolver.caltech.edu/CaltechAUTHORS:20130730143016951
Authors: Thill, Matthew; Hassibi, Babak
Year: 2012
DOI: 10.1109/Allerton.2012.6483352
The problem of designing low coherence matrices and lowcorrelation frames arises in a variety of fields, including compressed sensing, MIMO communications and quantum measurements. The challenge is that one must control the (^n_2) pairwise inner products of the columns of the matrix. In this paper, we follow the group code approach of David Slepian [1], which constructs frames using unitary group representations and which in general reduces the number of distinct inner products to n1. When n is a prime p, we present a carefully chosen representation which reduces the number of distinct inner products further to ^(n1)/_m, where m is the number of rows in the matrix. The resulting frames have superior performance to many earlier frame constructions and, in some cases, yield frames with optimally low coherence. We further expand a connection between frames and difference sets noted first in [2] to find bounds on the coherence when ^(n1)/_m = 2 and 3.https://authors.library.caltech.edu/records/d3hxv8bf35Tree codes improve convergence rate of consensus over erasure channels
https://resolver.caltech.edu/CaltechAUTHORS:20131220082835850
Authors: Sukhavasi, Ravi Teja; Hassibi, Babak
Year: 2012
DOI: 10.1109/CDC.2012.6425840
We study the problem of achieving average consensus between a group of agents over a network with erasure links. In the context of consensus problems, the unreliability of communication links between nodes has been traditionally modeled by allowing the underlying graph to vary with time. In other words, depending on the realization of the link erasures, the underlying graph at each time instant is assumed to be a subgraph of the original graph. Implicit in this model is the assumption that the erasures are symmetric: if at time t the packet from node i to node j is dropped, the same is true for the packet transmitted from node j to node i. However, in practical wireless communication systems this assumption is unreasonable and, due to the lack of symmetry, standard averaging protocols cannot guarantee that the network will reach consensus to the true average. In this paper we explore the use of channel coding to improve the performance of consensus algorithms. For symmetric erasures, we show that, for certain ranges of the system parameters, repetition codes can speed up the convergence rate. For asymmetric erasures we show that tree codes (which have recently been designed for erasure channels) can be used to simulate the performance of the original "unerased" graph. Thus, unlike conventional consensus methods, we can guarantee convergence to the average in the asymmetric case. The price is a slowdown in the convergence rate, relative to the unerased network, which is still often faster than the convergence rate of conventional consensus algorithms over noisy links.https://authors.library.caltech.edu/records/qygy33hf98Sparse Phase Retrieval: Convex Algorithms and Limitations
https://resolver.caltech.edu/CaltechAUTHORS:20150126071647266
Authors: Jaganathan, Kishore; Oymak, Samet; Hassibi, Babak
Year: 2013
DOI: 10.1109/ISIT.2013.6620381
We consider the problem of recovering signals from
their power spectral densities. This is a classical problem referred to in literature as the phase retrieval problem, and is of paramount importance in many fields of applied sciences. In general, additional prior information about the signal is required to guarantee unique recovery as the mapping from signals to power spectral densities is not onetoone. In this work, we assume that the underlying signals are sparse.
Recently, semidefinite programming (SDP) based approaches
were explored by various researchers. Simulations of these
algorithms strongly suggest that signals upto O(n^(1/2−ϵ)
sparsity can be recovered by this technique. In this work, we develop a tractable algorithm based on reweighted
ℓ_1minimization that recovers a sparse signal from its power spectral density for significantly higher sparsities, which is unprecedented. We also discuss the limitations of the existing SDP algorithms and provide a combinatorial algorithm which requires significantly fewer "phaseless" measurements to guarantee recovery.https://authors.library.caltech.edu/records/56epxfy851On the capacity of a communication system with energy harvesting and a limited battery
https://resolver.caltech.edu/CaltechAUTHORS:20150224072121144
Authors: Mao, Wei; Hassibi, Babak
Year: 2013
DOI: 10.1109/ISIT.2013.6620535
We consider the problem of determining the capacity of an energyharvesting transmitter with finite battery communicating over a discrete memoryless channel. When the battery is unlimited, or zero, the capacity has been determined, but it remains unknown for a finite nonzero battery. In this paper we assume that the harvested energy at each time, the total battery storage, and the transmitter signal energy at each time can be quantized to the same unit (i.e., the same energy interval). Under this assumption, we show that the capacity can be described using the VerdúHan general framework. If we further assume that the transmitted symbol at each time depends only on the energy currently available, and not on the entire past history of energy harvests and symbols transmitted, then we show that the system reduces to a finite state channel (FSC) with the required ergodic and Markov properties so that lower bounds on the capacity can be readily numerically computed. We conjecture that our numerical bounds are tight. Our numerical results indicate that even the minimal possible battery storage can reap a significant fraction of the infinite battery capacity.https://authors.library.caltech.edu/records/bhbqdtnh31Noisy estimation of simultaneously structured models: Limitations of convex relaxation
https://resolver.caltech.edu/CaltechAUTHORS:20150224071400977
Authors: Oymak, Samet; Jalali, Amin; Fazel, Maryam; Hassibi, Babak
Year: 2013
DOI: 10.1109/CDC.2013.6760840
Models or signals exhibiting low dimensional behavior (e.g., sparse signals, low rank matrices) play an important role in signal processing and system identification. In this paper, we focus on models that have multiple structures simultaneously; e.g., matrices that are both low rank and sparse, arising in phase retrieval, quadratic compressed sensing, and cluster detection in social networks. We consider the estimation of such models from observations corrupted by additive Gaussian noise. We provide tight upper and lower bounds on the mean squared error (MSE) of a convex denoising program that uses a combination of regularizers to induce multiple structures. In the case of low rank and sparse matrices, we quantify the gap between the MSE of the convex program and the best achievable error, and we present a simple (nonconvex) thresholding algorithm that outperforms its convex counterpart and achieves almost optimal MSE.
This paper extends prior work on a different but related problem: recovering simultaneously structured models from noiseless compressed measurements, where bounds on the number of required measurements were given. The present work shows a similar fundamental limitation exists in a statistical denoising setting.https://authors.library.caltech.edu/records/0zsc5dqs22Global dynamics of epidemic spread over complex networks
https://resolver.caltech.edu/CaltechAUTHORS:20150227070127397
Authors: Ahn, Hyoung Jun; Hassibi, Babak
Year: 2013
DOI: 10.1109/CDC.2013.6760600
In this paper we study the global dynamics of epidemic spread over complex networks for both discretetime and continuoustime models. In this setting, the state of the system at any given time is the vector obtained from the marginal probability of infection of each of the nodes in the network at that time. Convergence to the origin means that the epidemic eventually dies out. Linearizing the model around the origin yields a system whose state is an upper bound on the true state. As a result, whenever the linearized model is locally stable, the original model is globally stable, with the origin being its fixed point. When the linearized model is unstable the origin is not a stable fixed point and we show the existence of a unique second fixed point. In the continuoustime model, this second fixed point attracts all points in the state space other than the origin. In the discretetime setting we consider two models. In the first model, we show that the second fixed point attracts all points in the state space other than the origin. In the second model, however, we show this need not be the case. We therefore give conditions under which the second fixed point attracts all nonorigin points and show that for random ErdösRényi graphs this happens with high probability.https://authors.library.caltech.edu/records/8w9nw2gq50Reconstruction of integers from pairwise distances
https://resolver.caltech.edu/CaltechAUTHORS:20140221114925124
Authors: Jaganathan, Kishore; Hassibi, Babak
Year: 2013
DOI: 10.1109/ICASSP.2013.6638811
Given a set of integers, one can easily construct the set of their pairwise distances. We consider the inverse problem: given a set of pairwise distances, find the integer set which realizes the pairwise distance set. This problem arises in a lot of fields in engineering and applied physics, and has confounded researchers for over 60 years. It is one of the few fundamental problems that are neither known to be NPhard nor solvable by polynomialtime algorithms. Whether unique recovery is possible also remains an open question. In many practical applications where this problem occurs, the integer set is naturally sparse (i.e., the integers are sufficiently spaced), a property which has not been explored. In this work, we exploit the sparse nature of the integer set and develop a polynomialtime algorithm which provably recovers the set of integers (up to linear shift and reversal) from the set of their pairwise distances with arbitrarily high probability if the sparsity is O(n^(1/2ε)). Numerical simulations verify the effectiveness of the proposed algorithm.https://authors.library.caltech.edu/records/74wwsgpe81Frames From Groups: Generalized Bounds And Dihedral Groups
https://resolver.caltech.edu/CaltechAUTHORS:20140310084438643
Authors: Thill, Matthew; Hassibi, Babak
Year: 2013
DOI: 10.1109/ICASSP.2013.6638825
The problem of designing low coherence matrices and lowcorrelation
frames arises in a variety of fields, including compressed
sensing, MIMO communications and quantum measurements.
The challenge is that one must control the
n/2 pairwise inner products of the columns of the matrix. In this
paper, we follow the group code approach of David Slepian, which constructs frames using unitary group representations
and which in general reduces the number of distinct
inner products to n1. We examine representations of cyclic
groups as well as generalized dihedral groups, and we expand
upon previous results which bound the coherence of the resulting
frames.https://authors.library.caltech.edu/records/t23ffa5k32Optimal Largescale Storage Placement in Single Generator Single Load Networks
https://resolver.caltech.edu/CaltechAUTHORS:20140410095815023
Authors: Thrampoulidis, Christos; Bose, Subhonmesh; Hassibi, Babak
Year: 2013
DOI: 10.1109/PESMG.2013.6672589
Largescale storage will play an increasingly important role in future power grids. As a result, how to optimally place storage in such networks, is an important investment problem. Furthermore, since the allocation of storage resources is static, i.e., it is not feasible to move storage around in a dynamic fashion, it is important to derive optimal such allocations that are robust to the values of the load profiles and other network parameters, such as the line flow constraints. For a single generator single load network, and for a cost of generation that is quadratic in the generation power, we show that, for any given amount of storage resources, placing storage at the demand node is always optimal. This result is true regardless of the demand profile and flow constraints, and therefore is robust. As a byproduct of this result, for a fixed demand profile, we characterize the dependence of the optimal production cost on the flow constraints and on the available storage resources.https://authors.library.caltech.edu/records/d2cwykk883On frames from abelian group codes
https://resolver.caltech.edu/CaltechAUTHORS:20150224072315309
Authors: Thill, Matthew; Hassibi, Babak
Year: 2013
DOI: 10.1109/ISIT.2013.6620273
Designing low coherence matrices and lowcorrelation frames is a point of interest in many fields including compressed sensing, MIMO communications and quantum measurements. The challenge is that one must control the (^n_2) pairwise inner products between the frame elements. In this paper, we exploit the group code approach of David Slepian [1], which constructs frames using unitary group representations and which in general reduces the number of distinct inner products to n  1. We demonstrate how to efficiently find optimal representations of cyclic groups, and we show how basic abelian groups can be used to construct tight frames that have the same dimensions and inner products as those arising from certain more complex nonabelian groups. We support our work with theoretical bounds and simulations.https://authors.library.caltech.edu/records/gjpdbq5e35The squarederror of generalized LASSO: A precise analysis
https://resolver.caltech.edu/CaltechAUTHORS:20170201142815906
Authors: Oymak, Samet; Thrampoulidis, Christos; Hassibi, Babak
Year: 2013
DOI: 10.1109/Allerton.2013.6736635
We consider the problem of estimating an unknown but structured signal x0 from its noisy linear observations y = Ax_0 + z ∈ ℝ^m. To the structure of x0 is associated a structure inducing convex function f(·). We assume that the entries of A are i.i.d. standard normal N(0, 1) and z ~ N(0, σ^2I_m). As a measure of performance of an estimate x^* of x_0 we consider the "Normalized Square Error" (NSE) ∥x*  x0∥^2_2/σ^2. For sufficiently small σ, we characterize the exact performance of two different versions of the well known LASSO algorithm. The first estimator is obtained by solving the problem arg min_x ∥y  Ax∥_2 + λf(x). As a function of λ, we identify three distinct regions of operation. Out of them, we argue that "RON" is the most interesting one. When λ ∈ R_(ON), we show that the NSE is (D_f(x_0, λ))/(mD_f(x_0, λ)) for small σ, where D_f(x_0, λ) is the expected squareddistance of an i.i.d. standard normal vector to the dilated subdifferential λ · ∂f(x_0). Secondly, we consider the more popular estimator arg min_x 1/2∥y  Ax∥^2_2 + στ f(x). We propose a formula for the NSE of this estimator by establishing a suitable mapping between this and the previous estimator over the region RON. As a useful side result, we find explicit formulae for the optimal estimation performance and the optimal penalty parameters λ* and τ*.https://authors.library.caltech.edu/records/hzxd4p3r30On the distribution of energy storage in electricity grids
https://resolver.caltech.edu/CaltechAUTHORS:20150224071641038
Authors: Thrampoulidis, Christos; Bose, Subhonmesh; Hassibi, Babak
Year: 2013
DOI: 10.1109/CDC.2013.6761094
Distributed energy storage is a promising emerging technology for smart grid. In this paper, we address the question of optimally placing and sizing distributed storage resources in a network to minimize the cost of generation given a budget of available storage. For a nondecreasing convex generation cost, we prove that it is always optimal to place zero storage at generator buses that connect to rest of the grid via single links, regardless of demand profiles and network parameters. Hence, this defines a robust investment strategy for network planners. Besides, for a star network where the center is a generator bus and the other nodes are demand buses, we show that it is optimal not to place storage resources at the generator bus for small enough and large enough storage budget.https://authors.library.caltech.edu/records/d0ekxs4y27Simple Error Bounds for Regularized Noisy Linear Inverse Problems
https://resolver.caltech.edu/CaltechAUTHORS:20150121071420813
Authors: Thrampoulidis, Christos; Oymak, Samet; Hassibi, Babak
Year: 2014
DOI: 10.1109/ISIT.2014.6875386
Consider estimating a structured signal x_0 from linear, underdetermined and noisy measurements y = Ax_0+z, via solving a variant of the lasso algorithm: x̂ = arg min_x{∥yAx∥+2 + λf(x)}. Here, f is a convex function aiming to promote the structure of x_0, say ℓ_1norm to promote sparsity or nuclear norm to promote lowrankness. We assume that the entries of A are independent and normally distributed and make no assumptions on the noise vector z, other than it being independent of A. Under this generic setup, we derive a general, nonasymptotic and rather tight upper bound on the ℓ_2norm of the estimation error ∥x̂  x0∥2. Our bound is geometric in nature and obeys a simple formula; the roles of λ, f and x_0 are all captured by a single summary parameter δ(λ∂f(x_0)), termed the Gaussian squared distance to the scaled subdifferential. We connect our result to the literature and verify its validity through simulations.https://authors.library.caltech.edu/records/a8599rfd55Graph Clustering With Missing Data: Convex Algorithms and Analysis
https://resolver.caltech.edu/CaltechAUTHORS:20160401170949281
Authors: Korlakai Vinayak, Ramya; Oymak, Samet; Hassibi, Babak
Year: 2014
We consider the problem of finding clusters in an unweighted graph, when the graph is partially observed. We analyze two programs, one which works for dense
graphs and one which works for both sparse and dense graphs, but requires some a priori knowledge of the total cluster size, that are based on the convex optimization
approach for lowrank matrix recovery using nuclear norm minimization. For the commonly used Stochastic Block Model, we obtain explicit bounds on the
parameters of the problem (size and sparsity of clusters, the amount of observed data) and the regularization parameter characterize the success and failure of the
programs. We corroborate our theoretical findings through extensive simulations. We also run our algorithm on a real data set obtained from crowdsourcing an
image classification task on the Amazon Mechanical Turk, and observe significant performance improvement over traditional methods such as kmeans.https://authors.library.caltech.edu/records/84pna0ye54A case for orthogonal measurements in linear inverse problems
https://resolver.caltech.edu/CaltechAUTHORS:20150224071049082
Authors: Oymak, Samet; Hassibi, Babak
Year: 2014
DOI: 10.1109/ISIT.2014.6875420
We investigate the random matrices that have orthonormal rows and provide a comparison to matrices with independent Gaussian entries. We find that, orthonormality provides an inherent advantage for the conditioning. In particular, for any given subset S of ℝ^n, we show that orthonormal matrices have better restricted eigenvalues compared to Gaussians. We consider implications of this result for the linear inverse problems; in particular, we investigate the noisy sparse estimation setup and applications to restricted isometry property. We relate our findings to the results known for Gaussian processes and precise undersampling theorems. We then discuss and illustrate universality of the noise robustness behavior for partial unitary matrices including Hadamard and Discrete Cosine Transform.https://authors.library.caltech.edu/records/pj9bvdsa88Sharp performance bounds for graph clustering via convex optimization
https://resolver.caltech.edu/CaltechAUTHORS:20150106133535671
Authors: Korlakai Vinayak, Ramya; Oymak, Samet; Hassibi, Babak
Year: 2014
DOI: 10.1109/ICASSP.2014.6855219
The problem of finding clusters in a graph arises in several applications such as social networks, data mining and computer networks. A typical, convex optimizationapproach, that is often adopted is to identify a sparse plus lowrank decomposition of the adjacency matrix of the graph, with the (dense) lowrank component representing the clusters. In this paper, we sharply characterize the conditions for successfully identifying clusters using this approach. In particular, we introduce the "effective density" of a cluster that measures its significance and we find explicit upper and lower bounds on the minimum effective density that demarcates regions of success or failure of this technique. Our conditions are in terms of (a) the size of the clusters, (b) the denseness of the graph, and (c) regularization parameter of the convex program. We also present extensive simulations that corroborate our theoretical findings.https://authors.library.caltech.edu/records/axzsvcvn69Frames from generalized group fourier transforms and SL_2(F_q)
https://resolver.caltech.edu/CaltechAUTHORS:20150106132022513
Authors: Thill, Matthew; Muthukumar, Vidya; Hassibi, Babak
Year: 2014
DOI: 10.1109/ICASSP.2014.6854538
We explore the problem of deterministically constructing frames and matrices with low coherence, which arises in areas such as compressive sensing, spherical codes, and MIMO communications. In particular, we present a generalization of the familiar harmonic frame by selecting a subset of rows of the generalized discrete Fourier transform matrix over finite groups. We apply our methods to the group SL_2(F_q) and show how to produce frames with remarkably low coherence, for which we provide upper bounds.https://authors.library.caltech.edu/records/7pwsv2v194Estimating structured signals in sparse noise: A precise noise sensitivity analysis
https://resolver.caltech.edu/CaltechAUTHORS:20150203110637079
Authors: Thrampoulidis, Christos; Hassibi, Babak
Year: 2014
DOI: 10.1109/ALLERTON.2014.7028545
We consider the problem of estimating a structured signal x_0 from linear, underdetermined and noisy measurements y = Ax_0 + z, in the presence of sparse noise z. A natural approach to recovering x_0, that takes advantage of both the structure of xo and the sparsity of z is solving: x = arg min_x y − Ax1 subject to f(x) ≤ f(x_0) (constrained LAD estimator). Here, f is a convex function aiming to promote the structure of x_0, say ℓ_1norm to promote sparsity or nuclear norm to promote lowrankness. We assume that the entries of A and the nonzero entries of z are i.i.d normal with variances 1 and σ^2, respectively. Our analysis precisely characterizes the asymptotic noise sensitivity x – x_0^2_2/σ^2 in the limit σ^2 → 0. We show analytically that the LAD method outperforms the more popular LASSO method when the noise is sparse. At the same time its performance is no more than π/2 times worse in the presence of nonsparse noise. Our simulation results verify the validity of our theoretical predictions.https://authors.library.caltech.edu/records/nk2ndjyf98Capacity bounds for certain channels with states and the energy harvesting channel
https://resolver.caltech.edu/CaltechAUTHORS:20190212103510045
Authors: Mao, Wei; Hassibi, Babak
Year: 2014
DOI: 10.1109/ITW.2014.6970833
We study two types of channels in this paper: certain channels with states and energy harvesting channels. The first is based on Gallager's finite state channel with input constraints and causal CSIT. The second deals with two different scenarios with regard to the availability of energy information at the transmitter. For both channels we derive new capacity bounds, mostly using bounding techniques of Verdú and Han and Gallager.https://authors.library.caltech.edu/records/4wd5f28v90Recovering Structured Signals in Noise: LeastSquares Meets Compressed Sensing
https://resolver.caltech.edu/CaltechAUTHORS:20160901121710649
Authors: Thrampoulidis, Christos; Oymak, Samet; Hassibi, Babak
Year: 2015
DOI: 10.1007/9783319160429_4
The typical scenario that arises in most "big data" problems is one where the ambient dimension of the signal is very large (e.g., high resolution images, gene expression data from a DNA microarray, social network data, etc.), yet is such that its desired properties lie in some low dimensional structure (sparsity, lowrankness, clusters, etc.). In the modern viewpoint, the goal is to come up with efficient algorithms to reveal these structures and for which, under suitable conditions, one can give theoretical guarantees. We specifically consider the problem of recovering such a structured signal (sparse, lowrank, blocksparse, etc.) from noisy compressed measurements. A general algorithm for such problems, commonly referred to as generalized LASSO, attempts to solve this problem by minimizing a leastsquares cost with an added "structureinducing" regularization term (ℓ_ 1 norm, nuclear norm, mixed ℓ _2/ℓ_ 1 norm, etc.). While the LASSO algorithm has been around for 20 years and has enjoyed great success in practice, there has been relatively little analysis of its performance. In this chapter, we will provide a full performance analysis and compute, in closed form, the meansquareerror of the reconstructed signal. We will highlight some of the mathematical vignettes necessary for the analysis, make connections to noiseless compressed sensing and proximal denoising, and will emphasize the central role of the "statistical dimension" of a structured signal.https://authors.library.caltech.edu/records/cqrjkbxj69The proportional mean decomposition: A bridge between the Gaussian and Bernoulli ensembles
https://resolver.caltech.edu/CaltechAUTHORS:20160722153039699
Authors: Oymak, Samet; Hassibi, Babak
Year: 2015
DOI: 10.1109/ICASSP.2015.7178586
We consider illposed linear inverse problems involving the estimation of structured sparse signals. When the sensing matrix has i.i.d. standard normal entries, there is a fullfledged theory on the sample complexity and robustness properties. In this work, we propose a way of making use of this theory to get good bounds for the i.i.d. Bernoulli ensemble. We first provide a deterministic relation between the two ensembles that relates the restricted singular values. Then, we show how one can get nonasymptotic results with small constants for the Bernoulli ensemble. While our discussion focuses on Bernoulli measurements, the main idea can be extended to any discrete distribution with little difficulty.https://authors.library.caltech.edu/records/vk2ps0z687Recovering signals from the ShortTime Fourier Transform magnitude
https://resolver.caltech.edu/CaltechAUTHORS:20160722144524587
Authors: Jaganathan, Kishore; Eldar, Yonina C.; Hassibi, Babak
Year: 2015
DOI: 10.1109/ICASSP.2015.7178577
The problem of recovering signals from the ShortTime Fourier Transform (STFT) magnitude is of paramount importance in many areas of engineering and physics. This problem has received a lot of attention over the last few decades, but not much is known about conditions under which the STFT magnitude is a unique signal representation. Also, the recovery techniques proposed by researchers are mostly heuristic in nature. In this work, we first show that almost all signals can be uniquely identified by their STFT magnitude under mild conditions. Then, we consider a semidefinite relaxationbased algorithm and provide the first theoretical guarantees for the same. Numerical simulations complement our theoretical analysis and provide many directions for future work.https://authors.library.caltech.edu/records/zs8h6v6r92Precise error analysis of the LASSO
https://resolver.caltech.edu/CaltechAUTHORS:20160722161318049
Authors: Thrampoulidis, Christos; Panahi, Ashkan; Guo, Daniel; Hassibi, Babak
Year: 2015
DOI: 10.1109/ICASSP.2015.7178615
A classical problem that arises in numerous signal processing applications asks for the reconstruction of an unknown, ksparse signal x_0 ∈ ℝ^n from underdetermined, noisy, linear measurements y = Ax_0 + z ∈ ℝ^m. One standard approach is to solve the following convex program x̂ = arg min_x ∥y  Ax∥_2+λ∥x∥_1, which is known as the ℓ_2LASSO. We assume that the entries of the sensing matrix A and of the noise vector z are i.i.d Gaussian with variances 1/m and σ^2. In the large system limit when the problem dimensions grow to infinity, but in constant rates, we precisely characterize the limiting behavior of the normalized squared error ∥x̂  x_0∥_2^2/σ^2. Our numerical illustrations validate our theoretical predictions.https://authors.library.caltech.edu/records/j3cxyy2z09A numerical implementation of gridless compressed sensing
https://resolver.caltech.edu/CaltechAUTHORS:20160722153514271
Authors: Panahi, Ashkan; Viberg, Mats; Hassibi, Babak
Year: 2015
DOI: 10.1109/ICASSP.2015.7178590
Atomic norm denoising has been recently introduced as a generalization of the Least Absolute Shrinkage and Selection Operator (LASSO) to overcome the problem of offgrid parameters. The method has been found to possess many interesting theoretical properties. However, its implementation has been only discussed in a special case of spectral line estimation by uniform sampling. In this paper, we propose a general numerical method to solve the atomic norm denoising problem. The complexity of the proposed algorithm is proportional to the complexity of a singleparameter search in the parameter space and thus in many interesting cases, including frequency estimation it enjoys fast realization.https://authors.library.caltech.edu/records/q03ghjvn08Phase retrieval with masks using convex optimization
https://resolver.caltech.edu/CaltechAUTHORS:20151007102301183
Authors: Jaganathan, Kishore; Eldar, Yonina; Hassibi, Babak
Year: 2015
DOI: 10.1109/ISIT.2015.7282737
Signal recovery from the magnitude of the Fourier transform, or equivalently, from the autocorrelation, is a classical problem known as phase retrieval. Due to the absence of phase information, some form of additional information is required in order to be able to uniquely identify the underlying signal. In this work, we consider the problem of phase retrieval using masks. Due to our interest in developing robust algorithms with theoretical guarantees, we explore a convex optimizationbased framework. In this work, we show that two specific masks (each mask provides 2n Fourier magnitude measurements) or five specific masks (each mask provides n Fourier magnitude measurements) are sufficient for a convex relaxation of the phase retrieval problem to provably recover almost all signals (up to global phase). We also show that the recovery is stable in the presence of measurement noise. This is a significant improvement over the existing results, which require O(log^2 n) random masks (each mask provides n Fourier magnitude measurements) in order to guarantee unique recovery (up to global phase). Numerical experiments complement our theoretical analysis and show interesting trends, which we hope to explain in a future publication.https://authors.library.caltech.edu/records/6n5mqe1e21New capacity upper bounds and coding aspects for some channels with causal CSIT
https://resolver.caltech.edu/CaltechAUTHORS:20151007101738903
Authors: Mao, Wei; Hassibi, Babak
Year: 2015
DOI: 10.1109/ISIT.2015.7282430
We study two channels with causal CSIT: a finite state channel with input constraints and a finitebattery energy harvesting channel, considered in [1] and for the latter [2]–[5]. The capacity of these channels remains open and the calculation of the upper bounds often has a complexity double exponential in the block size N. In this paper we obtain an alternative upper bound which has a complexity linear in N. While, for any N, this bound is looser than the bound in [1], since it can be readily computed for very large values of N, it leads to numerically tighter bounds in many cases. Furthermore, for the energy harvesting channel we calculate the pairwise error probabilities of the ML decoder, which provides a useful guideline for the code design.https://authors.library.caltech.edu/records/ngphmvgc84Isotropically random orthogonal matrices: Performance of LASSO and minimum conic singular values
https://resolver.caltech.edu/CaltechAUTHORS:20151006110604022
Authors: Thrampoulidis, Christos; Hassibi, Babak
Year: 2015
DOI: 10.1109/ISIT.2015.7282516
Recently, the precise performance of the Generalized LASSO algorithm for recovering structured signals from compressed noisy measurements, obtained via i.i.d. Gaussian matrices, has been characterized. The analysis is based on a framework introduced by Stojnic and heavily relies on the use of Gordon's Gaussian minmax theorem (GMT), a comparison principle on Gaussian processes. As a result, corresponding characterizations for other ensembles of measurement matrices have not been developed. In this work, we analyze the corresponding performance of the ensemble of isotropically random orthogonal (i.r.o.) measurements. We consider the constrained version of the Generalized LASSO and derive a sharp characterization of its normalized squared error in the largesystem limit. When compared to its Gaussian counterpart, our result analytically confirms the superiority in performance of the i.r.o. ensemble. Our second result, derives an asymptotic lower bound on the minimum conic singular values of i.r.o. matrices. This bound is larger than the corresponding bound on Gaussian matrices. To prove our results we express i.r.o. matrices in terms of Gaussians and show that, with some modifications, the GMT framework is still applicable.https://authors.library.caltech.edu/records/4kkbx2j980Coding with Constraints: Minimum Distance Bounds and Systematic Constructions
https://resolver.caltech.edu/CaltechAUTHORS:20150224070606037
Authors: Halbawi, Wael; Thill, Matthew; Hassibi, Babak
Year: 2015
DOI: 10.1109/ISIT.2015.7282666
We examine an errorcorrecting coding framework in which each coded symbol is constrained to be a function of a fixed subset of the message symbols. With an eye toward distributed storage applications, we seek to design systematic codes with good minimum distance that can be decoded efficiently. On this note, we provide theoretical bounds on the minimum distance of such a code based on the coded symbol constraints. We refine these bounds in the case where we demand a systematic linear code. Finally, we provide conditions under which each of these bounds can be achieved by choosing our code to be a subcode of a ReedSolomon code, allowing for efficient decoding. This problem has been considered in multisource multicast network error correction. The problem setup is also reminiscent of locally repairable codes.https://authors.library.caltech.edu/records/9s5egrw425Beyond semidefinite relaxation: Basis banks and computationally enhanced guarantees
https://resolver.caltech.edu/CaltechAUTHORS:20151007110124683
Authors: Soltanalian, Mojtaba; Hassibi, Babak
Year: 2015
DOI: 10.1109/ISIT.2015.7282472
As a widely used tool in tackling general quadratic optimization problems, semidefinite relaxation (SDR) promises both a polynomialtime complexity and an a priori known suboptimality guarantee for its approximate solutions. While attempts at improving the guarantees of SDR in a general sense have proven largely unsuccessful, it has been widely observed that the quality of solutions obtained by SDR is usually considerably better than the provided guarantees. In this paper, we propose a novel methodology that paves the way for obtaining improved datadependent guarantees in a computational way. The derivations are dedicated to a specific quadratic optimization problem (called mQP) which lies at the core of many communication and active sensing schemes; however, the ideas may be generalized to other quadratic optimization problems. The new guarantees are particularly useful in accuracy sensitive applications, including decisionmaking scenarios.https://authors.library.caltech.edu/records/qkwp8qzj87Asymptotically Exact Error Analysis for the Generalized ℓ^2_2LASSO
https://resolver.caltech.edu/CaltechAUTHORS:20150227070457225
Authors: Thrampoulidis, Christos; Panahi, Ashkan; Hassibi, Babak
Year: 2015
DOI: 10.1109/ISIT.2015.7282810
Given an unknown signal x_0∈R^n and linear noisy measurements y=Ax_0 + σv ∈ ℝ^m, the generalized ℓ^2_2LASSO solves x^:=arg min_x 1/2∥y−Ax∥^2_2 + σλf(x). Here, f is a convex regularization function (e.g. ℓ_1norm, nuclearnorm) aiming to promote the structure of x_0 (e.g. sparse, lowrank), and, λ ≥ 0 is the regularizer parameter. A related optimization problem, though not as popular or wellknown, is often referred to as the generalized ℓ_2LASSO and takes the form x^ := arg min_x ∥y−Ax∥_2 + λf(x), and has been analyzed in [1]. [1] further made conjectures about the performance of the generalized ℓ^2_2LASSO. This paper establishes these conjectures rigorously. We measure performance with the normalized squared error NSE(σ) := ∥x^−x_0∥^2_2/σ^2. Assuming the entries of A and v be i.i.d. standard normal, we precisely characterize the "asymptotic NSE" aNSE := lim_(σ→0)NSE(σ) when the problem dimensions m,n tend to infinity in a proportional manner. The role of λ,f and x_0 is explicitly captured in the derived expression via means of a single geometric quantity, the Gaussian distance to the subdifferential. We conjecture that aNSE=sup_(σ>0)NSE(σ). We include detailed discussions on the interpretation of our result, make connections to relevant literature and perform computational experiments that validate our theoretical findings.https://authors.library.caltech.edu/records/s0qvssp419Precise highdimensional error analysis of regularized Mestimators
https://resolver.caltech.edu/CaltechAUTHORS:20160412092555507
Authors: Thrampoulidis, Christos; Abbasi, Ehsan; Hassibi, Babak
Year: 2015
DOI: 10.1109/ALLERTON.2015.7447033
A general approach for estimating an unknown
signal x_0 ∈ R^n from noisy, linear measurements
y = Ax_0 + z ∈ R^m is via solving a so called regularized
Mestimator: x := arg min_x L(yAx) + λf(x). Here, L is a convex loss function, f is a convex (typically, nonsmooth)
regularizer, and, λ > 0 a regularizer parameter.
We analyze the squared error performance xx_0^2_2
of such estimators in the highdimensional proportional
regime where m, n → ∞ and m/n → δ. We let the design
matrix A have entries iid Gaussian, and, impose minimal
and rather mild regularity conditions on the loss function,
on the regularizer, and, on the distributions of the noise
and of the unknown signal. Under such a generic setting,
we show that the squared error converges in probability
to a nontrivial limit that is computed by solving four
nonlinear equations on four scalar unknowns. We identify
a new summary parameter, termed the expected
Moreau envelope, which determines how the choice of
the loss function and of the regularizer affects the error
performance. The result opens the way for answering
optimality questions regarding the choice of the loss
function, the regularizer, the penalty parameter, etc.https://authors.library.caltech.edu/records/8fan5q7c21SIRS epidemics on complex networks: Concurrence of exact Markov chain and approximated models
https://resolver.caltech.edu/CaltechAUTHORS:20160216133200228
Authors: Azizan Ruhi, Navid; Hassibi, Babak
Year: 2015
DOI: 10.1109/CDC.2015.7402660
We study the SIRS (SusceptibleInfectedRecoveredSusceptible) spreading processes over complex networks, by considering its exact 3nstate Markov chain model. The Markov chain model exhibits an interesting connection with its 2nstate nonlinear "meanfield" approximation and the latter's corresponding linear approximation. We show that under the specific threshold where the diseasefree state is a globally stable fixed point of both the linear and nonlinear models, the exact underlying Markov chain has an O(log n) mixing time, which means the epidemic dies out quickly. In fact, the epidemic eradication condition coincides for all the three models. Furthermore, when the threshold condition is violated, which indicates that the linear model is not stable, we show that there exists a unique second fixed point for the nonlinear model, which corresponds to the endemic state. We also investigate the effect of adding immunization to the SIRS epidemics by introducing two different models, depending on the efficacy of the vaccine. Our results indicate that immunization improves the threshold of epidemic eradication. Furthermore, the common threshold for fastmixing of the Markov chain and global stability of the diseasefree fixed point improves by the same factor for the vaccinationdominant model.https://authors.library.caltech.edu/records/x2mdckdh21Phaseless superresolution using masks
https://resolver.caltech.edu/CaltechAUTHORS:20160523154036848
Authors: Jaganathan, Kishore; Saunderson, James; Fazel, Maryam; Eldar, Yonina C.; Hassibi, Babak
Year: 2016
DOI: 10.1109/ICASSP.2016.7472436
Phaseless superresolution is the problem of reconstructing a signal from its lowfrequency Fourier magnitude measurements. It is the combination of two classic signal processing problems: phase retrieval and superresolution. Due to the absence of phase and highfrequency measurements, additional information is required in order to be able to uniquely reconstruct the signal of interest. In this work, we use masks to introduce redundancy in the phaseless measurements. We develop an analysis framework for this setup, and use it to show that any superresolution algorithm can be seamlessly extended to solve phaseless superresolution (up to a global phase), when measurements are obtained using a certain set of masks. In particular, we focus our attention on a robust semidefinite relaxationbased algorithm, and provide reconstruction guarantees. Numerical simulations complement our theoretical analysis.https://authors.library.caltech.edu/records/abt4g1m533BER analysis of the box relaxation for BPSK signal recovery
https://resolver.caltech.edu/CaltechAUTHORS:20160523152436171
Authors: Thrampoulidis, Christos; Abbasi, Ehsan; Xu, Weiyu; Hassibi, Babak
Year: 2016
DOI: 10.1109/ICASSP.2016.7472383
We study the problem of recovering an ndimensional BPSK signal from m linear noisecorrupted measurements using the box relaxation method which relaxes the discrete set {±
1}^n to the convex set [1,1]^n to obtain a convex optimization algorithm followed by hard thresholding. When the noise and measurement matrix have iid standard normal entries, we obtain an exact expression for the bitwise probability of error P_e in the limit of n and m growing and m/n fixed. At high SNR our result shows that the P_e of box relaxation is within 3dB of the matched filter bound (MFB) for square systems, and that it approaches the (MFB) as m grows large compared to n. Our results also indicate that as m, n → ∞, for any fixed set of size k, the error events of the corresponding k bits in the box relaxation method are independent.https://authors.library.caltech.edu/records/m4q17xh059Simple algorithms and guarantees for low rank matrix completion over F_2
https://resolver.caltech.edu/CaltechAUTHORS:20170127135215938
Authors: Saunderson, James; Fazel, Maryam; Hassibi, Babak
Year: 2016
DOI: 10.1109/ISIT.2016.7541266
Let X* be a n_1 × n_2 matrix with entries in F_2 and rank r <; min(n_1, n_2) (often r ≪ min(n_1, n_2)). We consider the problem of reconstructing X* given only a subset of its entries. This problem has recently found numerous applications, most notably in network and index coding, where finding optimal linear codes (over some field Fq) can be reduced to finding the minimum rank completion of a matrix with a subset of revealed entries. The problem of matrix completion over reals also has many applications and in recent years several polynomialtime algorithms with provable recovery guarantees have been developed. However, to date, such algorithms do not exist in the finitefield case. We propose a linear algebraic algorithm, based on inferring lowweight relations among the rows and columns of X*, to attempt to complete X* given a random subset of its entries. We establish conditions on the row and column spaces of X* under which the algorithm runs in polynomial time (in the size of X*) and can successfully complete X* with high probability from a vanishing fraction of its entries. We then propose a linear programmingbased extension of our basic algorithm, and evaluate it empirically.https://authors.library.caltech.edu/records/ek7bx1v827Similarity clustering in the presence of outliers: Exact recovery via convex program
https://resolver.caltech.edu/CaltechAUTHORS:20160823111539026
Authors: Korlakai Vinayak, Ramya; Hassibi, Babak
Year: 2016
DOI: 10.1109/ISIT.2016.7541267
We study the problem of clustering a set of data points based on their similarity matrix, each entry of which represents the similarity between the corresponding pair of points. We propose a convexoptimizationbased algorithm for clustering using the similarity matrix, which has provable recovery guarantees. It needs no prior knowledge of the number of clusters and it behaves in a robust way in the presence of outliers and noise. Using a generative stochastic model for the similarity matrix (which can be thought of as a generalization of the classical Stochastic Block Model) we obtain precise bounds (not orderwise) on the sizes of the clusters, the number of outliers, the noise variance, separation between the mean similarities inside and outside the clusters and the values of the regularization parameter that guarantee the exact recovery of the clusters with high probability. The theoretical findings are corroborated with extensive evidence from simulations.https://authors.library.caltech.edu/records/twjjck0h46Robust causal transform coding for LQG systems with delay loss in communications
https://resolver.caltech.edu/CaltechAUTHORS:20170324084647017
Authors: Izadinasab, Mohammad Kazem; Bahari Sani, Amir Homayoun; Lahouti, Farshad; Hassibi, Babak
Year: 2016
DOI: 10.1109/ACC.2016.7526056
A networked controlled system (NCS) in which the plant communicates to the controller over a channel with random delay loss is considered. The channel model is motivated by recent development of tree codes for NCS, which effectively translates an erasure channel to one with random delay. A causal transform coding scheme is presented which exploits the plant state memory for efficient communications (compression) and provides robustness to channel delay loss. In this setting, we analyze the performance of linear quadratic Gaussian (LQG) closedloop systems and the design of the optimal controller. The design of the transform code for LQG systems is posed as a channel optimized source coding problem of minimizing a weighted mean squared error over the channel. The solution is characterized in two steps of obtaining the optimized causal encoding and decoding transforms and rate allocation across a set of transform coding quantizers. Numerical and simulation results for GaussMarkov sources and an LQG system demonstrate the effectiveness of the proposed schemes.https://authors.library.caltech.edu/records/xjwh16j175Balanced ReedSolomon codes
https://resolver.caltech.edu/CaltechAUTHORS:20160823155622638
Authors: Halbawi, Wael; Liu, Zihan; Hassibi, Babak
Year: 2016
DOI: 10.1109/ISIT.2016.7541436
We consider the problem of constructing linear MDS errorcorrecting codes with generator matrices that are sparsest and balanced. In this context, sparsest means that every row has the least possible number of nonzero entries, and balanced means that every column contains the same number of nonzero entries. Codes with this structure minimize the maximal computation time of computing any code symbol, a property that is appealing to systems where computational loadbalancing is critical. The problem was studied before by Dau et al. where it was shown that there always exists an MDS code over a sufficiently large field such that its generator matrix is both sparsest and balanced. However, the construction is not explicit and more importantly, the resulting MDS codes do not lend themselves to efficient error correction. With an eye towards explicit constructions with efficient decoding, we show in this paper that the generator matrix of a cyclic ReedSolomon code of length n and dimension k can always be transformed to one that is both sparsest and balanced, for all parameters n and k where k/n (n − k + 1) is an integer.https://authors.library.caltech.edu/records/y7ncktp924(Almost) practical tree codes
https://resolver.caltech.edu/CaltechAUTHORS:20160824092753974
Authors: Khina, Anatoly; Halbawi, Wael; Hassibi, Babak
Year: 2016
DOI: 10.1109/ISIT.2016.7541730
We consider the problem of stabilizing an unstable plant driven by bounded noise over a digital noisy communication link, a scenario at the heart of networked control. To stabilize such a plant, one needs realtime encoding and decoding with an error probability profile that decays exponentially with the decoding delay. The works of Schulman and Sahai over the past two decades have developed the notions of tree codes and anytime capacity, and provided the theoretical framework for studying such problems. Nonetheless, there has been little practical progress in this area due to the absence of explicit constructions of tree codes with efficient encoding and decoding algorithms. Recently, linear timeinvariant tree codes were proposed to achieve the desired result under maximumlikelihood decoding. In this work, we take one more step towards practicality, by showing that these codes can be efficiently decoded using sequential decoding algorithms, up to some loss in performance (and with some practical complexity caveats). We supplement our theoretical results with numerical simulations that demonstrate the effectiveness of the decoder in a control system setting.https://authors.library.caltech.edu/records/p28rn1f744Ratecost tradeoffs in control
https://resolver.caltech.edu/CaltechAUTHORS:20170221070702279
Authors: Kostina, Victoria; Hassibi, Babak
Year: 2016
DOI: 10.1109/ALLERTON.2016.7852366
Consider a distributed control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is minimize a quadratic cost function. The most basic special case of that cost function is the meansquare deviation of the system state from the desired state. We study the fundamental tradeoff between the communication rate r bits/sec and the limsup of the expected cost b, and show a lower bound on the rate necessary to attain b. The bound applies as long as the system noise has a probability density function. If target cost b is not too large, that bound can be closely approached by a simple lattice quantization scheme that only quantizes the innovation, that is, the difference between the controller's belief about the current state and the true state.https://authors.library.caltech.edu/records/22nz5yr497General performance metrics for the LASSO
https://resolver.caltech.edu/CaltechAUTHORS:20161102074552760
Authors: Abbasi, Ehsan; Thrampoulidis, Christos; Hassibi, Babak
Year: 2016
DOI: 10.1109/ITW.2016.7606820
A recent line of work has established accurate predictions of the mean squarederror (MSE) performance of nonsmooth convex optimization methods when used to recover structured signals (e.g. sparse, lowrank) from noisy linear (and possibly compressed) observations. Specifically, in a recent paper [15] we precisely characterized the MSE performance of a general class of regularized Mestimators using a framework that is based on Gaussian process methods. Here, we extend the framework to the analysis of a general class of Lipschitz performance metrics, which in addition to the standard MSE, includes the ℓ1reconstruction error, the probability of successfully identifying whether an element belongs to the support of a sparse signal, the empirical distribution of the error, etc. For concreteness, we primarily focus on the problem of sparse recovery under ℓ1regularized leastsquares (aka LASSO). We illustrate the validity of the theoretical predictions through numerical simulations and discuss the importance of their precise nature in optimally tuning the involved parameters of the reconstruction method.https://authors.library.caltech.edu/records/esepm1ms91Balanced Reed–Solomon Codes for All Parameters
https://resolver.caltech.edu/CaltechAUTHORS:20161102081439051
Authors: Halbawi, Wael; Liu, Zihan; Hassibi, Babak
Year: 2016
DOI: 10.1109/ITW.2016.7606866
We construct balanced and sparsest generator matrices for cyclic ReedSolomon codes with any length n and dimension k. By sparsest, we mean that each row has the least possible number of nonzeros, while balanced means that the number of nonzeros in any two columns differs by at most one. Codes allowing such encoding schemes are useful in distributed settings where computational loadbalancing is critical. The problem was first studied by Dau et al. who showed, using probabilistic arguments, that there always exists an MDS code over a sufficiently large field such that its generator matrix is both sparsest and balanced. Motivated by the need for an explicit construction with efficient decoding, the authors of the current paper showed that the generator matrix of a cyclic ReedSolomon code of length n and dimension k can always be transformed to one that is both sparsest and balanced, when n and k are such that k/n (n−k+1) is an integer. In this paper, we lift this condition and construct balanced and sparsest generator matrices for cyclic ReedSolomon codes for any set of parameters.https://authors.library.caltech.edu/records/err9wzh687Phase retrieval: an overview of recent developments
https://resolver.caltech.edu/CaltechAUTHORS:20170816082425540
Authors: Jaganathan, Kishore; Eldar, Yonina C.; Hassibi, Babak
Year: 2016
[no abstract]https://authors.library.caltech.edu/records/nct737wq75Ambiguities on convolutions with applications to phase retrieval
https://resolver.caltech.edu/CaltechAUTHORS:20170308150241370
Authors: Walk, Philipp; Jung, Peter; Pfander, Götz; Hassibi, Babak
Year: 2016
DOI: 10.1109/ACSSC.2016.7869569
In this work we characterize all ambiguities of the convolution on two fixed finitedimensional complex vector spaces. It will be shown that the convolution ambiguities correspond to factorization ambiguities in the zdomain, which are generated by swapping their zeros. We use this polynomial description to show a deterministic version of a recently introduced masked Fourier phase retrieval design. A semidefinite program can be used to recover exactly the noisefree input signals if they share no common factors. Then, we reformulate the problem as a deterministic blind deconvolution with knowledge of their autocorrelations. Moreover, numerically simulations show robustness against additive noise.https://authors.library.caltech.edu/records/m0wnvv5c26Fundamental Limits of BudgetFidelity Tradeoff in Label Crowdsourcing
https://resolver.caltech.edu/CaltechAUTHORS:20210105133359837
Authors: Lahouti, Farshad; Hassibi, Babak
Year: 2016
DOI: 10.48550/arXiv.1608.07328
Digital crowdsourcing (CS) is a modern approach to perform certain large projects using small contributions of a large crowd. In CS, a taskmaster typically breaks down the project into small batches of tasks and assigns them to socalled workers with imperfect skill levels. The crowdsourcer then collects and analyzes the results for inference and serving the purpose of the project. In this work, the CS problem, as a humanintheloop computation problem, is modeled and analyzed in an information theoretic ratedistortion framework. The purpose is to identify the ultimate fidelity that one can achieve by any form of query from the crowd and any decoding (inference) algorithm with a given budget. The results are established by a joint source channel (de)coding scheme, which represent the query scheme and inference, over parallel noisy channels, which model workers with imperfect skill levels. We also present and analyze a query scheme dubbed kary incidence coding and study optimized query pricing in this setting.https://authors.library.caltech.edu/records/12he5bqr65Multirate control over AWGN channels via analog joint sourcechannel coding
https://resolver.caltech.edu/CaltechAUTHORS:20170106132902522
Authors: Khina, Anatoly; Pettersson, Gustav M.; Kostina, Victoria; Hassibi, Babak
Year: 2016
DOI: 10.1109/CDC.2016.7799185
We consider the problem of controlling an unstable plant over an additive white Gaussian noise (AWGN) channel with a transmit power constraint, where the signaling rate of communication is larger than the sampling rate (for generating observations and applying control inputs) of the underlying plant. Such a situation is quite common since sampling is done at a rate that captures the dynamics of the plant and which is often much lower than the rate that can be communicated. This setting offers the opportunity of improving the system performance by employing multiple channel uses to convey a single message (output plant observation or control input). Common ways of doing so are through either repeating the message, or by quantizing it to a number of bits and then transmitting a channel coded version of the bits whose length is commensurate with the number of channel uses per sampled message. We argue that such "separated source and channel coding" can be suboptimal and propose to perform joint sourcechannel coding. Since the block length is short we obviate the need to go to the digital domain altogether and instead consider analog joint sourcechannel coding. For the case where the communication signaling rate is twice the sampling rate, we employ the Archimedean bispiralbased ShannonKotel'nikov analog maps to show significant improvement in stability margins and linearquadratic Gaussian (LQG) costs over simple schemes that employ repetition.https://authors.library.caltech.edu/records/s1p2jr3e02LQG control for systems with random unbounded communication delay
https://resolver.caltech.edu/CaltechAUTHORS:20170106132340959
Authors: Bengtsson, Fredrik; Hassibi, Babak; Wik, Torsten
Year: 2016
DOI: 10.1109/CDC.2016.7798406
In this paper LQG control over unreliable communication links is examined. That is to say, the communication channels between the controller and the actuators and between the sensors and the controller are unreliable. This is of growing importance as networked control systems and use of wireless communication in control are becoming increasingly common. A proposed approach is to use tree codes to turn lossy channels into ones with a random delay. The problem of how to optimize LQG control in this case is examined, and it is found that to optimize LQG control previous control signals must also be used. Only the situation where communication between the components is done with acknowledgments is examined. An optimal solution is derived for finite horizon discrete holdinput LQG control for this case. The solution is compared with standard LQG control in simulations, which demonstrate that a significant improvement in the cost can be achieved when the probability of delay is high.https://authors.library.caltech.edu/records/6jgmxjma91Improved bounds on the epidemic threshold of exact SIS models on complex networks
https://resolver.caltech.edu/CaltechAUTHORS:20170106135044789
Authors: Azizan Ruhi, Navid; Thrampoulidis, Christos; Hassibi, Babak
Year: 2016
DOI: 10.1109/CDC.2016.7798804
The SIS (susceptibleinfectedsusceptible) epidemic model on an arbitrary network, without making approximations, is a 2^nstate Markov chain with a unique absorbing state (the allhealthy state). This makes analysis of the SIS model and, in particular, determining the threshold of epidemic spread quite challenging. It has been shown that the exact marginal probabilities of infection can be upper bounded by an ndimensional linear timeinvariant system, a consequence of which is that the Markov chain is "fastmixing" when the LTI system is stable, i.e. when β/δ < 1/(λ_(max)(A)) (where β is the infection rate per link, δ is the recovery rate, and λ_(max)(A) is the largest eigenvalue of the network's adjacency matrix). This wellknown threshold has been recently shown not to be tight in several cases, such as in a star network. In this paper, we provide tighter upper bounds on the exact marginal probabilities of infection, by also taking pairwise infection probabilities into account. Based on this improved bound, we derive tighter eigenvalue conditions that guarantee fast mixing (i.e., logarithmic mixing time) of the chain. We demonstrate the improvement of the threshold condition by comparing the new bound with the known one on various networks with various epidemic parameters.https://authors.library.caltech.edu/records/35h16xbc43Tensorbased crowdsourced clustering via triangle queries
https://resolver.caltech.edu/CaltechAUTHORS:20170621171615546
Authors: Korlakai Vinayak, Ramya; Zrnic, Tijana; Hassibi, Babak
Year: 2017
DOI: 10.1109/ICASSP.2017.7952571
We consider the problem of crowdsourced clustering of a set of items based on queries of the similarity of triple of objects. Such an approach, called triangle queries, was proposed in [1], where it was shown that, for a fixed query budget, it outperforms clustering based on edge queries (i.e, comparing pairs of objects). In [1] the clustering algorithm for triangle and edge queries was identical and each triangle query response was treated as 3 separate edge query responses. In this paper we directly exploit the triangle structure of the responses by embedding them into a 3way tensor. Since there are 5 possible responses to each triangle query, it is a priori not clear how best to embed them into the tensor. We give sufficient conditions on nontrivial embedding such that the resulting tensor has a rank equal to the underlying number of clusters (akin to what happens with the rank of the adjacency matrix). We then use an alternating least squares tensor decomposition algorithm to cluster a noisy and partially observed tensor and show, through extensive numerical simulations, that it significantly outperforms methods that make use only of the adjacency matrix.https://authors.library.caltech.edu/records/r01168va97Phaseless superresolution in the continuous domain
https://resolver.caltech.edu/CaltechAUTHORS:20170621163013430
Authors: Cho, Myung; Thrampoulidis, Christos; Xu, Weiyu; Hassibi, Babak
Year: 2017
DOI: 10.1109/ICASSP.2017.7952870
Phaseless superresolution refers to the problem of superresolving a signal from only its lowfrequency Fourier magnitude measurements. In this paper, we consider the phaseless superresolution problem of recovering a sum of sparse Dirac delta functions which can be located anywhere in the continuous timedomain. For such signals in the continuous domain, we propose a novel Semidefinite Programming (SDP) based signal recovery method to achieve the phaseless superresolution. This work extends the recent work of Jaganathan et al. [1], which considered phaseless superresolution for discrete signals on the grid.https://authors.library.caltech.edu/records/5x9q4y1634Nearoptimal sample complexity bounds for circulant binary embedding
https://resolver.caltech.edu/CaltechAUTHORS:20170621165223982
Authors: Oymak, Samet; Thrampoulidis, Christos; Hassibi, Babak
Year: 2017
DOI: 10.1109/ICASSP.2017.7953380
Binary embedding is the problem of mapping points from a highdimensional space to a Hamming cube in lower dimension while preserving pairwise distances. An efficient way to accomplish this is to make use of fast embedding techniques involving Fourier transform e.g. circulant matrices. While binary embedding has been studied extensively, theoretical results on fast binary embedding are rather limited. In this work, we build upon the recent literature to obtain significantly better dependencies on the problem parameters. A set of N points in ℝ^n can be properly embedded into the Hamming cube {±1}^k with δ distortion, by using k ∼ δ^−3 log N samples which is optimal in the number of points N and compares well with the optimal distortion dependency δ^−2. Our optimal embedding result applies in the regime log N ≲ n^(1/3). Furthermore, if the looser condition log N ≲ √n holds, we show that all but an arbitrarily small fraction of the points can be optimally embedded. We believe the proposed techniques can be useful to obtain improved guarantees for other nonlinear embedding problems.https://authors.library.caltech.edu/records/8bgkw9jb36Multiple illumination phaseless superresolution (MIPS) with applications to phaseless DoA estimation and diffraction imaging
https://resolver.caltech.edu/CaltechAUTHORS:20170621154158283
Authors: Salehi, Fariborz; Jaganathan, Kishore; Hassibi, Babak
Year: 2017
DOI: 10.1109/ICASSP.2017.7952897
Phaseless superresolution is the problem of recovering an unknown signal from measurements of the "magnitudes" of the "low frequency" Fourier transform of the signal. This problem arises in applications where measuring the phase, and making highfrequency measurements, are either too costly or altogether infeasible. The problem is especially challenging because it combines the difficult problems of phase retrieval and classical superresolution. Recently, the authors in [1] demonstrated that by making three phaseless lowfrequency measurements, obtained by appropriately "masking" the signal, one can uniquely and robustly identify the phase using convex programming and obtain the same superresolution performance reported in [2]. However, the masks proposed in [1] are very specific and in many applications cannot be directly implemented. In this paper, we broadly extend the class of masks that can be used to recover the phase and show how their effect can be emulated in coherent diffraction imaging using multiple illuminations, as well as in directionofarrival (DoA) estimation using multiple sources to excite the environment. We provide numerical simulations to demonstrate the efficacy of the method and approach.https://authors.library.caltech.edu/records/emafqb2548BER analysis of regularized least squares for BPSK recovery
https://resolver.caltech.edu/CaltechAUTHORS:20171222075024697
Authors: Atitallah, Ismail Ben; Thrampoulidis, Christos; Kammoun, Abla; AlNaffouri, Tareq Y.; Hassibi, Babak; Alouini, MohamedSlim
Year: 2017
DOI: 10.1109/ICASSP.2017.7952960
This paper investigates the problem of recovering an ndimensional BPSK signal x_0 ∈ {1, 1}^n from mdimensional measurement vector y = Ax+z, where A and z are assumed to be Gaussian with iid entries. We consider two variants of decoders based on the regularized least squares followed by hardthresholding: the case where the convex relaxation is from {1, 1}^n to ℝ^n and the box constrained case where the relaxation is to [1, 1]^n. For both cases, we derive an exact expression of the bit error probability when n and m grow simultaneously large at a fixed ratio. For the box constrained case, we show that there exists a critical value of the SNR, above which the optimal regularizer is zero. On the other side, the regularization can further improve the performance of the box relaxation at low to moderate SNR regimes. We also prove that the optimal regularizer in the bit error rate sense for the unboxed case is nothing but the MMSE detector.https://authors.library.caltech.edu/records/xwj8qn7847The BOXLASSO with application to GSSK modulation in massive MIMO systems
https://resolver.caltech.edu/CaltechAUTHORS:20170816174203875
Authors: Atitallah, Ismail Ben; Thrampoulidis, Christos; Kammoun, Abla; AlNaffouri, Tareq Y.; Alouini, MohamedSlim; Hassibi, Babak
Year: 2017
DOI: 10.1109/ISIT.2017.8006695
The BOXLASSO is a variant of the popular LASSO that includes an additional boxconstraint. We propose its use as a decoder in modern Multiple Input Multiple Output (MIMO) communication systems with modulation methods such as the Generalized Space Shift Keying (GSSK) modulation, which produces constellation vectors that are inherently sparse and with bounded elements. In that direction, we prove novel explicit asymptotic characterizations of the squarederror and of the perelement error rate of the BOXLASSO, under iid Gaussian measurements. In particular, the theoretical predictions can be used to quantify the improved performance of the BOXLASSO, when compared to the previously used standard LASSO. We include simulation results that validate both these premises and our theoretical predictions.https://authors.library.caltech.edu/records/hyfprsha48Shortmessage communication and FIR system identification using Huffman sequences
https://resolver.caltech.edu/CaltechAUTHORS:20170816174357172
Authors: Walk, Philipp; Jung, Peter; Hassibi, Babak
Year: 2017
DOI: 10.1109/ISIT.2017.8006672
Providing shortmessage communication and simultaneous channel estimation for sporadic and fast fading scenarios is a challenge for future wireless networks. In this work we propose a novel blind communication and deconvolution scheme by using Huffman sequences, which allows to solve three important tasks at once: (i) determination of the transmit power (ii) identification of the instantaneous discretetime FIR channel if the channel delay is less than L/2 and (iii) simultaneously communicating L1 bits of information. Our signal reconstruction uses a recent semidefinite program that can recover two unknown signals from their autocorrelations and crosscorrelations. This convex algorithm shows numerical stability and operates fully deterministic without any further channel assumptions.https://authors.library.caltech.edu/records/v68b6ea348Error bounds for Bregman denoising and structured natural parameter estimation
https://resolver.caltech.edu/CaltechAUTHORS:20170816174255469
Authors: Jalali, Amin; Saunderson, James; Fazel, Maryam; Hassibi, Babak
Year: 2017
DOI: 10.1109/ISIT.2017.8006934
We analyze an estimator based on the Bregman divergence for recovery of structured models from additive noise. The estimator can be seen as a regularized maximum likelihood estimator for an exponential family where the natural parameter is assumed to be structured. For all such Bregman denoising estimators, we provide an error bound for a natural associated error measure. Our error bound makes it possible to analyze a wide range of estimators, such as those in proximal denoising and inverse covariance matrix estimation, in a unified manner. In the case of proximal denoising, we exactly recover the existing tight normalized mean squared error bounds. In sparse precision matrix estimation, our bounds provide optimal scaling with interpretable constants in terms of the associated error measure.https://authors.library.caltech.edu/records/z9bs4fj172Entropic Causality and Greedy Minimum Entropy Coupling
https://resolver.caltech.edu/CaltechAUTHORS:20170816170538199
Authors: Kocaoglu, Murat; Dimakis, Alexandros G.; Vishwanath, Sriram; Hassibi, Babak
Year: 2017
DOI: 10.1109/ISIT.2017.8006772
We study the problem of identifying the causal relationship between two discrete random variables from observational data. We recently proposed a novel framework called entropie causality that works in a very general functional model but makes the assumption that the unobserved exogenous variable has small entropy in the true causal direction. This framework requires the solution of a minimum entropy coupling problem: Given marginal distributions of m discrete random variables, each on n states, find the joint distribution with minimum entropy, that respects the given marginals. This corresponds to minimizing a concave function of n^m variables over a convex polytope defined by nm linear constraints, called a transportation polytope. Unfortunately, it was recently shown that this minimum entropy coupling problem is NPhard, even for 2 variables with n states. Even representing points (joint distributions) over this space can require exponential complexity (in n, m) if done naively. In our recent work we introduced an efficient greedy algorithm to find an approximate solution for this problem. In this paper we analyze this algorithm and establish two results: that our algorithm always finds a local minimum and also is within an additive approximation error from the unknown global optimum.https://authors.library.caltech.edu/records/wtbrrfda35Balanced and sparse TamoBarg codes
https://resolver.caltech.edu/CaltechAUTHORS:20170816172424731
Authors: Halbawi, Wael; Duursma, Iwan; Dau, Hoang; Hassibi, Babak
Year: 2017
DOI: 10.1109/ISIT.2017.8006682
We construct balanced and sparse generator matrices for Tamo and Barg's Locally Recoverable Codes (LRCs). More specifically, for a cyclic TamoBarg code of length n, dimension k and locality r, we show how to deterministically construct a generator matrix where the number of nonzeros in any two columns differs by at most one, and where the weight of every row is d + r − 1, where d is the minimum distance of the code. Since LRCs are designed mainly for distributed storage systems, the results presented in this work provide a computationally balanced and efficient encoding scheme for these codes. The balanced property ensures that the computational effort exerted by any storage node is essentially the same, whilst the sparse property ensures that this effort is minimal. The work presented in this paper extends a similar result previously established for ReedSolomon (RS) codes, where it is now known that any cyclic RS code possesses a generator matrix that is balanced as described, but is sparsest, meaning that each row has d nonzeros.https://authors.library.caltech.edu/records/hk9ehp6p45Performance of real phase retrieval
https://resolver.caltech.edu/CaltechAUTHORS:20170908075942456
Authors: Abbasi, Ehsan; Salehi, Fariborz; Hassibi, Babak
Year: 2017
DOI: 10.1109/SAMPTA.2017.8024478
This paper analyzes the meansquare error performance of the popular PhaseLift algorithm for phase retrieval, which is the problem of recovering an unknown signal from the magnitudes of a collection of linear measurements of the signal. This problem arises in many physical systems where only magnitudes can be measured. Our analysis approach is based on a novel comparison lemma, which upper bounds the performance of the convexoptimizationbased PhaseLift algorithm in terms of an auxiliary convex optimization algorithm which is much more amenable to analysis. An upshot of our analysis is that an ndimensional unknown signal can be recovered from the magnitudes of cn random linear measurements, where c > 1 is a constant (which empirically appears to be 3). This improves the best known earlier results which could only guarantee signal recovery with O(n log n) magnitudeonly measurements. in fact, and more explicitly, we show that the sufficient number of the measurements in the high SNR regime (which corresponds to noiseless phase retrieval) can be derived from solving a deterministic convex optimization in 3 variables.https://authors.library.caltech.edu/records/kv55378n69Constrained blind deconvolution using Wirtinger flow methods
https://resolver.caltech.edu/CaltechAUTHORS:20171221161105987
Authors: Walk, Philipp; Jung, Peter; Hassibi, Babak
Year: 2017
DOI: 10.1109/SAMPTA.2017.8024425
In this work we consider onedimensional blind deconvolution with prior knowledge of signal autocorrelations in the classical framework of polynomial factorization. In particular this univariate case highly suffers from several nontrivial ambiguities and therefore blind deconvolution is known to be illposed in general. However, if additional autocorrelation information is available and the corresponding polynomials are coprime, blind deconvolution is uniquely solvable up to global phase. Using lifting, the outer product of the unknown vectors is the solution to a (convex) semidefinite program (SDP) demonstrating that theoretically recovery is computationally tractable. However, for practical applications efficient algorithms are required which should operate in the original signal space. To this end we also discuss a gradient descent algorithm (Wirtinger flow) for the original nonconvex problem. We demonstrate numerically that such an approach has performance comparable to the semidefinite program in the noisy case. Our work is motivated by applications in blind communication scenarios and we will discuss a specific signaling scheme where information is encoded into polynomial roots.https://authors.library.caltech.edu/records/113gakvb312D phaseless superresolution
https://resolver.caltech.edu/CaltechAUTHORS:20171220143041947
Authors: Cho, Myung; Thramboulidis, Christos; Hassibi, Babak; Xu, Weiyu
Year: 2017
DOI: 10.1117/12.2275409
In phaseless superresolution, we only have the magnitude information of continuouslyparameterized signals in a transform domain, and try to recover the original signals from these magnitude measurements. Optical microscopy is one application where the phaseless superresolution for 2D signals arise. In this paper, we propose algorithms for performing phaseless superresolution for 2D or higherdimensional signals, and investigate their performance guarantees.https://authors.library.caltech.edu/records/4qygcy9j87Sequential coding of GaussMarkov sources with packet erasures and feedback
https://resolver.caltech.edu/CaltechAUTHORS:20180209081746219
Authors: Khina, Anatoly; Kostina, Victoria; Khisti, Ashish; Hassibi, Babak
Year: 2017
DOI: 10.1109/ITW.2017.8277955
We consider the problem of sequential transmission of GaussMarkov sources. We show that in the limit of large spatial block lengths, greedy compression with respect to the squared error distortion is optimal; that is, there is no tension between optimizing the distortion of the source in the current time instant and that of future times. We then extend this result to the case where at time t a random compression rate rt is allocated independently of the rate at other time instants. This, in turn, allows us to derive the optimal performance of sequential coding over packeterasure channels with instantaneous feedback. For the case of packet erasures with delayed feedback, we connect the problem to that of compression with side information that is known at the encoder and may be known at the decoder — where the most recent packets serve as side information that may have been erased, and demonstrate that the loss due to a delay by one time unit is rather small.https://authors.library.caltech.edu/records/5d8b1t5m45Algorithms for Optimal Control with FixedRate Feedback
https://resolver.caltech.edu/CaltechAUTHORS:20180126083658723
Authors: Khina, Anatoly; Nakahira, Yorie; Su, YuSu; Hassibi, Babak
Year: 2017
DOI: 10.1109/CDC.2017.8264569
We consider a discretetime linear quadratic Gaussian networked control setting where the (full information) observer and controller are separated by a fixedrate noiseless channel. The minimal rate required to stabilize such a system has been well studied. However, for a given fixed rate, how to quantize the states so as to optimize performance is an open question of great theoretical and practical significance. We concentrate on minimizing the control cost for firstorder scalar systems. To that end, we use the LloydMax algorithm and leverage properties of logarithmicallyconcave functions to construct the optimal quantizer that greedily minimizes the cost at every time instant. By connecting the globally optimal scheme to the problem of scalar successive refinement, we argue that its gain over the proposed greedy algorithm is negligible. This is significant since the globally optimal scheme is often computationally intractable. All the results are proven for the more general case of disturbances with logarithmicallyconcave distributions.https://authors.library.caltech.edu/records/9pbwtm0b81A Universal Analysis of LargeScale Regularized Least Squares Solutions
https://resolver.caltech.edu/CaltechAUTHORS:20190107105602343
Authors: Panahi, Ashkan; Hassibi, Babak
Year: 2017
A problem that has been of recent interest in statistical inference, machine learning and signal processing is that of understanding the asymptotic behavior of regularized least squares solutions under random measurement matrices (or dictionaries). The Least Absolute Shrinkage and Selection Operator (LASSO or leastsquares with ℓ_1 regularization) is perhaps one of the most interesting examples. Precise expressions for the asymptotic performance of LASSO have been obtained for a number of different cases, in particular when the elements of the dictionary matrix are sampled independently from a Gaussian distribution. It has also been empirically observed that the resulting expressions remain valid when the entries of the dictionary matrix are independently sampled from certain nonGaussian distributions. In this paper, we confirm these observations theoretically when the distribution is subGaussian. We further generalize the previous expressions for a broader family of regularization functions and under milder conditions on the underlying random, possibly nonGaussian, dictionary matrix. In particular, we establish the universality of the asymptotic statistics (e.g., the average quadratic risk) of LASSO with nonGaussian dictionaries.https://authors.library.caltech.edu/records/k5x99xaw30Distributed Solution of LargeScale Linear Systems Via Accelerated ProjectionBased Consensus
https://resolver.caltech.edu/CaltechAUTHORS:20180920110215437
Authors: AzizanRuhi, Navid; Lahouti, Farshad; Avestimehr, Salman; Hassibi, Babak
Year: 2018
DOI: 10.1109/ICASSP.2018.8462630
Solving a largescale system of linear equations is a key step at the heart of many algorithms in scientific computing, machine learning, and beyond. When the problem dimension is large, computational and/or memory constraints make it desirable, or even necessary, to perform the task in a distributed fashion. In this paper, we consider a common scenario in which a taskmaster intends to solve a largescale system of linear equations by distributing subsets of the equations among a number of computing machines/cores. We propose a new algorithm called Accelerated Projectionbased Consensus (APC) for this problem. The convergence behavior of the proposed algorithm is analyzed in detail and analytically shown to compare favorably with the convergence rate of alternative distributed methods, namely distributed gradient descent, distributed versions of Nesterov's accelerated gradient descent and heavyball method, the block Cimmino method, and ADMM. On randomly chosen linear systems, as well as on realworld data sets, the proposed method offers significant speedup relative to all the aforementioned methods.https://authors.library.caltech.edu/records/0zssrtmz45An Improved Initialization for LowRank Matrix Completion Based on RankL Updates
https://resolver.caltech.edu/CaltechAUTHORS:20180920111338769
Authors: Douik, Ahmed; Hassibi, Babak
Year: 2018
DOI: 10.1109/ICASSP.2018.8461826
Given a data matrix with partially observed entries, the lowrank matrix completion problem is one of finding a matrix with the lowest rank that perfectly fits the given observations. While there exist convex relaxations for the lowrank completion problem, the underlying problem is inherently nonconvex, and most algorithms (alternating projection, Riemannian optimization, etc.) heavily depend on the initialization. This paper proposes an improved initialization that relies on successive rankl updates. Further, the paper proposes theoretical guarantees under which the proposed initialization is closer to the unknown optimal solution than the all zeros initialization in the Frobenius norm. To cope with the problem of local minima, the paper introduces and uses random norms to change the position of the local minima while preserving the global one. Using a Riemannian optimization routine, simulation results reveal that the proposed solution succeeds in completing Gaussian partially observed matrices with a random set of revealed entries close to the informationtheoretical limits, thereby significantly improving on prior methods.https://authors.library.caltech.edu/records/y6wewfj480Improving Distributed Gradient Descent Using ReedSolomon Codes
https://resolver.caltech.edu/CaltechAUTHORS:20181126142648719
Authors: Halbawi, Wael; Azizan, Navid; Salehi, Fariborz; Hassibi, Babak
Year: 2018
DOI: 10.1109/ISIT.2018.8437467
Today's massivelysized datasets have made it necessary to often perform computations on them in a distributed manner. In principle, a computational task is divided into subtasks which are distributed over a cluster operated by a taskmaster. One issue faced in practice is the delay incurred due to the presence of slow machines, known as stragglers. Several schemes, including those based on replication, have been proposed in the literature to mitigate the effects of stragglers and more recently, those inspired by coding theory have begun to gain traction. In this work, we consider a distributed gradient descent setting suitable for a wide class of machine learning problems. We adopt the framework of Tandon et al. [1] and present a deterministic scheme that, for a prescribed permachine computational effort, recovers the gradient from the least number of machines f theoretically permissible, via an O(f^2) decoding algorithm. The idea is based on a suitably designed ReedSolomon code that has a sparsest and balanced generator matrix. We also provide a theoretical delay model which can be used to minimize the expected waiting time per computation by optimally choosing the parameters of the scheme. Finally, we supplement our theoretical findings with numerical results that demonstrate the efficacy of the method and its advantages over competing schemes.https://authors.library.caltech.edu/records/ytzpnksd32Further Progress on the GMMDS Conjecture for ReedSolomon Codes
https://resolver.caltech.edu/CaltechAUTHORS:20181126140601317
Authors: Yildiz, Hikmet; Hassibi, Babak
Year: 2018
DOI: 10.1109/ISIT.2018.8437308
Designing good error correcting codes whose generator matrix has a support constraint, i.e., one for which only certain entries of the generator matrix are allowed to be nonzero, has found many recent applications, including in distributed coding and storage, multiple access networks, and weakly secure data exchange. The dual problem, where the parity check matrix has a support constraint, comes up in the design of locally repairable codes. The central problem here is to design codes with the largest possible minimum distance, subject to the given support constraint on the generator matrix. An upper bound on the minimum distance can be obtained through a set of singleton bounds, which can be alternatively thought of as a cutset bound. Furthermore, it is well known that, if the field size is large enough, any random generator matrix obeying the support constraint will achieve the maximum minimum distance with high probability. Since random codes are not easy to decode, structured codes with efficient decoders, e.g., ReedSolomon codes, are much more desirable. The GMMDS conjecture of Dau et al states that the maximum minimum distance over all codes satisfying the generator matrix support constraint can be obtained by a Reed Solomon code. If true, this would have significant consequences. The conjecture has been proven for several special case: when the dimension of the code k is less than or equal to five, when the number of distinct support sets on the rows of the generator matrix m, say, is less than or equal to three, or when the generator matrix is sparsest and balanced. In this paper, we report on further progress on the GMMDS conjecture. 1. In particular, we show that the conjecture is true for all m less than equal to six. This generalizes all previous known results (except for the sparsest and balanced case, which is a very special support constraint).https://authors.library.caltech.edu/records/s9dm3y9d66A Riemannian Approach for GraphBased Clustering by Doubly Stochastic Matrices
https://resolver.caltech.edu/CaltechAUTHORS:20180906141856985
Authors: Douik, Ahmed; Hassibi, Babak
Year: 2018
DOI: 10.1109/SSP.2018.8450685
Convex optimization is a wellestablished area with applications in almost all fields. However, these convex methods can be rather slow and computationally intensive for high dimensional problems. For a particular class of problems, this paper considers a different approach, namely Riemannian optimization. The main idea is to view the constrained optimization problem as an unconstrained one over a restricted search space (the manifold). Riemannian optimization explicitly exploits the geometry of the problem and often reduces its dimension, thereby potentially allowing significant speedup as compared to conventional approaches. The paper introduces the doubly stochastic, the symmetric, and the definite multinomial manifolds which generalize the simplex. The method is applied to a convex and a nonconvex graphbased clustering problem. Theoretical analysis and simulation results demonstrate the efficiency of the proposed method over the state of the art as it outperforms conventional generic and specialized solvers, especially in high dimensions.https://authors.library.caltech.edu/records/wesnswdx28A Precise Analysis of PhaseMax in Phase Retrieval
https://resolver.caltech.edu/CaltechAUTHORS:20181126143043881
Authors: Salehi, Fariborz; Abbasi, Ehsan; Hassibi, Babak
Year: 2018
DOI: 10.1109/ISIT.2018.8437494
Recovering an unknown complex signal from the magnitude of linear combinations of the signal is referred to as phase retrieval. We present an exact performance analysis of a recently proposed convexoptimizationformulation for this problem, known as PhaseMax. Standard convexrelaxationbased methods in phase retrieval resort to the idea of "lifting" which makes them computationally inefficient, since the number of unknowns is effectively squared. In contrast, PhaseMax is a novel convex relaxation that does not increase the number of unknowns. Instead it relies on an initial estimate of the true signal which must be externally provided. In this paper, we investigate the required number of measurements for exact recovery of the signal in the large system limit and when the linear measurement matrix is random with iid standard normal entries. If n denotes the dimension of the unknown complex signal and m the number of phaseless measurements, then in the large system limit, m/n > 4/cos^2(θ) measurements is necessary and sufficient to recover the signal with high probability, where θ is the angle between the initial estimate and the true signal. Our result indicates a sharp phase transition in the asymptotic regime which matches the empirical result in numerical simulations.https://authors.library.caltech.edu/records/7cc5vn3h41Ratecost tradeoffs in scalar LQG control and tracking with side information
https://resolver.caltech.edu/CaltechAUTHORS:20190329091228468
Authors: Kostina, Victoria; Hassibi, Babak
Year: 2018
DOI: 10.1109/ALLERTON.2018.8635889
Consider a control problem in which a remote controller chooses its control action based on two kinds of information about the system state: the information it receives from the system via a rateconstrained feedback link, and side information  a noisy measurement of the system state it observes directly. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate, the expected cost b and the quality of side information. Due to a separation principle between estimation and control, we focus on the tracking problem, where the goal is to track the system state rather than to control it. We introduce the causal ratedistortion function with side information at the decoder. It is expressed in terms of directed mutual information, and it extends the classical (noncausal) WynerZiv ratedistortion function to realtime tracking problems with causality constraints and memory of the past at both encoder and decoder. We compute that function in the scalar linear Gaussian setting; we draw a link with the Kalman filter; we show that making side information available also at the encoder does not help to improve the optimal tradeoffs.https://authors.library.caltech.edu/records/2neckc6m85Optimum Linear Codes with Support Constraints over Small Fields
https://resolver.caltech.edu/CaltechAUTHORS:20190204162012565
Authors: Yildiz, Hikmet; Hassibi, Babak
Year: 2018
DOI: 10.1109/ITW.2018.8613535
The problem of designing a linear code with the largest possible minimum distance, subject to support constraints on the generator matrix, has recently found several applications. These include multiple access networks [3], [5] as well as weakly secure data exchange [4], [8]. A simple upper bound on the maximum minimum distance can be obtained from a sequence of Singleton bounds (see (3) below) and can further be achieved by randomly choosing the nonzero elements of the generator matrix from a field of a large enough size.https://authors.library.caltech.edu/records/qd2rdtgc82Learning without the Phase: Regularized PhaseMax Achieves Optimal Sample Complexity
https://resolver.caltech.edu/CaltechAUTHORS:20190416102710748
Authors: Salehi, Fariborz; Abbasi, Ehsan; Hassibi, Babak
Year: 2018
The problem of estimating an unknown signal, x_0 ϵ R^n, from a vector y ϵ R^m consisting of m magnitudeonly measurements of the form y_i = a_ix_o, where a_i's are the rows of a known measurement matrix A is a classical problem known as phase retrieval. This problem arises when measuring the phase is costly or altogether infeasible. In many applications in machine learning, signal processing, statistics, etc., the underlying signal has certain structure (sparse, lowrank, finite alphabet, etc.), opening of up the possibility of recovering x_0 from a number of measurements smaller than the ambient dimension, i.e., m < n. Ideally, one would like to recover the signal from a number of phaseless measurements that is on the order of the "degrees of freedom" of the structured x_0. To this end, inspired by the PhaseMax algorithm, we formulate a convex optimization problem, where the objective function relies on an initial estimate of the true signal and also includes an additive regularization term to encourage structure. The new formulation is referred to as regularized PhaseMax. We analyze the performance of regularized PhaseMax to find the minimum number of phaseless measurements required for perfect signal recovery. The results are asymptotic and are in terms of the geometrical properties (such as the Gaussian width) of certain convex cones. When the measurement matrix has i.i.d. Gaussian entries, we show that our proposed method is indeed orderwise optimal, allowing perfect recovery from a number of phaseless measurements that is only a constant factor away from the degrees of freedom. We explicitly compute this constant factor, in terms of the quality of the initial estimate, by deriving the exact phase transition. The theory well matches empirical results from numerical simulations.https://authors.library.caltech.edu/records/rp7t4c2g04EventTriggered Stochastic Control via Constrained Quantization
https://resolver.caltech.edu/CaltechAUTHORS:20190627084718957
Authors: Yildiz, Hikmet; Su, Yu; Khina, Anatoly; Hassibi, Babak
Year: 2019
DOI: 10.1109/dcc.2019.00124
We consider a discretetime linear quadratic Gaussian networked control setting where the (full information) observer and controller are separated by a fixedrate noiseless channel. We study the eventtriggered control setup in which the encoder may choose to either transmit a packet or remain silent. We recast this problem into that of fixedrate quantization with an extra symbol that corresponds to the silence event. This way, controlling the average transmission rate is possible by constraining the minimal probability of the silence symbol. We supplement our theoretical framework with numerical simulations.https://authors.library.caltech.edu/records/26b54s8k28Performance Analysis of Convex Data Detection in MIMO
https://resolver.caltech.edu/CaltechAUTHORS:20190424102427365
Authors: Abbasi, Ehsan; Salehi, Fariborz; Hassibi, Babak
Year: 2019
DOI: 10.1109/ICASSP.2019.8683890
We study the performance of a convex data detection method in large multipleinput multipleoutput (MIMO) systems. The goal is to recover an ndimensional complex signal whose entries are from an arbitrary constellation D⊂C, using m noisy linear measurements. Since the Maximum Likelihood (ML) estimation involves minimizing a loss function over the discrete set D^n, it becomes computationally intractable for large n. One approach is to relax to a D convex set and to utilize convex programing to solve the problem precise and then to map the answer to the closest point in the set D. We assume an i.i.d. complex Gaussian channel matrix and derive expressions for the symbol error probability of the proposed convex method in the limit of m, n → ∞. Prior work was only able to do so for real valued constellations such as BPSK and PAM. The main contribution of this paper is to extend the results to complex valued constellations. In particular, we use our main theorem to calculate the performance of the complex algorithm for PSK and QAM constellations. In addition, we introduce a closedform formula for the symbol error probability in the highSNR regime and determine the minimum number of measurements m required for consistent signal recovery.https://authors.library.caltech.edu/records/kne9m6rn92A Characterization of Stochastic Mirror Descent Algorithms and Their Convergence Properties
https://resolver.caltech.edu/CaltechAUTHORS:20190425082644083
Authors: Azizan, Navid; Hassibi, Babak
Year: 2019
DOI: 10.1109/ICASSP.2019.8682271
Stochastic mirror descent (SMD) algorithms have recently garnered a great deal of attention in optimization, signal processing, and machine learning. They are similar to stochastic gradient descent (SGD), in that they perform updates along the negative gradient of an instantaneous (or stochastically chosen) loss function. However, rather than update the parameter (or weight) vector directly, they update it in a "mirrored" domain whose transformation is given by the gradient of a strictly convex differentiable potential function. SMD was originally conceived to take advantage of the underlying geometry of the problem as a way to improve the convergence rate over SGD. In this paper, we study SMD, for linear models and convex loss functions, through the lens of H∞ estimation theory and come up with a minimax interpretation of the SMD algorithm which is the counterpart of the H∞ optimality of the SGD algorithm for linear models and quadratic loss. In doing so, we identify a fundamental conservation law that SMD satisfies and use it to study the convergence properties of the algorithm. For constant step size SMD, when the linear model is overparameterized, we give a deterministic proof of convergence for SMD and show that from any initial point, it converges to the closest point in the space of all parameter vectors that interpolate the data, where closest is in the sense of the Bregman divergence of the potential function. This property is referred to as implicit regularization: with an appropriate choice of the potential function one can guarantee convergence to the minimizer of any desired convex regularizer. For vanishing step size SMD, and in the standard stochastic optimization setting, we give a direct and elementary proof of convergence for SMD to the "true" parameter vector which avoids ergodic averaging or appealing to stochastic differential equations.https://authors.library.caltech.edu/records/gy2yr47k54Sparse Covariance Estimation from Quadratic Measurements: A Precise Analysis
https://resolver.caltech.edu/CaltechAUTHORS:20191004100332908
Authors: Abbasi, Ehsan; Salehi, Fariborz; Hassibi, Babak
Year: 2019
DOI: 10.1109/isit.2019.8849405
We study the problem of estimating a highdimensional sparse covariance matrix, Σ_0, from a finite number of quadratic measurements, i.e., measurements a^T_iΣ_0_ai which are quadratic forms in the measurement vectors a i resulting from the covariance matrix, Σ_0. Such a problem arises in applications where we can only make energy measurements of the underlying random variables. We study a simple LASSOlike convex recovery algorithm which involves a squared 2norm (to match the covariance estimate to the measurements), plus a regularization term (that penalizes the ℓ_1−norm of the nondiagonal entries of Σ_0 to enforce sparsity). When the measurement vectors are i.i.d. Gaussian, we obtain the precise error performance of the algorithm (accurately determining the estimation error in any metric, e.g., 2norm, operator norm, etc.) as a function of the number of measurements and the underlying distribution of Σ_0. In particular, in the noiseless case we determine the necessary and sufficient number of measurements required to perfectly recover Σ_0 as a function of its sparsity. Our results rely on a novel comparison lemma which relates a convex optimization problem with "quadratic Gaussian" measurements to one which has i.i.d. Gaussian measurements.https://authors.library.caltech.edu/records/f03mwjfy33NonNegative Matrix Factorization via LowRank Stochastic Manifold Optimization
https://resolver.caltech.edu/CaltechAUTHORS:20191004100332012
Authors: Douik, Ahmed; Hassibi, Babak
Year: 2019
DOI: 10.1109/isit.2019.8849441
Several realworld applications, notably in nonnegative matrix factorization, graphbased clustering, and machine learning, require solving a convex optimization problem over the set of stochastic and doubly stochastic matrices. A common feature of these problems is that the optimal solution is generally a lowrank matrix. This paper suggests reformulating the problem by taking advantage of the lowrank factorization X = UV^T and develops a Riemannian optimization framework for solving optimization problems on the set of lowrank stochastic and doubly stochastic matrices. In particular, this paper introduces and studies the geometry of the lowrank stochastic multinomial and the doubly stochastic manifold in order to derive firstorder optimization algorithms. Being carefully designed and of lower dimension than the original problem, the proposed Riemannian optimization framework presents a clear complexity advantage. The claim is attested through numerical experiments on realworld and synthetic data for Nonnegative Matrix Factorization (NFM) applications. The proposed algorithm is shown to outperform, in terms of running time, stateoftheart methods for NFM.https://authors.library.caltech.edu/records/cj5nvp7339Gabidulin Codes with Support Constraints
https://resolver.caltech.edu/CaltechAUTHORS:20200214103053761
Authors: Yildiz, Hikmet; Hassibi, Babak
Year: 2019
DOI: 10.1109/ITW44776.2019.8988992
Gabidulin codes are the first general construction of linear codes that are maximum rank distance (MRD). They have found applications in linear network coding, for example, when the transmitter and receiver are oblivious to the inner workings and topology of the network (the socalled incoherent regime). The reason is that Gabidulin codes can be used to map information to linear subspaces, which in the absence of errors cannot be altered by linear operations, and in the presence of errors can be corrected if the subspace is perturbed by a small rank. Furthermore, in distributed coding and distributed systems, one is led to the design of error correcting codes whose generator matrix must satisfy a given support constraint. In this paper, we give necessary and sufficient conditions on the support of the generator matrix that guarantees the existence of Gabidulin codes and general MRD codes. When the rate of the code is not very high, this is achieved with the same field size necessary for Gabidulin codes with no support constraint. When these conditions are not satisfied, we characterize the largest possible rank distance under the support constraints and show that they can be achieved by subcodes of Gabidulin codes. The necessary and sufficient conditions are identical to those that appear for MDS codes which were recently proven in [1], [2] in the context of settling the GMMDS conjecture.https://authors.library.caltech.edu/records/xsazvw0t67A Novel Riemannian Optimization Approach and Algorithm for Solving the Phase Retrieval Problem
https://resolver.caltech.edu/CaltechAUTHORS:20200402144617818
Authors: Douik, Ahmed; Salehi, Fariborz; Hassibi, Babak
Year: 2019
DOI: 10.1109/ieeeconf44664.2019.9049040
Several imaging applications require constructing the phase of a complex signal given observations of its amplitude. In most applications, a subset of phaseless measurements, say the discrete Fourier transform of the signal, form an orthonormal basis that can be exploited to speed up the recovery. This paper suggests a novel Riemannian optimization approach for solving the Fourier phase retrieval problem by studying and exploiting the geometry of the problem to reduce the ambient dimension and derive extremely fast and accurate algorithms. The phase retrieval problem is reformulated as a constrained problem and a novel Riemannian manifold, referred to as the fixednorms manifold, is introduced to represent all feasible solutions. The firstorder geometry of the Riemannian manifold is derived in closedform which allows the design of highly efficient optimization algorithms. Numerical simulations indicate that the proposed approach outperforms conventional optimizationbased methods both in accuracy and in convergence speed.https://authors.library.caltech.edu/records/57myhemy58Universality in Learning from Linear Measurements
https://resolver.caltech.edu/CaltechAUTHORS:20190628083750345
Authors: Abbasi, Ehsan; Salehi, Fariborz; Hassibi, Babak
Year: 2019
DOI: 10.48550/arXiv.1906.08396
We study the problem of recovering a structured signal from independently and identically drawn linear measurements. A convex penalty function f(⋅) is considered which penalizes deviations from the desired structure, and signal recovery is performed by minimizing f(⋅) subject to the linear measurement constraints. The main question of interest is to determine the minimum number of measurements that is necessary and sufficient for the perfect recovery of the unknown signal with high probability. Our main result states that, under some mild conditions on f(⋅) and on the distribution from which the linear measurements are drawn, the minimum number of measurements required for perfect recovery depends only on the first and second order statistics of the measurement vectors. As a result, the required of number of measurements can be determining by studying measurement vectors that are Gaussian (and have the same mean vector and covariance matrix) for which a rich literature and comprehensive theory exists. As an application, we show that the minimum number of random quadratic measurements (also known as rankone projections) required to recover a low rank positive semidefinite matrix is 3nr, where n is the dimension of the matrix and r is its rank. As a consequence, we settle the long standing open question of determining the minimum number of measurements required for perfect signal recovery in phase retrieval using the celebrated PhaseLift algorithm, and show it to be 3n.https://authors.library.caltech.edu/records/942t76dg78The Impact of Regularization on Highdimensional Logistic Regression
https://resolver.caltech.edu/CaltechAUTHORS:20190628084529981
Authors: Salehi, Fariborz; Abbasi, Ehsan; Hassibi, Babak
Year: 2019
DOI: 10.48550/arXiv.1906.03761
Logistic regression is commonly used for modeling dichotomous outcomes. In the classical setting, where the number of observations is much larger than the number of parameters, properties of the maximum likelihood estimator in logistic regression are well understood. Recently, Sur and Candes have studied logistic regression in the highdimensional regime, where the number of observations and parameters are comparable, and show, among other things, that the maximum likelihood estimator is biased. In the highdimensional regime the underlying parameter vector is often structured (sparse, blocksparse, finitealphabet, etc.) and so in this paper we study regularized logistic regression (RLR), where a convex regularizer that encourages the desired structure is added to the negative of the loglikelihood function. An advantage of RLR is that it allows parameter recovery even for instances where the (unconstrained) maximum likelihood estimate does not exist. We provide a precise analysis of the performance of RLR via the solution of a system of six nonlinear equations, through which any performance metric of interest (mean, meansquared error, probability of support recovery, etc.) can be explicitly computed. Our results generalize those of Sur and Candes and we provide a detailed study for the cases of ℓ²₂RLR and sparse (ℓ₁regularized) logistic regression. In both cases, we obtain explicit expressions for various performance metrics and can find the values of the regularizer parameter that optimizes the desired performance. The theory is validated by extensive numerical simulations across a range of parameter values and problem instances.https://authors.library.caltech.edu/records/p1b4520e41A Stochastic Interpretation of Stochastic Mirror Descent: RiskSensitive Optimality
https://resolver.caltech.edu/CaltechAUTHORS:20190628084200896
Authors: Azizan, Navid; Hassibi, Babak
Year: 2019
DOI: 10.1109/CDC40024.2019.9030229
Stochastic mirror descent (SMD) is a fairly new family of algorithms that has recently found a wide range of applications in optimization, machine learning, and control. It can be considered a generalization of the classical stochastic gradient algorithm (SGD), where instead of updating the weight vector along the negative direction of the stochastic gradient, the update is performed in a "mirror domain" defined by the gradient of a (strictly convex) potential function. This potential function, and the mirror domain it yields, provides considerable flexibility in the algorithm compared to SGD. While many properties of SMD have already been obtained in the literature, in this paper we exhibit a new interpretation of SMD, namely that it is a risksensitive optimal estimator when the unknown weight vector and additive noise are nonGaussian and belong to the exponential family of distributions. The analysis also suggests a modified version of SMD, which we refer to as symmetric SMD (SSMD). The proofs rely on some simple properties of Bregman divergence, which allow us to extend results from quadratics and Gaussians to certain convex functions and exponential families in a rather seamless way.https://authors.library.caltech.edu/records/dr2tqpaf91NonNegative Matrix Factorization via LowRank Stochastic Manifold Optimization
https://resolver.caltech.edu/CaltechAUTHORS:2022011276592900
Authors: Douik, Ahmed; Hassibi, Babak
Year: 2020
DOI: 10.1109/ita50056.2020.9244937
Several realworld applications, notably in nonnegative matrix factorization, graphbased clustering, and machine learning, require solving a convex optimization problem over the set of stochastic and doubly stochastic matrices. A common feature of these problems is that the optimal solution is generally a lowrank matrix. This paper suggests reformulating the problem by taking advantage of the lowrank factorization X = UV^T and develops a Riemannian optimization framework for solving optimization problems on the set of lowrank stochastic and doubly stochastic matrices. In particular, this paper introduces and studies the geometry of the lowrank stochastic multinomial and the doubly stochastic manifold in order to derive firstorder optimization algorithms. Being carefully designed and of lower dimension than the original problem, the proposed Riemannian optimization framework presents a clear complexity advantage. The claim is attested through numerical experiments on realworld and synthetic data for Nonnegative Matrix Factorization (NFM) applications. The proposed algorithm is shown to outperform, in terms of running time, stateoftheart methods for NFM.https://authors.library.caltech.edu/records/vchgqcfq23A Study of Generalization of Stochastic Mirror Descent Algorithms on Overparameterized Nonlinear Models
https://resolver.caltech.edu/CaltechAUTHORS:20200417131039768
Authors: Azizan, Navid; Lale, Sahin; Hassibi, Babak
Year: 2020
DOI: 10.1109/icassp40776.2020.9053864
We study the convergence, the implicit regularization and the generalization of stochastic mirror descent (SMD) algorithms in overparameterized nonlinear models, where the number of model parameters exceeds the number of training data points. Due to overparameterization, the training loss has infinitely many global minima where they define a manifold of interpolating solutions. To have an understanding of the generalization performance of SMD algorithms, it is important to characterize which global minima the SMD algorithms converge to. In this work, we first theoretically show that in the overparameterized nonlinear setting, if the initialization is close enough to the manifold of global minima, which is usually the case in the high overparameterization setting, using sufficiently small step size, SMD converges to a global minimum. We further prove that this global minimum is approximately the closest one to the initialization in Bregman divergence, demonstrating the approximate implicit regularization of SMD. We then empirically confirm that these theoretical results are observed in practice. Finally, we provide an extensive study of the generalization of SMD algorithms. In our experiments, we show that on the CIFAR10 dataset, SMD with ℓ₁₀ norm potential (as a surrogate for ℓ∞ ) consistently generalizes better than SGD (corresponding to an ℓ₂ norm potential), which in turn consistently outperforms SMD with ℓ₁ norm potential.https://authors.library.caltech.edu/records/985cch3h76Support Constrained Generator Matrices of Gabidulin Codes in Characteristic Zero
https://resolver.caltech.edu/CaltechAUTHORS:20200526154649443
Authors: Yildiz, Hikmet; Raviv, Netanel; Hassibi, Babak
Year: 2020
DOI: 10.1109/ISIT44484.2020.9174524
Gabidulin codes over fields of characteristic zero were recently constructed by Augot et al., whenever the Galois group of the underlying field extension is cyclic. In parallel, the interest in sparse generator matrices of Reed–Solomon and Gabidulin codes has increased lately, due to applications in distributed computations. In particular, a certain condition pertaining to the intersection of zero entries at different rows, was shown to be necessary and sufficient for the existence of the sparsest possible generator matrix of Gabidulin codes over finite fields. In this paper we complete the picture by showing that the same condition is also necessary and sufficient for Gabidulin codes over fields of characteristic zero.Our proof builds upon and extends tools from the finitefield case, combines them with a variant of the Schwartz–Zippel lemma over automorphisms, and provides a simple randomized construction algorithm whose probability of success can be arbitrarily close to one. In addition, potential applications for lowrank matrix recovery are discussed.https://authors.library.caltech.edu/records/04v49bah07Stabilizing Dynamical Systems with FixedRate Feedback using Constrained Quantizers
https://resolver.caltech.edu/CaltechAUTHORS:20200831150243516
Authors: Sabag, Oron; Kostina, Victoria; Hassibi, Babak
Year: 2020
DOI: 10.1109/isit44484.2020.9173929
The stabilization of unstable dynamical systems using ratelimited feedback links is investigated. In the scenario of a constantrate link and a noise with unbounded support, the fundamental limit of communication is known, but no simple algorithm to achieve it exists. The main challenge in constructing an optimal scheme is to fully exploit the communication resources while occasionally signaling the controller that a special operation needs to be taken due to a large noise observation. In this work, we present a simple and explicit algorithm that stabilizes the dynamical system and achieves the fundamental limits of communication. The new idea is to use a constrained quantizer in which certain patterns of sequences are avoided throughout the quantization process. These patterns are preserved to signal the controller that a zoomout operation should be initiated due to large noise observation. We show that the constrained quantizer has a negligible effect on the rate, so it achieves the fundamental limit of communication. Specifically, the rateoptimal algorithm is shown to stabilize any βmoment of the state if the noise has a bounded absolute (β +ϵ)moment for some ϵ > 0 regardless of the other noise characteristics.https://authors.library.caltech.edu/records/nm80zqzc03Fundamental limits of distributed tracking
https://resolver.caltech.edu/CaltechAUTHORS:20191213102421150
Authors: Kostina, Victoria; Hassibi, Babak
Year: 2020
DOI: 10.1109/ISIT44484.2020.9174006
Consider the following communication scenario. An ndimensional source with memory is observed by K isolated encoders via parallel channels, who causally compress their observations to transmit to the decoder via noiseless rateconstrained links. At each time instant, the decoder receives K new codewords from the observers, combines them with the past received codewords, and produces a minimum distortion estimate of the latest block of n source symbols. This scenario extends the classical oneshot CEO problem to multiple rounds of communication with communicators maintaining memory of the past.We prove a coding theorem showing that the minimum asymptotically (as n → ∞) achievable sum rate required to achieve a target distortion is equal to the directed mutual information from the observers to the decoder minimized subject to the distortion constraint and the separate encoding constraint. For the GaussMarkov source observed via K parallel AWGN channels, we solve that minimal directed mutual information problem, thereby establishing the minimum asymptotically achievable sum rate. Finally, we explicitly bound the rate loss due to a lack of communication among the observers; that bound is attained with equality in the case of identical observation channels.The general coding theorem is proved via a new nonasymptotic bound that uses stochastic likelihood coders and whose asymptotic analysis yields an extension of the BergerTung inner bound to the causal setting. The analysis of the Gaussian case is facilitated by reversing the channels of the observers.https://authors.library.caltech.edu/records/wqemhtva22The Minimal Directed Information Needed to Improve the LQG Cost
https://resolver.caltech.edu/CaltechAUTHORS:20210121152557490
Authors: Sabag, Oron; Tian, Peida; Kostina, Victoria; Hassibi, Babak
Year: 2020
DOI: 10.1109/cdc42340.2020.9304490
We study a linear quadratic Gaussian (LQG) control problem, in which a noisy observation of the system state is available to the controller. To lower the achievable LQG cost, we introduce an extra communication link from the system to the controller. We investigate the tradeoff between the improved LQG cost and the consumed communication (information) resources that are measured with the conditional directed information. The objective is to minimize the directed information over all encodingdecoding policies subject to a constraint on the LQG cost. The main result is a semidefinite programming formulation for the optimization problem in the finitehorizion scenario where the dynamical system may have timevarying parameters. This result extends the seminal work by Tanaka et al., where the direct noisy measurement of the system state at the controller is assumed to be absent. As part of our derivation to show the optimality of an encoder that transmits a Gaussian measurement of the state, we show that the presence of the noisy measurements at the encoder can not reduce the minimal directed information, extending a prior result of Kostina and Hassibi to the vector case. Finally, we show that the results in the finitehorizon case can be extended to the infinitehorizon scenario when assuming a timeinvariant system, but possibly a timevarying policy. We show that the solution for this optimization problem can be realized by a timeinvariant policy whose parameters can be computed explicitly from a finitedimensional semidefinite program.https://authors.library.caltech.edu/records/kq0fnm2245Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems
https://resolver.caltech.edu/CaltechAUTHORS:20221222222544264
Authors: Lale, Sahin; Azizzadenesheli, Kamyar; Hassibi, Babak; Anandkumar, Anima
Year: 2020
We study the problem of adaptive control in partially observable linear dynamical systems. We propose a novel algorithm, adaptive control online learning algorithm (AdaptOn), which efficiently explores the environment, estimates the system dynamics episodically and exploits these estimates to design effective controllers to minimize the cumulative costs. Through interaction with the environment, AdaptOn deploys online convex optimization to optimize the controller while simultaneously learning the system dynamics to improve the accuracy of controller updates. We show that when the cost functions are strongly convex, after T times step of agentenvironment interaction, AdaptOn achieves regret upper bound of polylog(T). To the best of our knowledge, AdaptOn is the first algorithm which achieves polylog(T) regret in adaptive control of unknown partially observable linear dynamical systems which includes linear quadratic Gaussian (LQG) control.https://authors.library.caltech.edu/records/96mdj7sz39RegretOptimal Controller for the FullInformation Problem
https://resolver.caltech.edu/CaltechAUTHORS:20210719210213488
Authors: Sabag, Oron; Goel, Gautam; Lale, Sahin; Hassibi, Babak
Year: 2021
DOI: 10.23919/ACC50511.2021.9483023
We consider the infinitehorizon, discretetime fullinformation control problem. Motivated by learning theory, as a criterion for controller design we focus on regret, defined as the difference between the linear quadratic regulator (LQR) cost of a causal controller (that has only access to past and current disturbances) and the LQR cost of a clairvoyant one (that has also access to future disturbances). In the fullinformation setting, there is a unique optimal noncausal controller that in terms of LQR cost dominates all other controllers, and we focus on the regret compared to this particular controller. Since the regret itself is a function of the disturbances, we consider the worstcase regret over all possible bounded energy disturbances, and propose to find a causal controller that minimizes this worstcase regret. The resulting controller has the interpretation of guaranteeing the smallest possible regret compared to the best noncausal controller that has can see the future, no matter what the disturbances are. We show that the regretoptimal control problem can be reduced to a Nehari extension problem, i.e., to approximate an anticausal operator with a causal one in the operator norm. In the statespace setting we obtain explicit formulas for the optimal regret and for the regretoptimal controller. The regretoptimal controller is the sum of the classical H₂ control law and an nth order controller (where n is the state dimension of the plant) obtained from the Nehari problem. The controller construction simply requires the solution to the standard LQR Riccati equation, in addition to two Lyapunov equations. Simulations over a range of plants demonstrates that the regretoptimal controller interpolates nicely between the H₂ and the H∞ optimal controllers, and generally has H₂ and H∞ costs that are simultaneously close to their optimal values. The regretoptimal controller thus presents itself as a viable option for control system design.https://authors.library.caltech.edu/records/4wmpjhay73Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting
https://resolver.caltech.edu/CaltechAUTHORS:20200403141835981
Authors: Lale, Sahin; Azizzadenesheli, Kamyar; Hassibi, Babak; Anandkumar, Anima
Year: 2021
DOI: 10.23919/ACC50511.2021.9483309
We study the problem of adaptive control in partially observable linear quadratic Gaussian control systems, where the model dynamics are unknown a priori. We propose LQGOPT, a novel adaptive control algorithm based on the principle of optimism in the face of uncertainty, to effectively minimize the overall control cost. We employ the predictor state evolution representation of the system dynamics and deploy a recently proposed closedloop system identification method, estimation, and confidence bound construction. LQGOPT efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model for further exploration and exploitation. We provide stability guarantees for LQGOPT, and prove the first Õ(√T) regret upper bound for adaptive control of linear quadratic Gaussian (LQG) systems with convex cost, where T is the time horizon of the problem.https://authors.library.caltech.edu/records/48g0qz7v45Feedback Capacity of MIMO Gaussian Channels
https://resolver.caltech.edu/CaltechAUTHORS:20210719210210078
Authors: Sabag, Oron; Kostina, Victoria; Hassibi, Babak
Year: 2021
DOI: 10.1109/ISIT45174.2021.9518088
Finding a computable expression for the feedback capacity of channels with nonwhite Gaussian, additive noise is a long standing open problem. In this paper, we solve this problem in the scenario where the channel has multiple inputs and multiple outputs (MIMO) and the noise process is generated as the output of a statespace model (a hidden Markov model). The main result is a computable characterization of the feedback capacity as a finitedimensional convex optimization problem. Our solution subsumes all previous solutions to the feedback capacity including the autoregressive movingaverage (ARMA) noise process of first order, even if it is a nonstationary process. The capacity problem can be viewed as the problem of maximizing the measurements' entropy rate of a controlled (policydependent) statespace subject to a power constraint. We formulate the finiteblock version of this problem as a sequential convex optimization problem, which in turn leads to a singleletter and computable upper bound. By optimizing over a family of timeinvariant policies that correspond to the channel inputs distribution, a tight lower bound is realized. We show that one of the optimization constraints in the capacity characterization boils down to a Riccati equation, revealing an interesting relation between explicit capacity formulae and Riccati equations.https://authors.library.caltech.edu/records/ypd9pp4p74Differentially Quantized Gradient Descent
https://resolver.caltech.edu/CaltechAUTHORS:20200214105624458
Authors: Lin, ChungYi; Kostina, Victoria; Hassibi, Babak
Year: 2021
DOI: 10.1109/ISIT45174.2021.9518254
Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay constraints. The only information that the server knows about the problem instance is what it receives from the worker via a ratelimited noiseless communication channel. We introduce the technique we call differential quantization (DQ) that compensates past quantization errors to make the descent trajectory of a quantized algorithm follow that of its unquantized counterpart. Assuming that the objective function is smooth and strongly convex, we prove that differentially quantized gradient descent (DQGD) attains a linear convergence rate of max{σ_(GD), ρ_n2^(R)}, where σ_(GD) is the convergence rate of unquantized gradient descent (GD), ρ_n is the covering efficiency of the quantizer, and R is the bitrate per problem dimension n. Thus at any R ≥ log₂ρ_n/σ_(GD), the convergence rate of DQGD is the same as that of unquantized GD, i.e., there is no loss due to quantization. We show a converse demonstrating that no GDlike quantized algorithm can converge faster than max{σ_(GD), 2^(R)}. Since quantizers exist with ρ_n → 1 as n → ∞ (Rogers, 1963), this means that DQGD is asymptotically optimal. In contrast, naively quantized GD where the worker directly quantizes the gradient attains only σ_(GD) + ρ_n2^(R). The technique of differential quantization continues to apply to gradient methods with momentum such as Nesterov's accelerated gradient descent, and Polyak's heavy ball method. For these algorithms as well, if the rate is above a certain threshold, there is no loss in convergence rate obtained by the differentially quantized algorithm compared to its unquantized counterpart. Experimental results on both simulated and realworld leastsquares problems validate our theoretical analysis.https://authors.library.caltech.edu/records/bf3gq1ke54Model Learning Predictive Control in Nonlinear Dynamical Systems
https://resolver.caltech.edu/CaltechAUTHORS:20220210721786000
Authors: Lale, Sahin; Azizzadenesheli, Kamyar; Hassibi, Babak; Anandkumar, Anima
Year: 2021
DOI: 10.1109/cdc45484.2021.9683670
We study the problem of online learning and control in partially observable nonlinear dynamical systems, where the model dynamics are unknown and the controlling agent has only access to the system outputs. We propose Model Learning Predictive Control (MLPC), an efficient online control framework that learns to control the unknown system and minimizes the overall control cost. MLPC employs Random Fourier Features (RFF) to represent the nonlinear system dynamics and learns the underlying system up to a confidence interval. Once a reliable estimate of the dynamics is obtained, MLPC deploys an MPC oracle with the estimated system dynamics for planning. MLPC occasionally updates the underlying model estimates and improves the accuracy and the effectiveness of the MPC policies. We derive a novel finitetime approximation error bound under RFF learning and provide stability guarantees for single trajectory online control. We show that MLPC attains O̅(T^(2/3)) regret after T time steps in online control of stable partially observable nonlinear systems against the controller that uses the same MPC oracle with the true system dynamics. We empirically demonstrate the performance of MLPC on the inverted pendulum task and show the flexibility of the proposed general framework via deploying different planning strategies for the controller design to achieve lowcost control policies.https://authors.library.caltech.edu/records/7dhjbk6x38Feedback Capacity of Gaussian Channels with Memory
https://resolver.caltech.edu/CaltechAUTHORS:20220804765679000
Authors: Sabag, Oron; Kostina, Victoria; Hassibi, Babak
Year: 2022
DOI: 10.1109/isit50566.2022.9834799
We consider the feedback capacity of a MIMO channel whose channel output is given by a linear statespace model driven by the channel inputs and a Gaussian process. The generality of our statespace model subsumes all previous studied models such as additive channels with colored Gaussian noise, and channels with an arbitrary dependence on previous channel inputs or outputs. The main result is a computable feedback capacity expression that is given as a convex optimization problem subject to a detectability condition. We demonstrate the capacity result on the autoregressive Gaussian noise channel, where we show that even a single timeinstance delay in the feedback reduces the feedback capacity significantly in the stationary regime. On the other hand, for large regression parameters, the feedback capacity can be achieved with delayed feedback. Finally, we show that the detectability condition is satisfied for scalar models and conjecture that it is true for MIMO models.https://authors.library.caltech.edu/records/x9vjf8fj59Asymptotic Distribution of Stochastic Mirror Descent Iterates in Average Ensemble Models
https://resolver.caltech.edu/CaltechAUTHORS:20230526662981000.5
Authors: Kargin, Taylan; Salehi, Fariborz; Hassibi, Babak
Year: 2023
DOI: 10.1109/icassp49357.2023.10096701
The stochastic mirror descent (SMD) algorithm is a general class of training algorithms that utilizes a mirror potential to influence the implicit bias of the training algorithm and includes stochastic gradient descent (SGD) as a special case. In this paper, we explore the performance of the SMD on meanfield ensemble models and generalize earlier results obtained for SGD. The evolution of the distribution of parameters is mapped to a continuous time process in the space of probability distributions. Our main result gives a nonlinear partial differential equation (PDE) to which the continuous time process converges in the asymptotic of large networks. The impact of the mirror potential appears through a multiplicative term that is equal to the inverse of its Hessian and defines a gradient flow over an appropriate Riemannian manifold. We provide numerical simulations which allow us to study and characterize the effect of the mirror potential on the performance of networks trained with SMD for some binary classification problems.https://authors.library.caltech.edu/records/zww2ks4408