Fully Decentralized and Federated Low Rank Compressive Sensing

In this work we develop a fully decentralized, federated, and fast solution to the recently studied Low Rank Compressive Sensing (LRCS) problem: recover an n× q lowrank matrix X? = [x1,x ? 2, . . . ,x ? q] from column-wise linear projections, yk := Akxk , k = 1,2, . . . ,q, when each yk is an mlength vector with m < n. An important application where this problem occurs and a decentralized solution is desirable is in federated sketching: efficiently compressing the vast amounts of distributed images/videos generated by smartphones and various other devices while respecting the users’ privacy. Images from different devices, once grouped by category, are pretty similar and hence the matrix formed by the vectorized images of a certain category is well-modeled as being low rank. A simple federated sketching solution is to left multiply the kth vectorized image by a random matrix Ak and to store only yk. When m n, this requires much lesser storage than storing the full image, and is much faster to implement than traditional image compression. Suppose there are p nodes (say p smartphones), and each stores a set of (q/p) sketches of its images. We develop a decentralized projected gradient descent (GD) based approach to jointly reconstruct the images of all the phones/users from their respective stored sketches. The algorithm is such that the phones/users never share their raw data (their subset of yks) but only summaries of this data with the other phones at each algorithm iteration. Also, the reconstructed images of user g are obtained only locally. Other users cannot reconstruct them. Only the column span of the matrix X? is reconstructed globally. By “decentralized” we mean that there is no central node to which all nodes are connected and thus the only way to aggregate the summaries from the various nodes is by use of an iterative consensus algorithm that eventually provides an estimate of the aggregate at each node, as long as the network is strongly connected. We validated the effectiveness of our algorithm via extensive simulation experiments.