Applied Harmonic Analysis and Data Processing

Massive data sets have their own architecture. Each data source has an inherent structure, which we should attempt to detect in order to utilize it for applications, such as denoising, clustering, anomaly detection, knowledge extraction, or classification. Harmonic analysis revolves around creating new structures for decomposition, rearrangement and reconstruction of operators and functions—in other words inventing and exploring new architectures for information and inference. Two previous very successful workshops on applied harmonic analysis and sparse approximation have taken place in 2012 and in 2015. This workshop was the an evolution and continuation of these workshops and intended to bring together world leading experts in applied harmonic analysis, data analysis, optimization, statistics, and machine learning to report on recent developments, and to foster new developments and collaborations. Mathematics Subject Classification (2010): 42-XX, 65Txx, 94Axx, 65K05, 15A52. Introduction by the Organisers The workshop Applied Harmonic Analysis and Data Processing was organized by Ingrid Daubechies (Durham), Gitta Kutyniok (Berlin), Holger Rauhut (Aachen) and Thomas Strohmer (Davis). This meeting was attended by 49 participants from three continents. Advances in technology and the ever-growing role of digital sensors and computers in science have led to an exponential growth in the amount and complexity of data we collect. Uncertainty, scale, non-stationarity, noise, and heterogeneity are fundamental issues impeding progress at all phases of the pipeline that creates knowledge from data. This means that the amount of new 724 Oberwolfach Report 14/2018 mathematical challenges arising from the need of data analysis and information processing is enormous, with their solution requiring fundamentally new ideas and approaches, with significant consequences in the practical applications. Applied Harmonic Analysis provides a range of techniques towards the problem of efficiently representing, analyzing, compressing, and processing with “Big Data”. Massive data sets have their own architecture. Each data source has an inherent structure, which we should attempt to detect in order to utilize it for applications, such as denoising, clustering, anomaly detection, knowledge extraction, recovery, etc. Harmonic analysis revolves around creating new structures for decomposition, rearrangement and reconstruction of operators and functions—in other words inventing and exploring new architectures for information and inference. Indeed, in the last three decades Applied Harmonic Analysis has been at the center of many significant new ideas and methods crucial in a wide range of signal and image processing applications, and in the analysis and processing of large data sets. For example, compressive sensing, sparse approximations and models, geometric multiscale analysis and diffusion geometry represent some quite recent important breakthroughs. Several new directions have emerged on the heels of compressive sensing: Lowrank matrix recovery aims at recovering a matrix with small rank from incomplete data. In particular, matrix completion recovers the matrix from only a small fraction of its entries. Since low-rank structures arise in numerous applications, one can expect an enormous impact. However, much of the theory so far deals with linear measurements, while in practice we often also face non-linear measurements, for instance in situations where only signal intensity can be obtained. Despite recent breakthroughs in the area of phase retrieval, many challenging mathematical problems problems remain open in these areas. Inverse problems arising in connection with massive, complex data sets pose tremendous challenges and require new mathematical tools. Numerous deep questions arise. How can we utilize ideas of sparsity and minimal information complexity in this context? Is there a unified view of such measures that would include sparsity, lowrankness, and others (such as low-entropy), as special cases? This may lead to a new theory that considers an abstract notion of simplicity in general inverse problems. An important emerging topic in this context is the design efficient non-convex algorithms with provable convergence guarantees. One of the most exciting developments in machine learning in the past five years is the advent of deep learning, which is a special form of a neural network. Deep neural networks, and in particular convolutional networks have recently achieved state-of-the-art results on several complex computer vision and speech recognition tasks. However, until now deep learning acts very much like a black box, since algorithms are often based on ad hoc rules without theoretical foundation, the learned representations lack intepretability; we do not really understand why certain deep networks succeed and and we do not know how to modify them for those cases where they fail. Thus, developing a mathematical foundation for deep Applied Harmonic Analysis and Data Processing 725 learning is an important and rather challening task in data science, and one part of this workshop was dedicated to this topic. This workshop was a concerted effort to bring together researchers with various backgrounds, including harmonic analysis, optimization, probabibility theory, group theory, approximation theory, computer science, machine learning, and electrical engineering. The workshop featured 27 talks, thereof several longer overview talks. Moreover, a session of short presentations of 3 minutes took place on Monday, which we call the 3 Minutes of Fame (following Andy Warhols concept of 15 minutes of fame). This session has meanwhile become a tradition and has proven to be an efficient vehicle to ensure that every participant had the possibility to advertise her resarch. At the same time it is very entertaining for the audience. Almost all of the attendees participated, ranging from PhD students to renowned professors, contributing to the success of this session. Some highlights of the program included: • Advanced sampling theory: One of the problems that link harmonic analysis with data processing is the sampling problem. The main theoretical issue is how the stability of sampling and recovery is related to the number or density of samples. Related issues are the questions of localization, non-uniform sampling, and last not least suitable numerical algorithms. Karlheinz Gröchenig presented a range of compelling results using tools from shift-invariant spaces and totally positive functions. Albert Cohen discussed function approximation from sampling in high dimensions using optimal weighted least squares approximation. Felix Krahmer talked about “unlimited sampling”, a mathematical framework for sampling that can overcome limitations in current analog-to-digital converters. • Nonlinear inverse problems: In many applications we can only acquire nonlinear measurements of the function of interest. Phase retrieval is but the most prominent example. Several talks were dedicated to nonlinear inverse problems. Babak Hassibi and Rima Alaifari both presented recent progress in the solution of the phase retrieval problem, while Yuxin Chen and Justin Romberg highlighted exciting progress in convex and nonconvex optimization for certain nonlinear problems. • Emerging theory of Deep Learning: Despite the huge practical successes of Deep Learning in recent years, the mathematical understanding of deep learning is in its infancy. Several talks aimed at to remedy this situation. Philipp Grohs demonstrated how to avoid the curse of dimensionality when solving Kolmogorov equation in high dimensions by means of deep learning. Mahdi Soltanolkoltabi and Remi Gribonval were among several speakers who presented theoretical progress towards understanding some of the heuristics behind neural networks. The organizers would like to take the opportunity to thank MFO for providing support and a very inspiring environment for the workshop. The magic of the place and the pleasant atmosphere contributed greatly to the success of the workshop. 726 Oberwolfach Report 14/2018 Acknowledgement: The MFO and the workshop organizers would like to thank the National Science Foundation for supporting the participation of junior researchers in the workshop by the grant DMS-1641185, “US Junior Oberwolfach Fellows”. Applied Harmonic Analysis and Data Processing 727 Workshop: Applied Harmonic Analysis and Data Processing