Wave-Informed Matrix Factorization withGlobal Optimality Guarantees

With the recent success of representation learning methods, which includes deep learning as a special case, there has been considerable interest in developing representation learning techniques that can incorporate known physical constraints into the learned representation. As one example, in many applications that involve a signal propagating through physical media (e.g., optics, acoustics, fluid dynamics, etc), it is known that the dynamics of the signal must satisfy constraints imposed by the wave equation. Here we propose a matrix factorization technique that decomposes such signals into a sum of components, where each component is regularized to ensure that it satisfies wave equation constraints. Although our proposed formulation is non-convex, we prove that our model can be efficiently solved to global optimality in polynomial time. We demonstrate the benefits of our work by applications in structural health monitoring, where prior work has attempted to solve this problem using sparse dictionary learning approaches that do not come with any theoretical guarantees regarding convergence to global optimality and employ heuristics to capture desired physical constraints. Representation learning has gained importance in fields that utilize data generated through physical processes, such as weather forecasting [1], manufacturing [2], structural health monitoring [3], acoustics [4], and medical imaging [5]. However, in general, the features learned through generic machine learning algorithms typically do not correspond to physically interpretable quantities. Yet, learning physically consistent and interpretable features can improve our understanding of 1) the physically viable information about the data and 2) the composition of the system or process generating it. As a result, in recent years modified learning paradigms, suited to different physical application domains, have begun to draw interest and discussion [6]. For example, several researchers have designed physics-informed neural networks to learn approximate solutions to a partial differential equation [7, 8, 9, 10, 11]. These have been recently applied to ultrasonic surface waves to extract velocity parameters and denoise data [12]. Similar physics-guided neural networks [6, 13, 14, 15] use physics-based regularizations to improve data processing, demonstrating that regression tasks (such as estimating the temperature throughout a lake) can be more accurate and robust when compared with purely physics-based solutions or purely data-driven solutions. These approaches demonstrate the strong potential for physics-informed machine learning but remain early in their study. Preprint. ar X iv :2 10 7. 09 14 4v 2 [ cs .L G ] 8 S ep 2 02 1 Likewise, several generative models of waves have recently been studied for applications in electroencephalography [16] and seismology [17], although these learning systems do not assume physical knowledge. Similarly, other prior work has also considered generative models of ultrasonic waves through the use of sparse signal processing [18], dictionary learning [19, 20, 21], and neural network sparse autoencoders [22]. These methods utilize the fact that many waves can be expressed as a small, sparse sum of spatial modes and attempt to extract these modes within the data [19]. These dictionary learning algorithms have further been combined with wave-informed regularizers to incorporate physical knowledge of wave propagation into the solution [23]. However, despite the wide-ranging applications for representation learning models for physicsconstrained problems, one of the key challenges associated with representation learning tasks is that they typically require one to solve a non-convex optimization problem. While this potentially presents considerable challenges due to the general difficulty of non-convex optimization, considerable attention has been devoted recently to the non-convex optimization problems that arise from representation learning problems, and despite non-convex optimization being hard in the general case, certain positive results have begun to emerge in certain settings. For example, within the context of low-rank matrix recovery, it has been shown that, in certain circumstances, gradient descent is guaranteed to converge to a local minimum and all local minima will be globally optimal [24, 25, 26, 27]. Further, other recent work has studied ‘structured’ matrix factorization problems, where one promotes various properties in the matrix factors by imposing some form of regularization on the factorized matrices to promote the desired properties (e.g., an `1 norm on a matrix factor when sparsity is desired in that factor) and shown that such problems can be solved to global optimality in certain circumstances, but whether such guarantees can be made depends critically on how the regularization on the factorized matrices is formulated in the model [28, 29, 30]. Paper Contributions and Related Work. In this work, we will specifically focus on data that is obtained from wave-like phenomena and note that many sensing and communications applications for representation learning center around the interpretation of waves and the wave equation to interrogate and characterize systems. For example, there are numerous applications for representation learning to aid in understanding and designing engineering systems with both electromagnetic waves (e.g., antenna design [31] and communication systems [32]) and mechanical waves (e.g., structural health monitoring [19] and micro-electromechanical device design [33]). Specifically, we derive a novel matrix factorization model which ensures that the recovered components satisfy wave equation constraints, allowing for direct interpretability of the recovered factors based on known physics and decomposing signals which consist of multiple oscillatory modes into the various components. Although our resulting formulation will also require solving a non-convex optimization problem, we show, using prior work on structured matrix factorization [28, 29, 30], that our model can be solved to global optimality in polynomial time. We demonstrate our approach’s advantages for two example applications, electrical waves in a multi-segment transmission line and mechanical waves on a fixed string. The first application is inspired from material characterization, where different materials have different wavenumbers, as arises in the health monitoring of transmission lines and structures [34, 12, 35]. The second application provides insights into how enforcing PDE constraints is beneficial in modal analysis, such as in the study of cantilever beams [36]. Prior work along these lines was given in [23], but in comparison, our algorithm provides global optimality guarantees (along with a certificate of guaranteed optimality), which is difficult to achieve for such non-convex problems. In addition, the formulation in [23] is incomplete, as it does not account for separability in deriving the formulation, which we motivate from separation of variables method. This work also differs in having a low-rank assumption on the matrix factorization which enables us to extend our method to matrix completion strategies on incomplete data. Further, [23] considers a dictionary learning approach of the data matrix after taking a Fourier transform to obtain a sparse frequency representation, whereas our approach can directly be applied in the time domain. 1 A Model for Wave-Informed Matrix Factorization In this section, we develop a novel matrix factorization framework which will allow us to decompose a matrix Y, with each column representing sampling along one dimension (e.g., space), as the product of two matrices DX>, where one of the matrices will be constrained to satisfy wave-equation constraints. Additionally, our model will learn the number of modes (columns in D and X) directly from the data without needing to specify this a priori.