Learning from High-Order Data: Hypergraphs and Smart Codes - Pena, Karelia Pena (ProQuest Dissertations & Theses, 2023)
Currently, there is an increasing need to develop tools that allow the processing and exploitation of the massive amount of data available in many fields, such as computer vision, biology, social sciences, computational image, and others. The proposed research focuses on developing novel theories and algorithms that enable learning from data containing high-order inter-relationships. In particular, in data modeled by graphs and hypergraphs structures, the applications range from embedding images into QR codes to classification and denoising in a myriad of fields. Graph signal processing (GSP) techniques are powerful tools that model complex relationships within large datasets and are now used in a number of applications in different areas, including data science, communication networks, epidemiology, and sociology. Simple graphs, however, can only model pairwise relationships among data, preventing their application in modeling networks with higher-order relationships. In representation learning, for instance, considering high-order relationships in data has recently shown to be superior in many applications. For this reason, some efforts have been made to generalize well-known graph signal processing techniques to more complex graphs, such as hypergraphs, which allow for capturing higher-order relationships among data. In this dissertation, we provide a new hypergraph signal processing framework (t-HGSP) based on a novel tensor-tensor product algebra that has emerged as a powerful tool for preserving the intrinsic structures of tensors. The proposed framework allows the generalization of traditional GSP techniques while keeping the dimensionality characteristic of the complex systems represented by hypergraphs. To this end, the core elements of the t-HGSP framework are introduced, including the shifting operators and the hypergraph signal. The hypergraph Fourier space is also defined, followed by the concept of bandlimited signals and sampling. These tools are not only useful in the areas of hypergraph signal processing but also enable representation learning applications such as tensor-based hypergraph neural networks. Key to the success of hypergraph-based methods is having a meaningful hypergraph that may only be readily available for some applications. Nonetheless, having laid down the essential tools for t-HGSP opens the door for learning the hypergraph topology that dictates the higher-order relationships among the data. Hence, in this work, we also address the challenge of learning the underlying hypergraph topology from the data. As in graph signal processing applications, we consider the case in which the data possesses certain regularity or smoothness on the hypergraph. Given the hypergraph spectrum and frequency coefficient definitions provided by the t-HGSP framework, we propose a method to learn the hypergraph Laplacian from a set of smooth signals by minimizing their total variation on the hypergraph (TVL-HGSP). Additionally, we introduce an alternative approach (PDL-HGSP) that takes advantage of primal-dual-based algorithms and approximations to reduce time and space complexity. Finally, in representation learning, we depart from existing matrix representations of hypergraphs and combine the proposed learning algorithms with novel tensor-based hypergraph convolutional neural networks to propose hypergraph learning-convolutional neural networks (t-HyperGLNN). Compared to state-of-the-art, the learned adjacency tensor provides a more robust representation in high dimensions and the hypergraph signal model joint effects among connected nodes. Throughout this dissertation, we validate, theoretically and experimentally, that the proposed methods offer significant gains over the state-of-the-art in different applications, which include clustering, denoising, classification, and embedding of QR codes. For the latter, we proposed a fast generation of visually pleasant and robust QR codes. The proposed framework leverages the proposed hypergraph-based algorithms and state-of-the-art deep-learning algorithms to embed a color image into a baseline QR code in seconds while keeping a maximum error probability during the decoding procedure.