Georgios B. Giannakis

Kernel-driven and Learnable Self Supervision over Graphs

Abstract

Self-supervision (SeSu) has gained popularity for “data-hungry” training of machine learning models, especially those involving large-scale graphs, where labeled samples are scarce or even unavailable. Main learning tasks in such setups are ill posed, and SeSu renders them well posed by relying on abundant unlabeled data as input, to yield low-dimensional embeddings of a reference (auxiliary) model output. In this talk, we first outline SeSu approaches, specialized reference models, and their links with auto-encoders, regularization, semi-supervised, transfer, and multi-view learning; but also their challenges and opportunities when multi-layer graph topologies and multi-view data are present, when nodal features are absent, and when the ad hoc selection of a reference model yields embeddings not optimally designed for the downstream main learning task. Next, we present our novel SeSu approach which selects the reference model to output either a prescribed kernel, or a learnable weighted superposition of kernels from a prescribed dictionary. As a result, the learned embeddings offer a novel, reduced-dimensionality estimate of the basis kernel, and thus an efficient parametric estimate of the main learning function at hand that belongs to a reproducing kernel Hilbert space. If time allows, we will also cover online variants for dynamic settings, and regret analysis founded on the so-termed neural-tangent-kernel framework to assess how effectively the learned embeddings approximate the underlying optimal kernel(s). We will wrap up with numerical tests using synthetic and real datasets to showcase the merits of kernel-driven and learnable (KeLe) SeSu relative to alternatives. The real data will also compare KeLe-SeSu with auto-encoders and graph neural networks (GNNs), and further test KeLe-SeSu on reference maps with masked-inputs and predicted-outputs that are popular in large language models (LLMs).

Short bio

Georgios B. Giannakis received his Diploma in Electrical Engineering (EE) from the National Technical U. of Athens, Greece, 1981. From 1982 to 1986 he was with the U. of Southern California (USC), where he received his MSc. in EE, 1983, MSc. in Mathematics, 1986, and Ph.D. in EE, 1986. He was with the U. of Virginia from 1987 to 1998, and since 1999 he has been with the U. of Minnesota (UMN), where he held an Endowed Chair of Telecommunications, served as director of the Digital Technology Center 2008-2021, and since 2016 he holds a UMN Presidential Chair in ECE.

His interests span the areas of statistical learning, communications, and networking — subjects on which he has published more than 500 journal papers, 800 conference papers, 26 book chapters, 2 edited books and 2 research monographs. Current research focuses on data science with applications to IoT, and power networks. He is the (co-) inventor of 37 issued patents, and the (co-)recipient of 11 best journal paper awards from the IEEE Signal Processing (SP) and Communications Societies, including the G. Marconi Prize. He received the IEEE-SPS N. Wiener Society Award (2019); EURASIP’s A. Papoulis Society Award (2020); Technical Achievement Awards from the IEEE-SPS (2000) and from EURASIP (2005); the IEEE ComSoc Education Award (2019); and the IEEE Fourier Technical Field Award (2015). He is a member of the Academia Europaea, Greece’s Academy of Athens, and Fellow of the US National Academy of Inventors, the European Academy of Sciences, UK’s Royal Academy of Engineering, Life Fellow of IEEE, and EURASIP. He has served the IEEE in several posts, including that of a Distinguished Lecturer for the IEEE-SPS.