Tutorial on Graph Learning: Principles, Challenges, and Open Directions

This page provides information and materials about “Graph Learning: Principles, Challenges, and Open Directions”, following the tutorial presented at The 41st International Conference on Machine Learning (ICML) 2024 on the 22nd of July, 2024 (Vienna, Austria). See the official website for more details.

Organizers

Panel

Tutorial Overview

This tutorial provides a comprehensive introduction to graph learning and GNNs, covering foundational concepts and recent advancements in the field. We begin with an overview of traditional graph representation and embedding methods, and then focus on modern approaches such as Graph Neural Networks (GNNs), Message Passing Networks (MPNNs), and Graph Transformers (GTs). The second part delves into the expressivity and generalizability of current GNN architectures. We will explore what functions and tasks GNNs can learn, referencing recent research that connects GNN expressivity with the Weisfeiler-Lehman (WL) graph isomorphism test. We will also discuss the generalizability of MPNNs, including their VC dimension and implications for model performance. Next, we address key information-flow challenges in graph learning architectures, such as under-reaching, over-smoothing, and over-squashing. We will highlight recent research aimed at understanding and alleviating these issues, including graph rewiring techniques. The tutorial will conclude with a panel discussion on future directions in graph machine learning. We will explore the limitations of the GNNs, graph foundation models and we will discuss the potential for integrating graph learning with large language models (LLMs) to enhance reasoning and complex data analysis capabilities.

Outline

• Introduction to Graph Theory and Graph Neural Networks (GNNs). [Speaker: Ameya Velingker]
• Expressivity and Generalizability of GNNs. [Speaker: Ameya Velingker]
• Challenges and Limitations of GNNs. [Speaker: Adrian Arnaiz-Rodriguez and Ameya Velingker]
• Panel Discussion on Future Directions of Graph Learning.
  • Panelists: Michael Bronstein, Michael Galkin (online), Christopher Morris, Bryan Perozzi
  • Moderated by: Adrian Arnaiz-Rodriguez and Ameya Velingker

ICML 2024

ICML (International Conference on Machine Learning) is one of the premier academic conferences in the field of machine learning and artificial intelligence. Held annually, ICML brings together researchers, practitioners, and industry leaders from around the world to share cutting-edge research, discuss emerging trends, and explore innovative applications of machine learning. The conference features a diverse program including keynote speeches, oral presentations, poster sessions, workshops, and tutorials. ICML tutorials, in particular, offer in-depth explorations of specific topics in machine learning, presented by experts in the field. These tutorials provide attendees with valuable opportunities to gain comprehensive knowledge on both foundational and advanced concepts, making them an integral part of the conference’s educational offerings.