Our work on how LLMs store relations selected as NeurIPS Spotlight paper

Our paper The Structure of Relation Decoding Linear Operators in Large Language Models by Miranda Anna Christ, Adrián Csiszárik, Gergely Becsó, and Dániel Varga was accepted at the NeurIPS 2025 conference as a Spotlight paper (~3% of submissions).

Healthcare

Prevention & Prediction

Patient Pathways

Patient Pathway Mission

Leveraging Hungary’s healthcare data assets for prevention, prediction, and decision support.

Global Sinkhorn Autoencoder — Optimal Transport on the latent representation of the full dataset

Publication
June 26, 2024
publications
We propose an Optimal Transport (OT)-based generative model from the Wasserstein Autoencoder (WAE) family of models, with the following innovative property: the optimization of the latent point positions takes place over the full training dataset rather than over a minibatch. Our contributions are the following: We define a new class of global Wasserstein Autoencoder models, and implement an Optimal Transport-based incarnation we call the Global Sinkhorn Autoencoder. We implement several metrics for evaluating such models, both in the unsupervised setting, and in a semi-supervised setting, which are the following: the global OT loss, which measures the OT loss on the full test dataset; the reconstruction error on the full test dataset; a so-called covered area which measures how well the latent points are matched; and two types of clustering measures.
June 26, 2024

Convergence and Generalization

Event
April 3, 2024
events
I will introduce some theoretical results related to the convergence and generalization capabilities of neural networks, in light of articles published at the NeurIPS 2023 conference in December. The first part of the presentation will summarize some important, earlier results, and then it will cover numerous articles based on these, which were presented at NeurIPS. The aim of the presentation is to provide a comprehensive overview of the current state of the field and the currently popular research directions.
April 3, 2024

Knot theory and AI

Event
March 20, 2024
events
I will overview some applications of supervised and reinforcement learning methods to knot theory that might be useful in other areas of mathematics.
March 20, 2024

Mode Combinability: Exploring Convex Combinations of Permutation Aligned Models

Publication
February 23, 2024
publications
We explore element-wise convex combinations of two permutation-aligned neural network parameter vectors $\Theta_A$ and $\Theta_B$ of size $d$. We conduct extensive experiments by examining various distributions of such model combinations parametrized by elements of the hypercube and its vicinity. Our findings reveal that broad regions of the hypercube form surfaces of low loss values, indicating that the notion of linear mode connectivity extends to a more general phenomenon which we call mode combinability.
February 23, 2024
The Team
The AI group at the institute brings together experts with backgrounds in both industry and academia. We place equal emphasis on theoretical foundations, thorough experimentation, and practical applications. Our close collaboration ensures a continuous exchange of knowledge between scientific research and applied projects.
Balázs Szegedy
Mathematical Theory
Attila Börcs, PhD
NLP, Modeling, MLOps
Adrián Csiszárik
Representation Learning, Foundations
Győző Csóka
NLP, MLOps
Domonkos Czifra
NLP, Foundations
Botond Forrai
Modeling
Péter Kőrösi-Szabó
Modeling
Gábor Kovács
NLP, Modeling
Judit Laki, MD PhD
Healthcare
Márton Muntag
Time Series, NLP, Modeling
Dávid Terjék
Generalization, Mathematical Theory
Dániel Varga
Foundations, Computer aided proofs
Pál Zsámboki
Reinforcement Learning, Geometric Deep Learning
Zsolt Zombori
Formal Reasoning
Péter Ágoston
Combinatory, Geometry
Beatrix Mária Benkő
Representation Learning
Jakab Buda
NLP
Diego González Sánchez
Generalization, Mathematical Theory
Melinda F. Kiss
Representation Learning
Ákos Matszangosz
Topology, Foundations
Alex Olár
Foundations
Gergely Papp
Modeling
Open Positions
The Rényi AI group is actively recruiting both theorists and practitioners.
Announcement: July 1, 2025
Deadline: rolling
applications
Rényi Institute is seeking Machine Learning Engineers to join our AI Research & Development team. Preferred Qualifications: • MLOps experience (especially in cloud environments) • Industry experience working on ML solutions
Announcement: July 1, 2025
Deadline: rolling
theory, applications
Rényi Institute is seeking Research Scientists to join our AI Research & Development team. You will have the privilege to work at a renowned academic institute and do what you love: do research and publish in the field of machine learning / deep learning.
Rényi AI - Building bridges between mathematics and artificial intelligence.