Our work on how LLMs store relations selected as NeurIPS Spotlight paper

Our paper The Structure of Relation Decoding Linear Operators in Large Language Models by Miranda Anna Christ, Adrián Csiszárik, Gergely Becsó, and Dániel Varga was accepted at the NeurIPS 2025 conference as a Spotlight paper (~3% of submissions).

Read the full story

Healthcare

Prevention & Prediction

Patient Pathways

Patient Pathway Mission

Leveraging Hungary’s healthcare data assets for prevention, prediction, and decision support.

Read the full story Tovább a teljes történethez

The table

All Blog posts Publications Events Grants Announcements

Convergence and Generalization

Event

April 3, 2024

events

I will introduce some theoretical results related to the convergence and generalization capabilities of neural networks, in light of articles published at the NeurIPS 2023 conference in December. The first part of the presentation will summarize some important, earlier results, and then it will cover numerous articles based on these, which were presented at NeurIPS. The aim of the presentation is to provide a comprehensive overview of the current state of the field and the currently popular research directions.

April 3, 2024

Knot theory and AI

Event

March 20, 2024

events

I will overview some applications of supervised and reinforcement learning methods to knot theory that might be useful in other areas of mathematics.

March 20, 2024

Mode Combinability: Exploring Convex Combinations of Permutation Aligned Models

Publication

February 23, 2024

publications

We explore element-wise convex combinations of two permutation-aligned neural network parameter vectors $\Theta_A$ and $\Theta_B$ of size $d$. We conduct extensive experiments by examining various distributions of such model combinations parametrized by elements of the hypercube and its vicinity. Our findings reveal that broad regions of the hypercube form surfaces of low loss values, indicating that the notion of linear mode connectivity extends to a more general phenomenon which we call mode combinability.

February 23, 2024

Training BERT Models to Carry Over a Coding System Developed on One Corpus to Another

Publication

February 22, 2024

publications

This paper describes how we train BERT models to carry over a coding system developed on the paragraphs of a Hungarian literary journal to another. The aim of the coding system is to track trends in the perception of literary translation around the political transformation in 1989 in Hungary. To evaluate not only task performance but also the consistence of the annotation, moreover, to get better predictions from an ensemble, we use 10-fold crossvalidation.

February 22, 2024

Previous 1 2 3 4 5 6 7 8 9 10 11 12 Next

The Team

The AI group at the institute brings together experts with backgrounds in both industry and academia. We place equal emphasis on theoretical foundations, thorough experimentation, and practical applications. Our close collaboration ensures a continuous exchange of knowledge between scientific research and applied projects.

All Theory Applications Former members

Balázs Szegedy

Mathematical Theory

Attila Börcs, PhD

NLP, Modeling, MLOps

Adrián Csiszárik

Representation Learning, Foundations

Győző Csóka

NLP, MLOps

Domonkos Czifra

NLP, Foundations

Botond Forrai

Modeling

Péter Kőrösi-Szabó

Modeling

Gábor Kovács

NLP, Modeling

Judit Laki, MD PhD

Healthcare

Márton Muntag

Time Series, NLP, Modeling

Dávid Terjék

Generalization, Mathematical Theory

Dániel Varga

Foundations, Computer aided proofs

Pál Zsámboki

Reinforcement Learning, Geometric Deep Learning

Zsolt Zombori

Formal Reasoning

Péter Ágoston

Combinatory, Geometry

Beatrix Mária Benkő

Representation Learning

Jakab Buda

NLP

Diego González Sánchez

Generalization, Mathematical Theory

Melinda F. Kiss

Representation Learning

Ákos Matszangosz

Topology, Foundations

Alex Olár

Foundations

Gergely Papp

Modeling

Open Positions

The Rényi AI group is actively recruiting both theorists and practitioners.

Machine Learning Engineer

Announcement: December 1, 2023

Deadline: rolling

applications

Rényi Institute is seeking Machine Learning Engineers to join our AI Research & Development team. Preferred Qualifications: • MLOps experience (especially in cloud environments) • Industry experience working on ML solutions

Research Scientist

Announcement: December 1, 2023

Deadline: rolling

theory, applications

Rényi Institute is seeking Research Scientists to join our AI Research & Development team. You will have the privilege to work at a renowned academic institute and do what you love: do research and publish in the field of machine learning / deep learning.

All Positions

Rényi AI - Building bridges between mathematics and artificial intelligence.