Our Research Focus in Responsible Data Science
Our primary mission is AI Safety, incorporating foundational computer science research with broader societal implications of AI.
We investigate methods to enhance the resilience of AI models against adversarial influences and dynamic environments. Our work includes:
- Uncertainty Quantification and Robustness: Measuring the confidence of AI models and ensuring the correctness of results.
- Preventing Adversarial Attacks: Understanding and mitigating adversarial threats to AI models.
- Factuality in Large Language Models: Assessing vulnerabilities and improving robustness in generative AI.
- Outlier and Changepoint Detection: Identifying distribution shifts and outliers.
- Visual Object Tracking: Developing reliable tracking systems under challenging real-world conditions.
Our research focuses on improving AI’s ability to adapt across diverse contexts and avoid reliance on superficial patterns. Topics include:
- Artificial General Intelligence (AGI): Exploring pathways toward more versatile and adaptable AI systems.
- Bias and Shortcuts: Investigating how AI models develop biases and designing methods to mitigate and counteract them.
- Multi-component Neuro-symbolic Generalization: Combining neural network learning with symbolic reasoning to build AI models with abstraction and logical inference capabilities.
We work on making AI systems more interpretable while ensuring data privacy and ethical compliance. Key areas include:
- Explainable AI: Enhancing model interpretability for users and stakeholders.
- Privacy: Preventing Membership Inference Attacks and improving the practicality of privacy preserving techniques, such as Differential Privacy.
- Machine Unlearning: Developing techniques to selectively remove learned information while preserving model performance.
- Synthetic Data Generation: Creating privacy-preserving realistic datasets for AI training and evaluation.
We apply AI research to critical domains, ensuring practical impact and responsible deployment. Our focus includes:
- AI for Finance: Improving risk assessment, fraud detection, and decision-making processes.
- AI for Healthcare: Enhancing diagnostics, patient care, and medical research through AI-driven insights.
- AI in Education: Integrating AI in educational settings and learning analytics.
- AI for Ethics: Refining LLMs to align with ethical guidelines, societal values, and user expectations.
News
- 🚀 04/2025: Martina Zannotti from the University of Camerino has started her 6 month visiting period at RDS. She is working on Spatio-Temporal Counterfactual Explainability. Welcome Martina!
- 🥳 03/2025: Chechen Yuan won the TUM EdTech Scholarship 2025. Congratulations!
- 🥳 03/2025: Tobias Leemann graduated with magna cum laude on March 27th, 2025. Congratulations, Dr. Leemann!
- 🏦 03/2025: Bardh Prenkaj won the Friedrich Schiedel Fellowship for Technology in Society. He plans to work on human-readable counterfactual explanations. Congrats!
- 🤝 02/2025: Together with Dino Pedreschi (University of Pisa) and Symeon Papadopoulos (CERTH-ITI), Gjergji Kasneci served on the examination board of Italy’s first national AI for Society doctorates. Big🙏 to Salvatore Ruggieri and many other colleagues for the fantastic organization.
- 🎤 02/2025: At the Munich Young Forum of the Munich Security Conference, Gjergji Kasneci discussed with EU MPs Eva Maydell and Damian von Boeselager, as well as TV journalist Claus Kleber, emerging AI developments and the associated risks in social networks.
- 📣 01/2025: TUM-internal collaboration led by Gjergji Kasneci proposes a blueprint for EU's tech leadership, building on the momentum of the EU AI Action Summit.
- 📄 01/2025: Our paper Attention Mechanisms Don’t Learn Additive Models: Rethinking Feature Importance for Transformers by Tobias Leemann, Alina Fastowski, Felix Pfeiffer, and Gjergji Kasneci was accepted at the Transactions of Machine Learning Research (TMLR). It is available here.
- 📄 11/2024: Our paper “RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting” by Shuo Yang, Bardh Prenkaj, and Gjergji Kasneci has been accepted to AAAI 2025. A preprint is available here.
- 🚀 11/2024: Marco Piangerelli and his PhD student, Martina Zannotti, visited us to give talks on their most recent projects: “Unsupervised Explainable Condition Monitoring” and “Exploring Explainability in Spatio-Temporal Graph Neural Networks for Time Series”. We enjoyed it a lot, thanks for stopping by!
- 📣 07/2024: We are happy to announce that our paper "Unifying Evolution, Explanation, and Discernment: A Generative Approach for Dynamic Graph Counterfactuals" by Bardh Prenkaj, Mario Villaizán-Vallelado, Tobias Leemann and Gjergji Kasneci has been accepted as an Oral Presentation at KDD 2024. A preprint is available here.
- 📄 07/2024: Our paper "Understanding Knowledge Drift in LLMs through Misinformation" by Alina Fastowski and Gjergji Kasneci has been accepted at the DELTA workshop at KDD 2024. A preprint is available on arXiv.
- 🚀 07/2024: We had the pleasure to host Giovanni Stilo and Andrea D'Angelo at RDS to present their recent research: “Bias, Fairness, and Explainability: Building Trust in AI Systems” and “Machine Unlearning: Safeguarding Privacy and Security in AI Models”.
- ⚙️ 06/2024: The workshop “DELTA: Discovering Drift Phenomena in Evolving Landscape” proposed by Bardh Prenkaj has been accepted to KDD 2024.
- 🚀 06/2024: Tobias Leemann started a 14-week internship at AWS in New York🗽
- 📄 02/2024: Our paper "Crowdsourcing is breaking my bank! A self-supervised Method for Fine-tuning Pre-trained Language models using Proximal Policy Optimization" by Shuo Yang and Gjergji Kasneci has been accepted at LREC-COLING 2024. A preprint is available on arXiv.
- 📣 01/2024: Our work "I Prefer Not to Say: Protecting User Consent in Models with Optional Personal Data" by Tobias Leemann, Martin Pawelczyk, Christian Thomas Eberle, and Gjergji Kasneci was accepted as an Oral Presentation at AAAI 2024. A preprint is available on arXiv.
- 📄 09/2023: We are happy to announce that our paper "Gaussian Membership Inference Privacy" by Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci has been accepted for publication at NeurIPS 2023. A preprint is available on arXiv.