Our Research Focus in Responsible Data Science
Our primary mission is AI Safety, incorporating foundational computer science research with broader societal implications of AI.
We investigate methods to enhance the resilience of AI models against adversarial influences and dynamic environments. Our work includes:
- Uncertainty Quantification and Robustness: Measuring the confidence of AI models and ensuring the correctness of results.
- Preventing Adversarial Attacks: Understanding and mitigating adversarial threats to AI models.
- Factuality in Large Language Models: Assessing vulnerabilities and improving robustness in generative AI.
- Outlier and Changepoint Detection: Identifying distribution shifts and outliers.
- Visual Object Tracking: Developing reliable tracking systems under challenging real-world conditions.
Our research focuses on improving AI’s ability to adapt across diverse contexts and avoid reliance on superficial patterns. Topics include:
- Artificial General Intelligence (AGI): Exploring pathways toward more versatile and adaptable AI systems.
- Bias and Shortcuts: Investigating how AI models develop biases and designing methods to mitigate and counteract them.
- Multi-component Neuro-symbolic Generalization: Combining neural network learning with symbolic reasoning to build AI models with abstraction and logical inference capabilities.
We work on making AI systems more interpretable while ensuring data privacy and ethical compliance. Key areas include:
- Explainable AI: Enhancing model interpretability for users and stakeholders.
- Privacy: Preventing Membership Inference Attacks and improving the practicality of privacy preserving techniques, such as Differential Privacy.
- Machine Unlearning: Developing techniques to selectively remove learned information while preserving model performance.
- Synthetic Data Generation: Creating privacy-preserving realistic datasets for AI training and evaluation.
We apply AI research to critical domains, ensuring practical impact and responsible deployment. Our focus includes:
- AI for Finance: Improving risk assessment, fraud detection, and decision-making processes.
- AI for Healthcare: Enhancing diagnostics, patient care, and medical research through AI-driven insights.
- AI in Education: Integrating AI in educational settings and learning analytics.
- AI for Ethics: Refining LLMs to align with ethical guidelines, societal values, and user expectations.
News
2026
Research Publications
- AI & Ethics, Springer Nature 2026: The Safety Failures We Are Not Instrumenting: A Perspective on Hidden Safety-Critical Challenges in Modern AI Systems— Gjergji Kasneci and Enklejda Kasneci.
- NeurIPS 2026: Consolidating Rewarded Perturbations for LLM Post-Training — Zheyu Zhang et al.
- COLM 2026: Safety Cost of Steering Vectors Is Separable and Reducible — Yuxiao Li and Gjergji Kasneci.
- TMLR 2026 | 2 papers:
- ICML 2026 | 2 papers:
- Active Tabular Augmentation via Policy-Guided Diffusion Inpainting — Zheyu Zhang et al.
- Position: Sycophancy is an Educational Safety Risk — Why LLM Tutors Need Sycophancy Benchmarks — Enkelejda Kasneci and Gjergji Kasneci.
- FAccT 2026: Auditing LLM-Governed Social Robots with Culture-Specific Moral Gradients — Carmen Ng and Gjergji Kasneci.
- ACL 2026 | 3 papers:
- Analysing the Safety Pitfalls of Steering Vectors — Yuxiao Li et al.
- Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models — Chenchen Yuan et al.
- SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation — Shuo Yang et al.
- XAI 2026 | 2 papers:
- Explainable Federated Learning: Detecting Drift with Feature-Level Attribution Using SAGE — Rohit Jagtani et al.
- TabSCM: A Practical Framework for Generating Realistic Tabular Data — Sven Jacob et al.
- ICLR 2026: Reinforcement Unlearning via Group Relative Policy Optimization — Efstratios Zaradoukas et al.
Talks, People & Academic Highlights
- August | Invited Talk: Gjergji Kasneci presented Blueprints for Trustworthy Artificial Intelligence at the Nanyang Technological University (NTU), Singapore, invited by Dr. Teo Chew Lee, NIE-NTU
- June | Panel Discussion: Gjergji Kasneci joined fellow experts at the International Symposium on the Future of Learning to discuss responsible AI in education and associated challenges.
- February | Visiting Researcher: Davide Gabrielli (Sapienza University of Rome) joins RDS for a six-month research visit focusing on transformer invertibility.
January | Doctoral Defense: Stefan Kraft successfully defended his dissertation, Explainable AI for Clinical Decision Support, on 9 January 2026. Congratulations!
2025
Research Publications
- EMNLP 2025 | 4 papers:
- Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models — Zheyu Zhang et al.
- Doubling Your Data in Minutes: Ultra-fast Tabular Data Generation via LLM-Induced Dependency Graphs — Shuo Yang et al.
- From Confidence to Collapse in LLM Factual Robustness — Alina Fastowski et al.
- CURE: Controlled Unlearning for Robust Embeddings — Mitigating Conceptual Shortcuts in Pre-Trained Language Models — Aysenur Kocak et al.
- AIES 2025 | 4 papers:
- GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy — Jan Batzner et al.
- Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency — Jan Batzner et al.
- Adoption of Explainable Natural Language Processing: Perspectives from Industry and Academia on Practices and Challenges — Mahdi Dhaini et al.
- When Explainability Meets Privacy: An Investigation at the Intersection of Post-hoc Explainability and Differential Privacy in the Context of Natural Language Processing — Mahdi Dhaini et al.
- ACL Findings 2025: Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models — Chenchen Yuan et al.
- ICML 2025 | 4 papers:
- Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers — Roman Abramov et al.
- Graph Inverse Style Transfer for Counterfactual Explainability — Bardh Prenkaj et al.
- SCISSOR: Mitigating Semantic Bias through Cluster-Aware Siamese Networks for Robust Classification — Shuo Yang et al.
- Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents — Michael Kirchhof et al.
- FAccT 2025: Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods — Mahdi Dhaini et al.
- XAI 2025: EvalxNLP: A Framework for Benchmarking Post-Hoc Explainability Methods on NLP Models — Mahdi Dhaini et al.
- TMLR 2025: Attention Mechanisms Don’t Learn Additive Models: Rethinking Feature Importance for Transformers — Tobias Leemann et al.
Talks, Awards & Academic Highlights
- October | Keynote: Gjergji Kasneci presented Structure-Aware Actionable Recourse and Graph Counterfactuals at the ELLIS Winter School on Causality and Explainable AI in Paris.
- July | Doctoral Graduation: Xuan Zhao successfully completed his doctorate on 30 July 2025. Congratulations!
- June | Invited Talk: Gjergji Kasneci presented Blueprints for Trustworthy Artificial Intelligence at the University of Tokyo.
- June | Research Exchange: Bardh Prenkaj, Felix, and Alina gave invited talks at the University of Camerino, Italy, covering graph counterfactual explainability, artificial general intelligence, and adversarial attacks on LLMs.
- April | Visiting Researcher: Martina Zannotti (University of Camerino) joined RDS for a six-month research visit on spatio-temporal counterfactual explainability.
- March | Scholarship: Chenchen Yuan received the TUM EdTech Scholarship 2025.
- March | Doctoral Graduation: Tobias Leemann successfully defended his doctorate on 27 March 2025. Congratulations!
- March | Fellowship: Bardh Prenkaj received the Friedrich Schiedel Fellowship for Technology in Society for research on human-readable counterfactual explanations.
- February | Academic Service: Gjergji Kasneci served on the examination board for Italy's first national AI for Society doctorates, alongside Dino Pedreschi and Symeon Papadopoulos.
- February | Public Engagement: Gjergji Kasneci discussed AI developments and risks in social networks at the Munich Young Forum of the Munich Security Conference.
- January | Policy Contribution: A TUM collaboration led by Gjergji Kasneci published a blueprint for European technology leadership.