The First Workshop on
Hybrid Human-Machine Learning and Decision Making


September 22, 2023

Turin, Italy


In the past, machine learning and decision-making have been treated as independent research areas. However, with the increasing emphasis on human-centered AI, there has been a growing interest in combining these two areas. Researchers have explored approaches that aim to complement human decision-making rather than replace it, as well as strategies that leverage machine predictions to improve overall decision-making performance.

Despite these advances, our understanding of this topic is still in its infancy and that there is much to be learned about the interplay between human and machine learning and decision making. To facilitate this exploration, there is a need for interdisciplinary events where researchers from multiple fields can come together to share their perspectives and insights.

The goal of this workshop is to bring together researchers with diverse backgrounds and expertise to explore effective hybrid machine learning and decision making. This will include approaches that explicitly consider the human-in-the-loop and the downstream goals of the human-machine system, as well as decision making strategies and HCI principles that promote rich and diverse interactions between humans and machines. Additionally, cognitive and legal aspects will be considered to identify potential pitfalls and ensure that trustworthy and ethical hybrid decision-making systems are developed.

Key Dates
  • Paper Submission Deadline: 12 June 2023 23 June 2023

  • Paper Author Notification: 12 July 2023

  • Workshop date: 22 September 2023

Invited Speakers
Prof. Marco Zenati

Chief of Cardiac Surgery and Director of the Medical Robotics & Computer-Assisted Surgery Lab at Harvard University . He pioneered the introduction of AI technology in surgical operating rooms. The research he leads has a direct impact on the clinical care he delivers on a daily basis.

See Marco's Webpage
Prof. Ece Kamar

Partner Research Area Manager at Microsoft Research and Affiliate Faculty with the University of Washington . She oversees the research area on human-centered AI, where she advance the state-of-the-art in Responsible AI, human-AI collaboration, sensing, signal processing, productivity, future of work and mental well-being.

See Ece's Webpage
Prof. Manuel Gomez Rodriguez

Tenured faculty at the Max Planck Institute for Software Systems in Germany. He is conducting cutting-edge research on improving decision-making through machine learning, and developing large-scale data mining methods for the analysis and modeling of large real-world networks and processes taking place over them.

See Manuel's Webpage
Prof. Sylvie Delacroix

Professor in Law and Ethics at the Birmingham Law School and Fellow of the Alan Turing Institute . She combines research on philosophy, ethics, law and regulation to address fundamental questions on acceptance and use of technology for individuals and society.

See Sylvie's Webpage
Prof. Alessandro Bozzon

Professor of Human-Centered Artificial Intelligence and Head of the Department of Sustainable Design Engineering at the Delft University of Technology. He is an expert on human-computer interaction, human computation, user modeling, and machine learning.

See Alessandro's Webpage
Call for Papers

The HLDM 2023 workshop aims at gathering together a diverse set of researchers addressing the different aspects that characterize effective hybrid decision-making. These range from machine learning approaches that explicitly account for the human-in-the-loop and the downstream goal of the human-machine system, to decision-making strategies and HCI principles encouraging a rich and diverse interaction between the human and the machine, to cognitive aspects pinpointing potential pitfalls, misunderstandings, and sub-optimal behavior, legal and regulatory aspects highlighting requirements and constraints that trustworthy and ethical hybrid decision making systems should satisfy.

We invite submissions on a broad range of topics revolving around hybrid human-machine learning and decision-making, including but not limited to:

The goal of the workshop is to foster discussion on the most promising research directions and the most relevant challenges revolving around hybrid human-machine learning and decision making. We thus accept the following types of submissions:

  1. Extended abstracts (4 pages + references) presenting work-in-progress, position papers, or open problems with clear and concise formulations of current challenges. Extended abstracts should be anonymized (double-blind review process) and formatted according to the ECMLPKDD 2023 guidelines. Accepted extended abstracts (no less than four pages overall) will be included in the Workshop proceedings of ECMLPKDD 2023, unless authors request otherwise.

  2. Research papers (14 pages + references) presenting novel original work not published elsewhere. Research papers should be anonymized (double-blind review process) and formatted according to the ECMLPKDD 2023 guidelines. Accepted research papers will be included in the Workshop proceedings of ECMLPKDD 2023 unless the authors explicitly check the opt-out option upon submission. Double-submission of research papers is forbidden unless the opt-out option is checked. In this latter case, the submission is considered non-archival.

  3. Resubmission of already accepted papers. The camera-ready version of the paper should be submitted (including author information), enriched with a cover page reporting information on where the paper has been accepted, and why it is of interest for the workshop. These submissions are non-archival.

We encourage all qualified candidates to submit a paper regardless of age, gender, sexual orientation, religion, country of origin, or ethnicity. All accepted papers will be presented as posters and linked to the workshop page. The best contributions will be allocated a 15 min presentation during the workshop to maximize their visibility and impact. Submitting a paper to the workshop means that if the paper is accepted at least one author should present the paper at the workshop.

Key Dates:

How to submit:

Go here and create a new submission for the “Towards Hybrid Human-Machine Learning and Decision Making (HLDM)” workshop.

Workshop Chairs
Andrea Passerini

Associate Professor at the Department of Information Engineering and Computer Science (DISI) of the University of Trento and Adjunct Professor at Aalborg University. He is director of the Structured Machine Learning Group and coordinator of the Research Program on Deep and Structured Machine Learning, both at DISI. His research interests include structured machine learning, neuro-symbolic integration, explainable and interactive machine learning, preference elicitation and learning with constraints.

See Andrea's Webpage
Fabio Casati

Principal Machine Learning Architect at Servicenow as well as technical lead for the AI Trust and Governance group in Servicenow research. Fabio focuses on designing, architecting and deploying AI-powered workflows for enterprise customers. He is working on AI applied to workflows and on quality in AI. He is also Professor at the University of Trento, working on crowdsourcing and hybrid human-machine computations, focusing on applications that have direct positive impact on society through tangible artefacts adopted by the community.

See Fabio's Webpage
Burcu Sayin

Postdoctoral Researcher at the Department of Information Engineering and Computer Science (DISI) of the University of Trento. Her research interests include hybrid intelligence, trustworthy AI, cost-sensitive machine learning, and active learning. Specifically, she works on cooperative human-machine intelligence.

See Burcu's Webpage
Anna Monreale

Associate Professor at the Department of Computer Science of the University of Pisa and an Adjunct Professor at the Faculty of Computer Science of the Dalhousie University. She is vice-coordinator of the National PhD in Artificial Intelligence for the Society of the University of Pisa. Her research interests include Big Data Analytics, Artificial Intelligence, Privacy-by-Design in big data and AI, and Explainable AI.

See Anna's Webpage
Roberto Pellungrini

Assistant Professor at Scuola Normale Superiore, Classe di Scienze. His research interests include Big Data Analytics, Data Privacy, and Explainable AI and currently working on Hybrid Decision Making algorithms.

See Roberto's Webpage
Paula Gürtler

Research Assistant in the Global Governance, Regulation, Innovation and Digital Economy (GRID) unit at CEPS. She has a background and research interest in Applied Ethics. In particular, she works on the Ethics of AI and the challenges of regulating it effectively.

See Paula's Webpage
Program Committee
Charalampos Z. Patrikakis

University of West Attica

Agathe Balayn

Delft University of Technology

Shiwali Mohan

Palo Alto Research Center

Addison Lin Wang

Hong Kong University of Science and Technology

Stefano Teso

University of Trento

Jie Yang

Delft University of Technology

Soumya Banerjee

University of Cambridge

Mutlu Cukurova

University College London

Giovanni De Toni

Fondazione Bruno Kessler, University of Trento

Roberto Dessi

Meta AI / Universitat Pompeu Fabra

Artur Bogucki

Centre for European Policy Studies

Yong Wang

Singapore Management University

Michele Lombardi

University of Bologna

Carlo Metta


Andrea Bontempelli

University of Trento

Salvatore Rinzivillo


Mattia Setzu

University of Pisa

Wolfgang Stammer

Technical University of Darmstadt

Matt Roach

Swansea University

Daniele Regoli

Intesa Sanpaolo

Tommaso Turchi

University of Pisa

Gaole He

Delft University of Technology


For any information please contact hldm-2023@unitn.it

Drop Us A Line