In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems.
- Date of publication:
- October 23, 2020
- IEEE Computer Graphics and Applications
- Page number(s):
- Issue Number:
- Publication note:
Nadia Boukhelifa, Anastasia Bezerianos, Remco Chang, Christopher Collins, Steven Mark Drucker, Alexander Endert, Jessica Hullman, Christopher L. North, Michael Sedlmair, Theresa-Marie Rhyne: Challenges in Evaluating Interactive Visual Machine Learning Systems. IEEE Computer Graphics and Applications 40(6): 88-96 (2020)