
Human-AI Partnership for Mixed-Initiative Fact-Checking [ACM UIST] - 100ideas
http://fcweb.pythonanywhere.com/
======
100ideas
_Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative
Fact-Checking An T. Nguyen, Aditya Kharosekar, Saumyaa Krishnan, Siddhesh
Krishnan, Elizabeth Tate, Byron C. Wallace, Matthew Lease_ (UIST '18: ACM User
Interface Software and Technology Symposium Session: Crowds and Human-AI
Partnership)

Abstract: Fact-checking, the task of assessing the veracity of claims, is an
important, timely, and challenging problem. While many automated fact-checking
systems have been recently proposed, the human side of the partnership has
been largely neglected: how might people understand, interact with, and
establish trust with an AI fact-checking system? Does such a system actually
help people better assess the factuality of claims? In this paper, we present
the design and evaluation of a mixed-initiative approach to fact-checking,
blending human knowledge and experience with the efficiency and scalability of
automated information retrieval and ML. In a user study in which participants
used our system to aid their own assessment of claims, our results suggest
that individuals tend to trust the system: participant accuracy assessing
claims improved when exposed to correct model predictions. However, this trust
perhaps goes too far: when the model was wrong, exposure to its predictions
often degraded human accuracy. Participants given the option to interact with
these incorrect predictions were often able improve their own performance.
This suggests that transparent models are key to facilitating effective human
interaction with fallible AI models.

video:
[https://www.youtube.com/watch?v=3GDA3jSzRgs](https://www.youtube.com/watch?v=3GDA3jSzRgs)

DOI:
[https://doi.org/10.1145/3242587.3242666](https://doi.org/10.1145/3242587.3242666)

PDF: [http://www.cs.utexas.edu/~atn/nguyen-
uist18.pdf](http://www.cs.utexas.edu/~atn/nguyen-uist18.pdf)

code: [https://github.com/thanhan/uist18](https://github.com/thanhan/uist18)

author: [https://www.cs.utexas.edu/~atn/](https://www.cs.utexas.edu/~atn/)

