Ken Holstein
Assistant Professor
- Pittsburgh PA UNITED STATES
Ken Holstein's research focuses broadly on AI-augmented work and improving how we design and evaluate AI systems for real-world use.
Biography
Ken's research focuses broadly on AI-augmented work and improving how we design and evaluate AI systems for real-world use. Ken draws on approaches from human–computer interaction (HCI), AI, design, cognitive science, learning sciences, statistics, and machine learning, among other areas.
Ken is deeply interested in: (1) understanding the gaps between human and artificial intelligence across a range of contexts, and (2) using this knowledge to design systems that respect human work, elevating human expertise and on-the-ground knowledge rather than diminishing it. To support these goals, Ken's research develops new approaches and tools that support better incorporation of diverse human expertise across the AI development lifecycle.
Ken's work has been generously supported by the National Science Foundation (NSF), CMU’s Block Center for Technology and Society, Northwestern’s CASMI & UL Research Institutes, Institute for Education Sciences (IES), Cisco Research, Jacobs Foundation, Amazon Research, CMU’s Metro21 Smart Cities Institute, and Prolific.
Areas of Expertise
Media Appearances
That Message From Your Doctor? It May Have Been Drafted by A.I.
The New York Times online
2024-09-24
Health care administrators often refer to Art as a low-risk use of A.I., since ideally a provider is always reading through the drafts and correcting mistakes.
The characterization annoys researchers who study how humans work in relation to artificial intelligence. Ken Holstein, a professor at the human-computer interaction institute at Carnegie Mellon, said the portrayal “goes against about 50 years of research.”
Funding New Research to Operationalize Safety in Artificial Intelligence
Northwestern Engineering online
2023-02-17
Kenneth Holstein, assistant professor in the Human-Computer Interaction Institute at Carnegie Mellon University, will study how to support effective AI-augmented decision-making in the context of social work. In this domain, predictions regarding human behavior are fundamentally uncertain and ground truth labels upon which an AI system is trained — for example, whether an observed behavior is considered socially harmful — often represent imperfect proxies for the outcomes human decision-makers are interested in modeling.
In Sudden Alarm, Tech Doyens Call for a Pause on ChatGPT
WIRED online
2023-03-29
Others working in tech also expressed misgivings about the letter's focus on long-term risks, as systems available today including ChatGPT already pose threats. “I find recent developments very exciting,” says Ken Holstein, an assistant professor of human-computer interaction at Carnegie Mellon University, who asked his name be removed from the letter a day after signing it as debate emerged among scientists about the best demands to make at this moment.
These glasses give teachers superpowers
The Hechinger Report online
2018-10-04
Lumilo is the brainchild of a team at Carnegie Mellon University. Ken Holstein, a doctoral candidate at the university, designed the app with significant input from teachers like Mawhinney who use cognitive tutors in their classrooms. The project treads new ground for the use of artificial intelligence in schools.
‘Smart’ glasses for teachers help pupils learn
Tes Magazine online
2018-06-27
“By alerting teachers in real-time to situations, the ITS [intelligent tutoring system] may be ill-suited to handle on its own. Lumilo facilitates a form of mutual support or co-orchestration between the human teacher and the AI tutor,” said Ken Holstein, lead author on the study, together with Bruce M. McLaren and Vincent Aleven.
Media
Social
Industry Expertise
Accomplishments
CMU Teaching Innovation Award
2022
Prototyping Algorithmic Experiences (PAX)
Graduate Student Poster Grand Prize
2022
Grefenstette Tech Ethics Symposium
Best Paper Award
2023
IEEE Conference on Secure and Trustworthy Machine Learning (SaTML’23)
Best Paper Award
2023
ACM CHI Conference on Human Factors in Computing Systems (CHI’23)
Best Paper Award
2023
ACM Conference on Fairness, Accountability, and Transparency (FAccT’23)
Education
Carnegie Mellon University
Ph.D.
Human–Computer Interaction
Carnegie Mellon University
M.S.
Human–Computer Interaction
University of Pittsburgh
B.S.
Psychology (Cognitive focus)
Affiliations
- Association for Computing Machinery (ACM) : Member
- Design Justice Network (DJN) : Member
Links
Event Appearances
Supporting Effective AI-Augmented Decision-Making in Social Contexts
Toward a Safety Science of AI Northwestern University, Evanston, IL
Fostering Critical AI Literacy Among Frontline Workers, the Public, & AI Developers
HCI + Design Thought Leaders Lecture Northwestern University, Evanston, IL
Designing for Complementarity in AI-Augmented Work
UCI Informatics Seminar Series University of California Irvine (UCI), Irvine, CA
Research Grants
Supporting Effective AI-Augmented Decision-Making in Content Moderation
Block Center
2022 - 2023
Supporting Effective AI-Augmented Decision-Making in Social Contexts
Center for Advancing Safety of Machine Intelligence (CASMI) and UL
2023 - 2025
Bridging Policy Gaps in the Life Cycle of Public Algorithmic Systems
Block Center
2022 - 2023
Scaffolding Responsible AI Practice at the Earliest Stages of Ideation, Problem Formulation and Project Selection
PwC
2023 - 2024
AI-Augmented Illustration through Conversational Interaction
Prolific
2023
Articles
Validating LLM-as-a-Judge Systems Under Rating Indeterminacy
Advances in Neural Information Processing SystemsLuke Guerdan, Solon Barocas, Kenneth Holstein, Hanna Wallach, Steven Wu, Alexandra Chouldechova
2026-04-23
The LLM-as-a-judge paradigm, in which a judge LLM system replaces human raters in rating the outputs of other generative AI (GenAI) systems, plays a critical role in scaling and standardizing GenAI evaluations. To validate such judge systems, evaluators assess human–judge agreement by first collecting multiple human ratings for each item in a validation corpus, then aggregating the ratings into a single, per-item gold label rating. For many items, however, rating criteria may admit multiple valid interpretations, so a human or LLM rater may deem multiple ratings “reasonable” or “correct”. We call this condition rating indeterminacy. Problematically, many rating tasks that contain rating indeterminacy rely on forced-choice elicitation, whereby raters are instructed to select only one rating for each item. In this paper, we introduce a framework for validating LLM-as-a-judge systems under rating indeterminacy. We draw theoretical connections between different measures of judge system performance under different human–judge agreement metrics, and different rating elicitation and aggregation schemes. We demonstrate that differences in how humans and LLMs resolve rating indeterminacy when responding to forced-choice rating instructions can heavily bias LLM-as-a-judge validation. Through extensive experiments involving 11 real-world rating tasks and 9 commercial LLMs, we show that standard validation approaches that rely upon forced-choice ratings select judge systems that are highly suboptimal, performing as much as 31% worse than judge systems selected by our approach that uses multi-label “response set” ratings to account for rating indeterminacy. We conclude with concrete recommendations for more principled approaches to LLM-as-a-judge validation.
Ethics of AI in education: Towards a community-wide framework
International Journal of Artificial Intelligence in EducationWayne Holmes, Kaska Porayska-Pomsta, Ken Holstein, Emma Sutherland, Toby Baker, Simon Buckingham Shum, Olga C Santos, Mercedes T Rodrigo, Mutlu Cukurova, Ig Ibert Bittencourt, Kenneth R Koedinger
2021-04-09
Cited 3,228 times as of Nov. 2026.
While Artificial Intelligence in Education (AIED) research has at its core the desire to support student learning, experience from other AI domains suggest that such ethical intentions are not by themselves sufficient. There is also the need to consider explicitly issues such as fairness, accountability, transparency, bias, autonomy, agency, and inclusion. At a more general level, there is also a need to differentiate between doing ethical things and doing things ethically, to understand and to make pedagogical choices that are ethical, and to account for the ever-present possibility of unintended consequences. However, addressing these and related questions is far from trivial. As a first step towards addressing this critical gap, we invited 60 of the AIED community’s leading researchers to respond to a survey of questions about ethics and the application of AI in educational contexts. In this paper, we first introduce issues around the ethics of AI in education. Next, we summarise the contributions of the 17 respondents, and discuss the complex issues that they raised. Specific outcomes include the recognition that most AIED researchers are not trained to tackle the emerging ethical questions. A well-designed framework for engaging with ethics of AIED that combined a multidisciplinary approach and a set of robust guidelines seems vital in this context.
A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms
2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)2023
Recent research increasingly brings to question the appropriateness of using predictive tools in complex, real-world tasks. While a growing body of work has explored ways to improve value alignment in these tools, comparatively less work has centered concerns around the fundamental justifiability of using these tools. This work seeks to center validity considerations in deliberations around whether and how to build data-driven algorithms in high-stakes domains.
Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems2023
Machine learning models with high accuracy on test data can still produce systematic failures, such as harmful biases and safety issues, when deployed in the real world. To detect and mitigate such failures, practitioners run behavioral evaluation of their models, checking model outputs for specific types of inputs. Behavioral evaluation is important but challenging, requiring that practitioners discover real-world patterns and validate systematic failures.
Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency2023
An emerging body of research indicates that ineffective cross-functional collaboration – the interdisciplinary work done by industry practitioners across roles – represents a major barrier to addressing issues of fairness in AI design and development. In this research, we sought to better understand practitioners’ current practices and tactics to enact cross-functional collaboration for AI fairness, in order to identify opportunities to support more effective collaboration.
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency2023
A growing literature on human-AI decision-making investigates strategies for combining human judgment with statistical models to improve decision-making. Research in this area often evaluates proposed improvements to models, interfaces, or workflows by demonstrating improved predictive performance on “ground truth’’ labels. However, this practice overlooks a key difference between human judgments and model predictions.