Alert button
Picture for Eser Kandogan

Eser Kandogan

Alert button

Towards Multifaceted Human-Centered AI

Jan 09, 2023
Sajjadur Rahman, Hannah Kim, Dan Zhang, Estevam Hruschka, Eser Kandogan

Figure 1 for Towards Multifaceted Human-Centered AI

Human-centered AI workflows involve stakeholders with multiple roles interacting with each other and automated agents to accomplish diverse tasks. In this paper, we call for a holistic view when designing support mechanisms, such as interaction paradigms, interfaces, and systems, for these multifaceted workflows.

* Workshop on Human-Centered AI at NeurIPS 2022 
Viaarxiv icon

MEGAnno: Exploratory Labeling for NLP in Computational Notebooks

Jan 08, 2023
Dan Zhang, Hannah Kim, Rafael Li Chen, Eser Kandogan, Estevam Hruschka

Figure 1 for MEGAnno: Exploratory Labeling for NLP in Computational Notebooks
Figure 2 for MEGAnno: Exploratory Labeling for NLP in Computational Notebooks
Figure 3 for MEGAnno: Exploratory Labeling for NLP in Computational Notebooks
Figure 4 for MEGAnno: Exploratory Labeling for NLP in Computational Notebooks

We present MEGAnno, a novel exploratory annotation framework designed for NLP researchers and practitioners. Unlike existing labeling tools that focus on data labeling only, our framework aims to support a broader, iterative ML workflow including data exploration and model development. With MEGAnno's API, users can programmatically explore the data through sophisticated search and automated suggestion functions and incrementally update task schema as their project evolve. Combined with our widget, the users can interactively sort, filter, and assign labels to multiple items simultaneously in the same notebook where the rest of the NLP project resides. We demonstrate MEGAnno's flexible, exploratory, efficient, and seamless labeling experience through a sentiment analysis use case.

* Data Science with Human-in-the-loop (DaSH) @ EMNLP 2022. Demo: https://meganno.github.io 
Viaarxiv icon

HEIDL: Learning Linguistic Expressions with Deep Learning and Human-in-the-Loop

Jul 25, 2019
Yiwei Yang, Eser Kandogan, Yunyao Li, Walter S. Lasecki, Prithviraj Sen

Figure 1 for HEIDL: Learning Linguistic Expressions with Deep Learning and Human-in-the-Loop
Figure 2 for HEIDL: Learning Linguistic Expressions with Deep Learning and Human-in-the-Loop

While the role of humans is increasingly recognized in machine learning community, representation of and interaction with models in current human-in-the-loop machine learning (HITL-ML) approaches are too low-level and far-removed from human's conceptual models. We demonstrate HEIDL, a prototype HITL-ML system that exposes the machine-learned model through high-level, explainable linguistic expressions formed of predicates representing semantic structure of text. In HEIDL, human's role is elevated from simply evaluating model predictions to interpreting and even updating the model logic directly by enabling interaction with rule predicates themselves. Raising the currency of interaction to such semantic levels calls for new interaction paradigms between humans and machines that result in improved productivity for text analytics model development process. Moreover, by involving humans in the process, the human-machine co-created models generalize better to unseen data as domain experts are able to instill their expertise by extrapolating from what has been learned by automated algorithms from few labelled data.

Viaarxiv icon