Report: HCOMP 2016 Workshop on Mathematical Foundations of Human Computation

The HCOMP Workshop on Mathematical Foundations of Human Computation was held at HCOMP 2016 in Austin, Texas on November 3, 2016. The goal of the workshop was to bring together researchers across disciplines to discuss the future of research on the mathematical foundations of human computation, with a particular emphasis on the ways in which theorists can learn from the existing empirical literature on human computation and the ways in which applied and empirical work on human computation can benefit from mathematical foundations.

There has been great progress in human computation in the past decade. However, human computation is still not well understood from a foundational mathematical perspective.  Mathematical foundations are important to influence and shape the future of human computation. They provide frameworks for formalizing desirable properties of systems (e.g., correctness, optimality, scalability, privacy, fairness), predicting the impact of design decisions, designing systems with provable guarantees, and performing counterfactual analysis, which is notoriously hard to do empirically.

The workshop engaged researchers across different disciplines. The workshop featured an opening talk to set the stage for the day and four invited talks covering broad topics including a model of human conscious computation with applications to password generation, meta-unsupervised-learning methods, the design of a programming framework for humans in the loop, and how to increase productivity using (potentially crowdsourced) microtasks. The program also included six contributed talks and ample time reserved for general discussion. The slides for many of the talks are available on the workshop website.

The workshop concluded with an open problem session in which participants delivered short informal pitches of open problems related to the workshop theme. The main themes of the open problems that were proposed include:

  • The characterization of the important properties of computational models of human cognition. For example, how do we classify or quantify the hardness of problems in human computation?
  • Evaluation metrics for human computation problems. What properties do we want human computation to achieve?
  • The design of models that accurately incorporate human behavior. At a minimum, this requires diving into the literature from other disciplines, such as psychology and behavioral economics. Ideally it should be explored in an interdisciplinary manner.
  • How to develop theory and design algorithms that are robust to errors in modeling and experiments.

 

Workshop organizers:

Shuchi Chawla, University of Wisconsin – Madison
Chien-Ju Ho, Cornell University
Michael Kearns, University of Pennsylvania
Jenn Wortman Vaughan, Microsoft Research
Santosh Vempala, Georgia Tech

About the author

Chien-Ju Ho

View all posts