CrowdCamp Report: The Microtask Liberation Front: Worker-friendly Task Recommendation

As crowdsourced marketplaces like Amazon’s Mechanical Turk have grown, tool builders have focused the majority of their attention on requesters.  The research community has produced methods for improving result quality, weeding out low-quality work, and optimizing crowd-powered workflows, all geared toward helping requesters.  On the other hand, the community has done a decent job of studying crowd workers, but has not devoted much effort to building usable tools that improve the lives of workers.  At CrowdCamp, we worked on a browser plugin called MTLF that we hope will improve Turkers’ task-finding and work experiences.

A prototype of the MTLF browser plugin
A prototype of the MTLF browser plugin

After installing MTLF, a Turker logs into MTurk.  Our prototype asks them to prioritize their preferences for income, task diversity, or fun.  After completing a task, they are asked to provide a binary rating (hot/not) of a task.  They are then asked whether they want a new task or more of the same task.  Instead of having the Turker wade through the existing difficult-to-grok list of available tasks, MTLF automatically pops up a new task on the Turker’s screen.  As Turkers change their priorities and grade tasks, MTLF’s recommendation algorithm leverages the joint work histories of many workers to identify tasks that match individual worker interests and preferences.  The goal of our tool is to improve worker satisfaction and reduce worker search time and frustration.

We’re not the first to take on the challenge of improving the lives of workers.  Turkopticon is a wonderful tool for Turkers to share information on requesters.  Turkers themselves have identified a number of other tools to help them with their process.  None of these tools, however, optimize crowd workers’ preferences in quite the automated way that requester-oriented tools currently do.  As we build on our prototype, we hope to ingest information from sources like Turkopticon to inform our recommendation algorithms.

While our prototype has a working interface and backend to store user preferences, we’re working hard on more features for a usable first version.  Our next steps include exploring sources of data other than worker preferences, building an initial task recommender, and co-designing and iterating on our initial interface with the help of Turkers.  We’d love your help—our github repository has a list of open needs that you can help out with!

Jonathan Bragg, University of Washington
Lydia Chilton, University of Washington
Daniel Haas, UC Berkeley
Rafael Leano, University of Nebraska-Lincoln
Adam Marcus, Locu/GoDaddy
Jeff Rzeszotarski, Carnegie Mellon University

About the author

Adam Marcus

View all posts