Remembering David Martin and his ethnographic studies of crowdworkers

This post is a late tribute to Dr. David B. Martin, who passed away in June 2016. Dave was an ethnographer, inherently interested in the tension between tools, progress and the reality of everyday life. Dave had spent the last few years doing ethnomethodological studies of crowdwork and crowdworkers, together with Jacki O’Neill, Ben Hanrahan, and myself at Xerox Research Centre Europe, bringing forth the invisible, hidden work, and various issues of the worker communities into public view. Through ethnomethodology we studied how people collaborated with each other, how they communicated, and the mundane things they did to make crowdwork work. His vision and efforts in the area of crowdsourcing will continue to live on through his work, constantly inspiring us.

The team’s first encounter with crowdsourcing was in 2011, when Dave and Jacki began studying outsourcing work to investigate if crowdsourcing could be used for work that is traditionally outsourced. Following the tradition of the work practice studies at Xerox, they conducted ethnographic studies of BPO work [1], specifically of low-skilled, piece-rate work of healthcare form digitization [2] to see if it could be crowdsourced. Dave studied at-home work in the US and Jacki studied in-office data entry work in India. These studies showed how the outsourcers’ current workflow was sensitively orchestrated to achieve high quality and rapid turnaround times at minimum cost. Even in a low-skilled, piece-rate environment, it was the subtleties of the employer-employee relationship that enabled the work to be done on time and to quality.

The Xerox team then went on to explore crowdsourcing further, especially the lived work of microtask crowdwork: the work practices that make crowdwork work as experienced by crowdworkers. At this point the research team expanded from Dave and Jacki, and got Shourya Roy and colleagues from the Xerox India lab involved. And later joined, Ben Hanrahan, our in-house crowdwork developer at Xerox lab in Grenoble (and speaker at HCOMP at ‘Remembering David B. Martin and his Ethnographic Studies of Crowd Workers’ event [3] on the 31st of October 2016), and myself, PhD candidate from University of Nottingham, UK.

Our interest in crowdsourcing was to enquire: Who are the people who do crowdwork? What are their work practices? What are their personal and work lives like? Why do they this kind of work? What are their expectations from, and issues with crowdwork? How do the various technologies they use fit with and support these practices?

Our enquiries began with Dave’s study [4] of the Turker Nation Forum, predominantly Turkers based in the US, and my PhD, studying the Turkers based in India. Dave’s study was ground-breaking both methodologically (being an EM ethnography of a forum) and in its findings. It painted the first detailed picture of the lived world of crowdworking. Turkers oriented to turking as ‘work’ and AMT as a ‘labour market’ which came as a surprise to much of the research community. The study showed the motivations and ethical codes the workers followed, and the need for fairness and relationship-building in crowdwork. In platforms like Amazon Mechanical Turk not only does it offer no support for the management of rapid, high quality workflows, but also deliberately designs out the relationship between workers and the organisation, reduces accountability and replaces complex social, organisational & financial motivators almost solely with monetary ones.

Before publication of ‘Being a Turker’, Dave sought and got approval from members of the Turker Nation forum for his paper, as he wanted to ensure he was truly representing the concerns of the Turker community. He truly prized the emails and forum comments about his paper from the members of the turker community, and although we do not have these communications with us, it shows us what really mattered to him.

Dave, Jacki and Ben were part of the analysis of the ethnographic data I collected in India in 2013. Dave took over my phd supervision from Jacki, in 2014, honed by analytical skills and mentored me on the ethnomethodological journey through my phd. The rigorous discussions and debates with him on Zimmerman, Becker and so on helped frame the key discussions in my thesis. With Andy Crabtree and Tom Rodden from University of Nottingham we wrote the ‘Understanding Indian Crowdworkers’ as an introduction[5] to the Indian turkers’ story. The first major contribution from the phd was the ‘Turk-life in India’ paper [6] that showed the various levels of digital and English-language literacy amongst turkers, the role infrastructure and technology played in turking and social nature of turking. ‘Turking in a global labour market’ was a comparative[7] narrative of workers based in the US and India, the understandings they had of the market and of each other in a transnational labour platform that brought them head-on into a very competitive work environment and what that meant for designers, policy makers, researchers and activists. This also led us to design a conceptual tool [8] – TurkBench, meant to provide dynamic scheduling and provide crowdworkers with personalized market visualization and session management.

Our collective research into crowdsourcing showed that turkers were collaborative agents, who cared about their work, the money they earned, their reputation and relationship with the requesters. It also brought to the fore the discussion on design, since in crowdsourcing, the labour market is embodied through the technology you create, and thus have far-reaching consequences in the lives of the crowdworkers, not just in their ability to successfully complete quality work, but also in their working conditions and standards of living. Dave wanted to ensure that technology-designers were not designing in a bubble, and were aware of their responsibilities, and the consequences their design had on ‘real’ people.

Dave had been working to support this cause since his first brush with crowdsourcing. One of his final projects was the book chapter “Understanding The Crowd: ethical and practical matters in the academic use of crowdsourcing” which highlights the principles of ‘professional ethics’ that were put in place for the safety and well-being of the research subjects, in this case, the crowdworkers, reminding researchers of ‘ethical conduct’ during crowd-based research, providing references and further guidance to use when using the crowd in supporting empirical work. He ignited a spark in the conversations at a Dagstuhl crowdsourcing Seminar [9] in November 2015 “Evaluation in the Crowd: Crowdsourcing and Human-Centred Experiments” where computer scientists from the fields like visualization, psychology, graphics, multimedia assembled to discuss the role of crowdsourcing in empirical research work. Dave challenged the perspectives of the academicians present, reminding them to never overlook the people who provided experimental data for them. The book arising from that seminar is to be published by Springer in early 2017.

The last event he took part in at was at the University of Oxford at the Connected Life Conference [10] in June 2016, where he, along with colleagues, participated in an interdisciplinary debate on socio-digital practices of collective action, continuing his fight to design technologies and advance policy work for workers doing crowdwork, who are ignored, hidden behind algorithmic distribution and assembly of work.

Drawing to a close, I’d like to borrow Jacki’s words from her video at HCOMP 2016, something that echoes with everyone who had known him, met him at conferences or at a pub.
“Dave will be remembered for his passion for people, philosophy, a good argument and a pint. He conducted ethnomethodology without indifference. His sense of fairness and integrity permeated both his social and his work life. And in the fight to design technologies which give agency to typically undervalued workers, we have lost a kindred spirit. The greatest tribute we can give to Dave, is to remember that the crowd has a human face.”

We miss you Dave.

Neha Gupta


[1] Relationship-based Business Process Crowdsourcing?
Jacki O.Neill and David Martin. In IFIP Conference on Human-Computer Interaction, pp. 429-446, 2013.

[2] Form digitization in BPO: from outsourcing to crowdsourcing?
Jacki O’Neill, Shourya Roy, Antonietta Grasso, and David Martin. In Proceedings of the 31st ACM SIGCHI conference on Human factors in computing systems, pp. 197-206, 2013.
[3] HCOMP talk
[4] Being a Turker
David Martin, Benjamin V. Hanrahan, Jacki O’Neill, and Neha Gupta. In Proceedings of the 17th ACM CSCW Conference on Computer Supported Cooperative Work & Social Computing, pp. 224-235, 2014.

[5] Understanding Indian Crowdworkers
Neha Gupta, Andy Crabtree, Tom Rodden, David Martin, and Jacki O.Neill. Back to the Future of Organizational Work: Crowdsourcing and Digital Work Marketplaces Workshop at CSCW 2014.

[6] Turk-Life in India
Neha Gupta, David Martin, Benjamin V. Hanrahan, and Jacki O’Neill. In Proceedings of the 18th ACM GROUP: International Conference on Supporting Group Work, pp. 1-11. 2014.

[7] Turking in a Global Labour Market
David Martin, Jacki O.Neill, Neha Gupta, and Benjamin V. Hanrahan. In Proceedings of the 19th ACM CSCW Conference on Computer Supported Cooperative Work & Social Computing, pp. 39-77, 2016

[8] TurkBench: Rendering the Market for Turkers
Benjamin V. Hanrahan, Jutta K. Willamowski, Saiganesh Swaminathan, and David B. Martin. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), pp.1613-1616.

[9] Dagstuhl seminar

[10] Oxford Conference

About the author


PhD Candidate, University of Nottingham, UK.
Investigating crowdsourcing in India, case of Amazon Mechanical Turk.

View all posts

1 Comment

  • David Martin submitted a statement to the Crowd Statement Marathon that we organized at the Trust in Crowd Work workshop ( co-located with WebSci 2016. I would like to add it here to honor his memory.

    “Trust in crowdsourcing comes through human relationships even though they are diminished by the technology: the qualities of the exchanges determine the level of trust. ” – David Martin 2016.