Blind people often need to access visual information, but don’t have a sighted companion available to answer questions. While automated tools like OCR and object recognition can make some visual information accessible, other complex questions still require a human’s input.
We created VizWiz Social to connect blind users’ visual questions with sighted workers or friends who can answer them. VizWiz Social is a mobile phone application that allows the user to take a photograph of something, record a question about it, and send it to anonymous crowd workers or to their social network for answers. Our paper discusses what we learned from the 40,748 questions asked by 5,329 blind users in the first year of the app’s availability.
We created a taxonomy of the types of questions asked to find out what visual information was most desired by our blind users. We also analyzed the features of the photographs and questions send in by user, and user behaviors over time. Some interesting results include:
- The most popular type of questions were Identification questions (41%), where users asked for simple identifications of objects (such as food or drink items). However, there were also a large number of Description questions (24%) and Reading questions (17%), which required more complex analysis and human insight to answer.
- Most questions were perceived by our raters to be objective and time-sensitive, perhaps due to the fast response times available from object recognition or pre-recruited crowd workers.
- Users who had a poor first experience with our application – either because their photographs were low-quality or or because the answers they received were unsatisfactory – had a higher-than-typical abandonment rate.
These results improve our understanding of the problems blind people face, and may help motivate new projects more accurately targeted to help blind people live more independently in their everyday lives.
For more, see our full paper, Visual Challenges in the Everyday Lives of Blind People.
Erin Brady, ROCHCI @ University of Rochester
Meredith Ringel Morris, Microsoft Research
Yu Zhong, ROCHCI @ University of Rochester
Samuel White, ROCHCI @ University of Rochester
Jeffrey Bigham, ROCHCI @ University of Rochester