WearMail

Email search is often difficult for tasks such as:

  • What is my flight confirmation number?
  • What is my ACM member number?
  • Where is my meeting with Dan?
  • Was there an event I was supposed to go to today?
  • What deadlines do I have coming up?

Additionally, often we need answers to these questions on the go, such as when we’re in a taxi to the airport.

WearMail System

We built WearMail – a system where you can speak to your watch, and the watch will search your inbox. When the user requests specific types of information, such as flight confirmation numbers, and it triggers a special search that returns only the specific data.

jeffwearmail1-sm jeffwearmail2-sm

Currently, WearMail works on any AndroidWear and will search GMail using the API provided by Google.

Crowd Constructed Queries

We deployed two surveys in order to determine how well the crowd is able to generate useful Gmail queries based on natural language queries from the watch. In the first survey, we asked both workers on Amazon Mechanical Turk and workshop attendees to provide keyword search terms for three questions:

  1. “What is my Delta flight confirmation for today?”
  2. “I want to find my ACM Membership Number in my email.”
  3. “What room was I supposed to meet Dan Weld in today?”

Overall, both groups did reasonably well in constructing queries from the example questions, although most simply used queries from the original questions, e.g., “Meeting Dan Weld”, “ACM Membership”, “Delta confirmation”. Some workers tried to add additional information, such as today’s date. Workshop members were able to add a bit of additional expertise in formulating their queries, especially for the ACM Membership number. One query included the word “registration” and the other included the word “renewal,” presumably because workshop attendees thought these keywords would find those emails where the membership number was most likely to be mentioned.

Interfaces for Crowds to Create Search Patterns

We also asked survey participants to provide information that could be useful for constructing regular expression queries, both in terms of minimum and maximum range values and in terms of whether the target terms contained numbers, letters, or a combination of both. The results were largely inconsistent, but a preliminary interface for this approach is shown in the figure below. As a result, we hypothesized that a more promising approach may be to ask workers to find examples of the target terms on the internet, and to generalize from those. This worked reasonably for some — you can find examples of flight confirmation numbers, license plates, and room numbers. But, workers could not find other examples, such as ACM membership numbers. With our current UI, we had mixed success in getting workers to generalize the examples they found to other examples that could be reasonable.

Screen Shot 2016-09-01 at 6.16.24 PM

 

WearMail was one of the group projects pursued at the CMO-BIRS 2016 WORKSHOP ON MODELS AND ALGORITHMS FOR CROWDS AND NETWORKS.

About the author

hmslydia

View all posts