Crowd Sourcing Data Collection through Amazon Mechanical Turk

Report No. ARL-MR-0848
Authors: Cynthia Pierce and Nicholas Fung
Date/Pages: September 2013; 38 pages
Abstract: Crowdsourcing is an increasingly popular technique used to complete complex tasks or collect large amounts of data. This report documents the effort to employ crowdsourcing using the Mechanical Turk service hosted by Amazon. The task was to collect labeling data on several thousands of short videos clips as such labels would be perceived by a human. The approach proved to be viable, collecting large amounts of data in a relatively short time frame, but required specific considerations for the population of workers and impersonal medium through which data were collected.
Distribution: Approved for public release
  Download Report ( 1.498 MBytes )
If you are visually impaired or need a physical copy of this report, please visit and contact DTIC.

Last Update / Reviewed: September 1, 2013