Very cool and interesting site! Nice way to use Amazon’s Mechanical Turk, imho. This is from the site:
Using AMT to run web-based experiments allows researchers to obtain large quantities of data in a cheap and efficient way. However, there is lack of evidence regarding the actual reliability of AMT’s workforce as a subject pool; as any new instrument, AMT requires to be tested thoroughly in order to be used confidently. This blog aims at collecting any individual effort made in order to validate AMT as a research tool. It reports results from experiments that increase or decrease the reliability of AMT as an online subject pool, as well as general guidance for running experiments through this service.
The blog is administered by Gabriele Paolacci and Massimo Warglien. As a social scientist, you are strongly encouraged to participate to this project. We are welcoming any effort helping to cumulate knowledge about the reliability of AMT as a tool for experimental research. In particular, we hope to receive both new contributions, along the lines suggested in this blog, and notices of any existing evidence that may enhance our understanding of the validity of AMT as a source of experimental subjects and data. Comments are also welcome.
Experimental Turk
