Very cool and interesting site! Nice way to use Amazon’s Mechanical Turk, imho. This is from the site:

Using AMT to run web-based experiments allows researchers to obtain large quantities of data in a cheap and efficient way. However, there is lack of evidence regarding the actual reliability of AMT’s workforce as a subject pool; as any new instrument, AMT requires to be tested thoroughly in order to be used confidently. This blog aims at collecting any individual effort made in order to validate AMT as a research tool. It reports results from experiments that increase or decrease the reliability of AMT as an online subject pool, as well as general guidance for running experiments through this service.

The blog is administered by Gabriele Paolacci and Massimo Warglien. As a social scientist, you are strongly encouraged to participate to this project. We are welcoming any effort helping to cumulate knowledge about the reliability of AMT as a tool for experimental research. In particular, we hope to receive both new contributions, along the lines suggested in this blog, and notices of any existing evidence that may enhance our understanding of the validity of AMT as a source of experimental subjects and data. Comments are also welcome.

Experimental Turk

If you have not yet checked out the Amazon Mechanical Turk for your art experiments, then you should. Turk is very popular for running short and simple art studies and experiments. For example, if you want to know how many people would describe a single art work, or take a survey from a big crowd to know which color they want for a new art product you are releasing, and you want to get the data really fast, then you can use this Amazon suite for your best mechanical Turk art experiments.

Ten Thousand Cents

So does mechanical Turk work well? Yes it does! A lot of people have made a lot of nice write-ups about artistic experiments they made using Turk. They primarily tested Turk’s reliability for simple art experiments and they got quite good results. Another great analysis they got show that Turk is cheap, fast, and as good as machine learning data.

If you really want to have the best mechanical Turk experiments, then it’s good if you could post your questions to Turk to find out or interact with other researchers doing the same thing. You can find a lot of information online as to how you can entice people to participate in your experiment.

Amazon mechanical Turk was actually not designed with researchers, scientists, or artists in mind, nor does it have the rights of human subjects on their mind in looking for participants. There are times when this system can be frustrating when doing your experiment especially more complex studies; however, it can be one great tool for a lot of things. This means that best mechanical Turk art experiments are really possible.

Mechanical Turk Art Experiment Examples? Here you go.

• Ten Thousand Cents
• The Sheep Market
• Faces of Mechanical Turk

Red Box Video

So I was thinking the other day (I know, I know, dangerous…) and it struck me that Amazon could take the red box video idea and merge it with Mechanical Turk for something really revolutionary. If you live in North America, I’m sure you’re familiar with the red box video service that you can find at McDonald’s, Walmart, etc. They’re all over now. Basically, it’s a vending machine for DVDs.

The idea of kiosks got me thinking, though. What if, instead of a retail machine, it was something entirely different from Amazon – a Mechanical Turk kiosk – a Green Box. Picture this. Worker comes to McDonald’s, answers a few simple questions and gets to choose money or menu. If menu is picked, depending on how many tasks he completed, pictures of food and drink comes up. Maybe 5 tasks for a Big Mac. The details would have to be worked out. Continue Reading