Very cool and interesting site! Nice way to use Amazon’s Mechanical Turk, imho. This is from the site:

Using AMT to run web-based experiments allows researchers to obtain large quantities of data in a cheap and efficient way. However, there is lack of evidence regarding the actual reliability of AMT’s workforce as a subject pool; as any new instrument, AMT requires to be tested thoroughly in order to be used confidently. This blog aims at collecting any individual effort made in order to validate AMT as a research tool. It reports results from experiments that increase or decrease the reliability of AMT as an online subject pool, as well as general guidance for running experiments through this service.

The blog is administered by Gabriele Paolacci and Massimo Warglien. As a social scientist, you are strongly encouraged to participate to this project. We are welcoming any effort helping to cumulate knowledge about the reliability of AMT as a tool for experimental research. In particular, we hope to receive both new contributions, along the lines suggested in this blog, and notices of any existing evidence that may enhance our understanding of the validity of AMT as a source of experimental subjects and data. Comments are also welcome.

Experimental Turk

Mechanical Turk is one of the suites of Amazon Web Services. It is a crowd-sourcing marketplace that allows computer programs to coordinate human intelligence to carry out the tasks that can’t be done by computers. People use Mechanical Turk because it allows them to make money.

It basically has two main players: Turkers and Requesters. Turkers work for requesters; however, some requesters can treat turkers poorly such as disapproving their work and not paying them, and this isn’t good for a requester.

To help you make the most out of this Amazon Web Service’s suite, here are three quick mechanical turk tips for Requesters:

• First of all, you need to be nice to people in the suite. Do not act like a jerk. You should reserve disapprovals as well as other hostile acts; otherwise, you’ll find it hard to look for Turkers who will be willing to work for you for $0.2.
• The second tip to remember is to appeal to the kindness of the people in the suite. Keep in mind that many people do not really do it for cash, although some people do. Most turkers really do it to help requesters; thus appreciate this kindness.
• The third tip is to be clear and detailed about what you really need. Reducing the pain level working to complete your assigned tasks means that more and more people would prefer to work for you in the future.

There you go – the three quick mechanical turk tips that you can use to make the most out of mechanical turk!

Amazon Mechanical Turk Requester - Resource Center

If you have not yet checked out the Amazon Mechanical Turk for your art experiments, then you should. Turk is very popular for running short and simple art studies and experiments. For example, if you want to know how many people would describe a single art work, or take a survey from a big crowd to know which color they want for a new art product you are releasing, and you want to get the data really fast, then you can use this Amazon suite for your best mechanical Turk art experiments.

Ten Thousand Cents

So does mechanical Turk work well? Yes it does! A lot of people have made a lot of nice write-ups about artistic experiments they made using Turk. They primarily tested Turk’s reliability for simple art experiments and they got quite good results. Another great analysis they got show that Turk is cheap, fast, and as good as machine learning data.

If you really want to have the best mechanical Turk experiments, then it’s good if you could post your questions to Turk to find out or interact with other researchers doing the same thing. You can find a lot of information online as to how you can entice people to participate in your experiment.

Amazon mechanical Turk was actually not designed with researchers, scientists, or artists in mind, nor does it have the rights of human subjects on their mind in looking for participants. There are times when this system can be frustrating when doing your experiment especially more complex studies; however, it can be one great tool for a lot of things. This means that best mechanical Turk art experiments are really possible.

Mechanical Turk Art Experiment Examples? Here you go.

• Ten Thousand Cents
• The Sheep Market
• Faces of Mechanical Turk