To evaluate subjective experimental result on Amazon Mechanical Turk is a lot of fun. We spent just 1 dollar, and get 80 responds.
More interesting, those workers ( or they might be a graduate student like you and me) gave us very serious and special comments. I collected them and posted here.
In one of our test image, we accidentally recognize a girl as a boy, and the worker said: Might be a girl, not a boy.
The same thing happened when we recognize a cow as a sheep: I believe it is a cow, not a sheep.
Well, most of the replies are like: Thank you; Good Hit; Nice to do this hit...
And the result we collected are pretty reasonable and useful for our Computational Linguistic final project.
Well, if you want to try the feeling of being a King, just spend several dollars to Amazon Mechanical Turk.
And after did 7 missions, I also earned 1.** dollars...