Request Inspection Copy

If you are an Academic or Teacher and wish to consider this book as a prescribed textbook for your course, you may be eligible for a complimentary inspection copy. Please complete this form, including information about your position, campus and course, before adding to cart.

* Required Fields

To complete your Inspection Copy Request you will need to click the Checkout button in the right margin and complete the checkout formalities. You can include Inspection Copies and purchased items in the same shopping cart, see our Inspection Copy terms for further information.

Any Questions? Please email our text Support Team on


Email this to a friend

* ALL required Fields

Order Inspection Copy

An inspection copy has been added to your shopping cart

Conducting Online Research on Amazon Mechanical Turk and Beyond

by Leib Litman and Jonathan Robinson SAGE Publications, Inc
Pub Date:
Pbk 304 pages
AU$95.00 NZ$98.26
Product Status: Available in Approx 14 days
add to your cart
& Academics:
Conducting Online Research on Amazon Mechanical Turk and Beyond, by Leib Litman and Jonathan Robinson, provides researchers with step-by-step technical information on this important research platform. The book gives a broad view of the MTurk ecosystem and customs, hones in on common researcher pitfalls, and provides detailed data on sampling, ethics, and experimentation.

Chapter 1: Introduction
A Scientific Revolution in the Making
A Brief History of Online Research in the Social and Behavioral Sciences: From HTML 2.0 to Mechanical Turk
The use of online samples in applied behavioral research
Amazon Mechanical Turk
Chapter 2: The Mechanical Turk Ecosystem - Leib Litman, Cheskie Rosenzweig, Jonathan Robinson
How Quality is Maintained
Reputation Mechanism
Selectively Recruiting Specific Workers
Protections for Workers
Communicating with Workers
A Worker's Perspective
Worker Communities
Chapter 3: Conducting a Study on Mechanical Turk
Sample Project
Setting up a Requester Account on Mechanical Turk
Creating a HIT
The 'Design Layout' tab
Monitoring Progress on the Requester's Dashboard
When a Worker Runs Out of Time
Sample Study Results
Conducting Follow-up Studies Using Requester-Issued Qualifications
Appendix A: Checklist for best practices of setting up a Mechanical Turk HIT
Chapter 4: API and Third Party Apps
Third Party API-based Platforms
Common uses for API scripts and third party API-based apps
Chapter 5: Data quality Issues on Mturk - Jesse Chandler, Gabriele Paolacci, David Hauser
Defining and Measuring Data Quality
Measuring Individual Participant Data Quality
Causes of and Cures for Poor Data Quality
Concluding Thoughts
Chapter 6: Who are the Mechanical Turk Workers?
Sources of data
Location of Workers in the US
Demographics of Mechanical Turk
Chapter 7: Sampling Mechanical Turk Workers: Problems and Solutions
Sampling on Mechanical Turk
Sources of sampling bias
The Problem of Superworkers
Time-of-day Effects
Pay Rate
Sampling Best Practices
Chapter 8: Data Representativeness of Mechanical Turk samples
Representativeness, surveys, and survey sampling
The methodology of survey sampling
Mechanical Turk as a sampling frame
The fit-for-purpose framework
Chapter Overview:
Chapter 9: Conducting Longitudinal Research on Amazon Mechanical Turk
Why Longitudinal Research?
Retention, Longitudinal Research, and MTurk
Case Studies
Best practices for longitudinal research
Chapter 10: Beyond Mechanical Turk: Using Online Market Research Platforms
Limitations of MTurk
Online probability-based panels
Online market research platforms
Overall comparisons between Mechanical Turk and market research platforms
Chapter 11: Conducting ethical online research: A data-driven approach
Historical background
Risk of harm in online research
Research on sensitive topics
A deeper dive into controversial and complex issues
Economics of Mechanical Turk: Considerations for setting wages
Setting wages: Considerations of ethics and methodology
Considerations for rejecting, blocking and disqualifying workers.
Practical advice for requester/worker interactions
Appendix A

"I am looking forward to being able to hand this to my students and know they'll be ready to get started on MTurk and really understand the whole system (i.e., not just the mechanics of getting data, but really "getting" the culture and so on). I would probably also find myself citing this book as it discusses many of the issues I personally research.” 

Dr. Leib Litman received a PhD in experimental psychology from the City University of New York. His early research was in the area of implicit learning, focusing on the relationship between conscious and unconscious information processing. After receiving his Ph.D , he was a postdoctoral fellow and research scientist in cognitive neuroscience at New York University where he studied episodic memory using fMRI. Leib's current methodological research interests are in the area of online data collection, especially in the application of online data collection tools to complex research designs, such as dyadic studies, experience sampling, intensive longitudinal designs, qualitative video and focus group interviews, and global data collection.

Leib is currently an Associate Professor of psychology at Lander College in Queens, New York, and Director of Research at (formerly TurkPrime). Leib lives in Riverdale, New York with his wife Sarah and their two girls Bella and Rebecca.

Dr. Jonathan Robinson has been creating software since he was 12 and started his first software company with a few high school friends at age 15. He continued to develop his software skills and apply his expertise to more complex problems as he completed his BA at Queens College, and his MS and PhD with a dissertation in the field of robotic vision and digital curve fitting while at the City University of New York. 


Dr. Robinson conceived of using Amazon's Mechanical Turk for participants before there was public academic discussion of its application to research. He designed and implemented the core (previously TurkPrime) system together with his co-author, Dr. Leib Litman. CloudResearch was the first cloud based participant management system that successfully adapted Mechanical Turk for research by designing the features researchers need and managing the complexity that may get in the way of collecting high quality data .


Dr. Robinson is the co-CEO and Chief Technology Officer of CloudResearch. Dr Robinson is also currently an Associate Professor and Chair of computer science at Lander College, a division of Touro college. In that capacity he mentors, inspires and educates the next generation of software engineers and introduces them to his passion for melding cutting edge technologies with solving real world problems in the areas of academic and corporate research, digital image computation, approximation, and web application engineering.