Type Members

1.  case class Approval(message: String) extends AssignmentEvaluation with Product with Serializable

   Approval of an assignment.

   Approval of an assignment.

   message

   the message shown with the approval (I think the worker can see this?)

2.  case class Assignment[Response](hitTypeId: String, hitId: String, assignmentId: String, workerId: String, acceptTime: Long, submitTime: Long, response: Response, feedback: String) extends Product with Serializable

   Represents a single annotator's response to a HIT.

   Represents a single annotator's response to a HIT.

   Similar to the "Assignment" type from the MTurk API, except that it only corresponds to a *finished* annotation, which may have been accepted or rejected. Also is well-typed for the purposes of an experiment, parametrized over the type of Response you want.

   Another bit of post-processing: this accommodates for a "feedback" field separate from the actual response, to make working with response data easier, instead of having to embed the feedback in the response field.

   Response

   the desired data representation for annotators' responses

   hitId

   the ID of the HIT this assignment was for

   assignmentId

   the unique ID given to this assignment on MTurk

   workerId

   the ID of the worker who did this assignment

   acceptTime

   the time (millis from epoch) when the worker accepted the HIT

   submitTime

   the time (millis from epoch) when the worker submitted the HIT

   response

   the worker's response to the HIT

3.  sealed trait AssignmentEvaluation extends AnyRef

   Represents an accept/reject decision for an assignment.

4.  case class HIT[Prompt](hitTypeId: String, hitId: String, prompt: Prompt, creationTime: Long) extends Product with Serializable

   Represents a HIT that has already been uploaded to MTurk.

   Represents a HIT that has already been uploaded to MTurk. It may not still be on MTurk, but it should still be saved to disk.

   Parameterized over a desired data representation for "prompts", which are transformed by a TaskSpecification into questions for the MTurk interface.

   Prompt

   the data representation of a question

   hitId

   the unique ID assigned to this HIT by MTurk

   prompt

   the data used to create the question shown to workers

   creationTime

   the time (millis from epoch) that the HIT was created

5.  trait HITDataService extends AnyRef

   API for services that store the data returned by workers across HITs on MTurk.

   API for services that store the data returned by workers across HITs on MTurk. Semantics are append-only; you can save HITs and Assignments and get them later, but cannot delete any data.

   Not expected to be thread-safe in general. TODO: implement this using the interpreter pattern instead.

6.  case class HITInfo[Prompt, Response](hit: HIT[Prompt], assignments: List[Assignment[Response]]) extends Product with Serializable

   Data structure to hold all data relevant to a particular HIT.

   Data structure to hold all data relevant to a particular HIT. TODO: should include rejected assignments as well.

7.  class InMemoryHITDataService extends HITDataService

   Simple in-memory implementation of HITDataService for use in testing and sample tasks.

8.  case class Rejection(message: String) extends AssignmentEvaluation with Product with Serializable

   Rejection of an assignment.

   Rejection of an assignment.

   The message parameter should be used to give an explanation of why the work was rejected. I also like to pair it with advice on how to avoid another rejection.

   message

   the message shown to the worker with the rejection