The quality of data in content analysis, in surveys with open-ended questions, in the observation of unstructured social events, and so on, critically depends on the reliability with which primary observations are assigned to categories, scaled, or measured. To help assure valid interpretations, agreement between two independent observers is measured. When agreement is due merely to chance, data may have little to do with the phenomena studied. In order for such data to be empirically meaningful, a high degree of inter-observer agreement must be demonstrated.