Introduction to Designing Qualitative Research

||Introduction || Sampling || Phases of Data Collection and Analysis || Validity || Disciplined Subjectivity || Standards of Adequacy || Course Overview Map || References ||
Source: http://trochim.human.cornell.edu/kb/gif/kbroad.gif

Introduction

Qualitative research is naturalistic inquiry, because the data collection strategies used are interactive to discover the natural flow of the events and processes. Most qualitative research deals with people's individual and collective social actions, beliefs, thoughts, and perceptions.

In this session, we will focus on designing qualitative research: its sampling strategies, phases of data collection and analysis, validity and reliability issues, and finally the qualitative research standards.

Sampling

In qualitative research, the research sites and participants are selected following a strategy called purposeful sampling. Purposeful sampling, in contrast to probabilistic sampling, is "selecting information-rich cases for study in depth" (Patton, 1990, p. 169) when one wants to understand something about those cases without needing or desiring to generalize to all such cases. Types of purposeful sampling include site selection, comprehensive sampling, maximum variation sampling, network sampling, and sampling by case type.

Purposeful Sampling Strategies
Sampling StrategyDescription
Site SelectionSelect site where specific events are expected to occur
Comprehensive sampling Choose entire group by criteria
Maximum variation samplingSelect to obtain maximum differences of perceptions about a topic among information-rich informants or group
Network samplingEach successive person or group is nominated by a prior person as appropriate for a profile or attribute
Sampling by case type
Extreme-case samplingChoose extreme cases after knowing the typical or average case-e.g., outstanding successes, crisis events
Intense-case samplingSelect cases that are intense but not extreme illustrations
Typical-case samplingKnow the typical characteristics of a group and sample by cases
Unique-case samplingChoose the unusual or rare case of some dimension or event
Reputational-case samplingObtain the recommendation of knowledgeable experts for the best examples
Critical-case samplingIdentify the case that can illustrate some phenomenon dramatically
Concept/theory-based samplingSelect by information-rich persons or situations known to experience the concept or to be attempting to implement the concept/theory
Combination of purposeful sampling strategiesChoose various sampling strategies as needed or desired for purposes, especially in large-scale studies and lengthy process studies
McMillan and Schumacher (1997)

Phases of Data Collection and Analysis Strategies

Qualitative phases of data collection and analyses are interactive research processes that occur in overlapping cycles. There are no rigid step by step procedures to follow, both data collection and analysis progress simultaneously toward the completion of the research project.

Validity of Qualitative Research

Validity refers to the best available approximation to the truth of propositions. There are two types of validity according to Campbell and Stanley (1963). Internal validity refers to the approximate validity with which we infer that a relationship between two variables is causal. External validity refers to the approximate validity with which we can infer that the presumed causal relationship can be generalized to and across alternate measures of the cause and effect and across different types of persons, settings, and times. Although validity, i.e. internal and external validity, is developed in the context of quantitative research, in order to ensure compatibility, people still use the same terms for qualitative research but not necessarily with the same interpretations.

In qualitative research, internal validity is affected by qualitative research designs, and external validity is related to the extension of qualitative findings.

Qualitative Design Validity

Qualitative research design validity concerns these questions: Do researchers actually observe what they think they observe? Do researchers actually hear the meanings that they think they hear? Thus, the internal validity of qualitative research is the degree to which the interpretations and concepts have mutual meanings between the participants and the researcher.

The following strategies are used to enhance the validity of qualitative research design:

Strategies to Enhance Design Validity
StrategyDescription
Prolonged and persistent field workAllows interim data analysis and corroboration to ensure the match between findings and participant reality
Participant language; verbatim accountsObtain literal statements of participants and quotations from documents
Low-inference descriptorsRecord precise, almost literal, and detailed descriptions of people and situations
Multiple researchersAgreement on descriptive data collected by a research team
Mechanically recorded dataUse of tape recorders, photographs and videotapes
Participant researcherUse of participant recorded perceptions in diaries or anecdotal records for corroboration
Member checking Check informally with participants for accuracy during data collection; frequently done in participant observation studies
Participant reviewAsk each participant to review researcher's synthesis of all interviews with the person for accuracy of representation; frequently done in interview studies
Negative cases or discrepant dataActively search for, record, analyze, and report negative cases of discrepant data that are an exception to patterns or that modify patterns found in the data

Source: McMillan and Schumacher (1997)

Extension of Qualitative Findings

In order to maximize the external validity in qualitative research, the intent is to provide for the extension of findings, as grounded theory or as an analytical synthesis that enables others to understand similar situations and apply these findings in subsequent research. Knowledge is produced not by replication but by the preponderance of evidence found in separate case studies over time or in more structured quantitative designs. The following ten design components affect the extension of qualitative findings:

Components that Enable Others to Discover Similar Phenomena or Apply the findings
StrategyAdequate Description in the Study
Research RoleThe social relationship of the researcher with the participants
Informant selectionCriteria, rationale, and decision process used in purposeful sampling
Social contextThe physical, social, interpersonal, and functional social scenes of data collection
Data collection strategiesThe multimethods employed, including ethnographic observation, ethnographic interview, and documents
Data analysis strategiesData analysis process described
Authentic narrativeThick description presented as an analytical narrative
TypicalityDistinct characteristics of groups and/or sites presented
Analytical premisesThe initial theoretical framework or concept that informs the study and others added as the study progresses
Alternative explanationsRetrospective delineation of all plausible or rival explanations for interpretations
Other criteria by purpose (after study completed)
PhenomenologyUnderstand the complexity of the situation; generates additional case studies
Grounded theoryConcepts or mini-theory relate to social science; generates verification research with more structured designs
Critical traditionsHistorical revisionism and situatedness to erode ignorance; empowers participants; generates further research; action stimulus

Source: McMillan and Schumacher (1997)

Disciplined Subjectivity in Qualitative Research

Even though reliability is not quite applicable in qualitative research, a different criterion, disciplined subjectivity is proposed. Disciplined subjectivity is the researcher's rigorous self-monitoring, continuous self-questioning and reevaluation of all phases of the research process. There are following strategies to minimize research bias, thus to reduce the subjectivity:

Strategies to Monitor and Evaluate Researcher Subjectivity and Perspective
StrategyDescription
Peer debrieferSelect a colleague who facilitates the logical analysis of data and interpretation
Field logMaintain a log of dates, time, places, persons, and activities to obtain access to informants and for each data set collected
Field journalRecord the decisions made during the emerging design and the rationale; include judgments of data validity
Ethical considerations recordedRecord the ethical dilemmas, decisions, and actions in field journal
Audibilityrecord data management techniques, codes, categories, and decision-rules as a "decision trail"
Formal corroboration of initial findingsConduct formal confirmation activities such as a survey, focus groups, or interviews

Source: McMillan and Schumacher (1997)

Standards of Adequacy in Qualitative Research

Qualitative designs are judged by several criteria. Below are typical questions that researchers might ask of their designs or that reviewers may use to critique a qualitative design:

  1. Is the phenomenon to be studied clearly articulated and delimited?
  2. Is the purpose of the case study described?
  3. Which purposeful sampling technique to identify information-rich cases will be used? Does the sampling strategy seem likely to obtain information-rich groups or cases?
  4. Is the desired minimum sample size stated? Does the sample size seem logical to yield rich data about the phenomenon within a reasonable length of time?
  5. Is the design presented in sufficient detail to enhance validity; that is, does it specify essential strategies such as prolonged field work, collection verbatim accounts with descriptive data, and negative case search?
  6. Which multiple data collection strategies are planned to increase the agreement on the description of the phenomenon and its meanings between the researcher and participants? Does the researcher have knowledge and experience with the proposed strategies, or has he or she done a preliminary study?
  7. Does the design suggest the emergent nature of the study?
  8. Which strategies does the researcher plan to employ to minimize potential bias and observer effect?
  9. Which design components included to encourage the usefulness and the logical extension of the findings? Are there that which could be employed, and if so, which ones?
  10. Does the researcher specify how informed consent, confidentiality, anonymity, and other ethical principles will be handled in the field?

References

  1. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research on teaching. In N. L. Gage (Ed.), Handbook of research on teaching. Chicago: Rand NcNailly.
  2. Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.
  3. McMillan, J. H., & Schumacher, S. S. (1997). Research in Education: A Conceptual Introduction. New York: Longman.
  4. Internet reference: Bill Huitt's page on Internal and external validity
  5. Internet reference: Judy Norris's Qualitative Research Page