Potential Issues

Methodological and Ethical Issues with Case Study

One longstanding critique of case study methodology, most common among quantitative researchers working from a positivist or postpositivist paradigm but largely dismissed by the qualitative/constructivist community, relates to concerns about validity and reliability that arise because of the subjectivity and biases of the researcher (Merriam, 2009). Qualitative case study methodologists generally agree that in this type of research, the researcher is the primary instrument of data collection, and that subjectivity and bias should be acknowledged but cannot be eliminated. Stake (1995) addresses questions about validity and reliability by proposing “triangulation,” a set of “protocols which do not depend on mere intuition and good intention to ‘get it right’” (p. 107).

Protocols I am using to address these concerns include theory triangulation (observing the case from different theoretical standpoints), methodological triangulation (using multiple methods to answer the same question about a case), and member checking (asking participants to review sections of writing in which their words and actions are featured).

In addition to reliability and validity, generalizability has been another frequently cited concern about case study research, again particularly from quantitative and postpositivist corners. The issue arises especially in discussions of instrumental case design, where the case is chosen to illustrate some larger problem or issue. Can the findings of an instrumental case study, conducted in a particular context, ever be generalized to other contexts? Few, if any, would argue that the results of single case study could be generalized in the usual sense of the word—i.e., from a sample to a population. Nonetheless, Merriam (2009) points to the inclusion of more than one case as a strategy for increasing external validity or generalizability. She explains that “readers can learn vicariously from an encounter with the case through the researcher’s narrative description” (p. 51).

Stake (1995) calls this process “naturalistic generalization.” Pointing out that “case study seems a poor basis for generalization,” (p. 7), Stake defines naturalistic generalization as “conclusions arrived at through personal engagement in life’s affairs or by vicarious experiences so well constructed that the person feels as if it happened to themselves” (Stake, 1995, p. 85).

In a case study, then, the researcher can write in such a way as to provide these vicarious experiences for readers, who can take the insights they receive and “generalize” them in ways that are useful to their own contexts. For Thomas (2011), likewise, in an effective case study, “readers are left to make up their own minds; they are left to piece together conclusions for themselves” (p. 208). Through the process of abduction, case study researchers move away from generalization and toward “the ‘exemplary knowledge’ of abduction and phronesis . . . an example viewed and heard in the context of another’s experience . . . but used in the context of one’s own” (Thomas, 2011, p. 215). These understandings of naturalistic generalization and exemplary knowledge make case study an especially appropriate methodology for educational research, where context matters deeply but where similarities among various school contexts may make it easier for educators reading a study to vicariously experience and imaginatively reconstruct and apply conclusions in their own professional contexts.

Other frequently cited limitations of case study include the significant costs in time and money of doing it well, the dependence on the sensitivity and integrity of the researcher for quality control, and the potential for bias or unethical conduct in the selection of data for inclusion in the analysis (Merriam, 2009).

Methodological and Ethical Issues with Practitioner Inquiry

One way of understanding what various modes of practitioner inquiry have in common, and how they collectively differ from other research paradigms, is to analyze common criticisms of this research paradigm. Cochran-Smith and Lytle (2009) identify six broad critiques, having to do with knowledge, methods, science, ethics, personal and professional development, and politics. The epistemological and methodological critiques, which have followed practitioner inquiry from its beginnings (Currin, 2019; Groundwater-Smith & Mockler, 2009), are of special relevance here. Critics of practitioner inquiry often call into question whether practitioners doing research in their own contexts have the methodological skill to do it adequately and whether, in any case, the results of their research count as “knowledge” in the sense generally meant among academics.

Cochran-Smith and Lytle (2009) argue that this critique reflects a static and transmissionist view of knowledge, an understanding of teachers as primarily consumers (not producers) of knowledge, and an effort to “preserve the hegemony of outside expertise” (p. 47). As Finch (2021) points out, however, few empirical studies have examined the systematicity and methodological rigor of practitioner research, leaving it open to this critique. The AERA Panel on Research and Teacher Education recommended evaluating practitioner research by focusing on the use of systematic processes, such as evidence of clear research questions and data collection and analysis methods that fit the questions (Cochran-Smith, 2005). Similarly, and around the same time, the Conference on English Education called for practitioner research that makes its methodological approach explicit and uses systematic, detailed data analysis strategies (DiPardo et al., 2006).

In line with those recommendations, Finch (2021) conducted a content analysis of 74 articles, all sole-authored or first-authored by practitioners doing research on their own practice, published in 10 literacy journals that include practitioners in their target audience. The author found that 85 percent of the authors had or were pursuing advanced degrees. Forty-three of the 74 articles had explicitly stated research questions; 43 also specified a research design, with some of those citing related literature (while many only specified “practitioner inquiry” or “action research,” other designs included case study, ethnography, mixed methods, and experimental research). Forty of the articles collected data from three or more sources, and 17 explicitly considered the trustworthiness of their analyses. Finch (2021) concluded that her findings support the argument that teachers are indeed producers and not just consumers of knowledge but recommended that practitioners engaged in research more clearly describe their systematicity and intentionality and partner with university researchers when possible.

Other perceived limitations include the following charges: that it is not “scientific” because it does not involve random selection, control and experimental groups, and generalization; that it is unethical because practitioners engaged in research cannot avoid conflicts of interest or the coercion of participants; that it is either too innocuous (a critique from “critical” scholars) or too activist (a critique from “traditional” scholars); and that as a form of professional development it tends either to reinforce the status quo or to stroke the ego of the practitioner-researcher (Cochran-Smith & Lytle, 2009).

Potential Methodological and Ethical Issues with This Study

Case study and practitioner inquiry can be combined in a way that exacerbates rather than mitigates the potential problems with each. The two most significant potential limitations of this integrated approach, I believe, concern methodological rigor and ethics; problems in both of these areas seem to arise from a natural hope that a massively time-consuming effort like a major study will produce desirable results.

No doubt everyone involved in my study will want to believe that the planning and instructional strategies and materials we develop are positively influencing student knowledge and behavior with respect to the post-truth phenomenon, that our efforts are making a difference. That desire could inadvertently influence data selection and analysis, which is both a methodological and an ethical problem, and it could also create conflicts of interest between what we hope to find and what we know is best for students. We will have to practice systematic awareness of our own subjectivity (Peshkin, 1988) and especially of any tendency to select data that confirms our bias toward success and to ignore disconfirming data. We will have to decide on criteria for gauging the efficacy of our work and stick to it.

I, especially, as the lead researcher and case writer, will have to be completely transparent about not only my own subjectivity but also about every step in the process, every decision, and the rationale behind it—which will mean systematically writing research memos and recording insights, questions, frustrations, and contradictions. Perhaps even more importantly, we all will have to commit to putting the learning needs of students first, avoiding any student recruitment efforts that could be interpreted as coercion, abandoning ideas when it becomes clear they are not in students’ best interest, and reflecting continually on our own relationship to the work as both researchers and practitioners.

In some ways, integrating case strategies may help mitigate the limitations of using each separately. For example, including practitioner-participants as co-investigators in a case study could potentially address some concerns about cost and time, and about the integrity, sensitivity, and potential bias of the lead researcher; each practitioner could offer input into data selection and analysis and spot potential bias—a form of triangulation. Conversely, the use of case study design, if done well—with references to methodological literature, clear and detailed definitions and descriptions of cases, and transparent and well-organized accounts of data collection and analysis processes—could provide the methodological rigor that is often thought to be missing from some practitioner research.