Doing Successful Literature Reviews

Lesson A-5

Assessing the Interventions (When
Applicable) That Were Studied

Some research examines the effects of an intervention. The intervention may be changes in the physical or social environment, implementation of a new policy, the introduction of an innovative program, or an experimental treatment. The intervention may be naturally occurring, humanly induced (but not at the behest or within the control of the researcher), or it may be induced specifically for the research. While medical research and laboratory psychology experiments can often describe the intervention in one paragraph, that is rarely the case in most educational, human development, and human resource development research.

To adequately understand an intervention, we generally need to know the following:

What was done or happened?
With what intensity?
For what duration?
How?
Where?
When?
To whom?
With what variations for all the above?

One serious problem in educational and worksite demonstrations and experiments is that complex interventions are often not implemented as intended, and that is not reported by the researchers. The results-whether favorable or unfavorable-are mistakenly attributed to the planned intervention rather than the actual one.

The best evidence of the actual intervention is detailed data collection on the above listed characteristics at several points in time over the period the intervention is in place. This is commonly done in qualitative evaluations but less commonly done in quantitative research and evaluation. The second best evidence of the actual intervention is an indication that checks were made to ascertain that the intervention did occur as expected. The third best evidence, applicable to demonstrations and experiments, is an indication that care was taken to train the personnel responsible for implementing the innovation, providing them with the tools needed for implementation, monitoring of implementation, and offering corrective feedback when implementation began to go astray.

Sometimes the intended intervention is deliberately altered during the study to correct shortcomings and take advantage of opportunities that had not initially been recognized. That is fine, provided the actual intervention, as it evolved over time, is well documented. More often, educational and human resource interventions fall short of the plans because of inadequate planning, training, and monitoring; because of insufficient resources; or because of a lack of cooperation and sometimes sabotage by those disapproving of the intervention. It is important to know when that is the case, but if the researchers are responsible for the intervention, there is a tendency for them not to report their implementation shortcomings.

Key Assessment Questions: 5. What intervention was actually implemented, with what intensity and duration, and how did it perhaps vary across subjects, time, and other important dimensions?

Return Home or Advance to Lesson A-6

Last Update: June 29, 2000