This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Heckman, J. J. Articles by Dabos, M. Search for Related Content PubMed Articles by Heckman, J. J. Articles by Dabos, M. Social Bookmarking                   What's this?

Do We Need Experimental Data To Evaluate the Impact of Manpower Training On Earnings?

University of Chicago and NORC

V. Joseph Hotz

University of Chicago and NORC

Marcelo Dabos

University of Chicago and NORC

This article assesses several recent studies in the manpower training evaluation literature claiming that (1) nonexperimental methods of program evaluation produce unreliable estimates of program impacts and (2) randomized experiments are necessary to produce reliable ones. We present a more optimistic statement about the value of nonexperimental methods in analyzing the effects of training programs on earnings. Previous empirical demonstrations of the sensitivity of estimates of program impact to alternative non experimental procedures either do not test the validity of the testable assumptions that justify the nonexperimental procedures or else disregard the inference from such tests. We reanalyze data from the National Supported Work Demonstration experiment (NSW) utilized by LaLonde and Fraker and Maynard and reexamine the performance of nonexperimental estimates of the net impact of the NSW program on the posttraining earnings of young high school dropouts and adult women. Using several simple strategies for testing the appropriateness of alternative formulations of such estimators, we show that a number of the nonexperimental estimators used in these studies can be rejected. Although we eliminate a number of nonexperimental estimators by such tests, we are able to find estimators that are not rejected by these tests. Estimators not rejected by such tests yield net impact estimates that lead to the same inference about the impact of the program as the experimental estimates. The empirical results from our limited study provide tangible evidence that the recent denunciation of nonexperimental methods forevaluating manpower training effects is premature.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				K. Wells, J. Miranda, M. L. Bruce, M. Alegria, and N. Wallerstein Bridging Community Intervention and Mental Health Services Research Am J Psychiatry, June 1, 2004; 161(6): 955 - 963. [Abstract] [Full Text] [PDF]

				A. J. Reynolds and J. A. Temple Quasi-Experimental Estimates of the Effects of a Preschool Intervention: Psychometric and Econometric Comparisons Eval Rev, August 1, 1995; 19(4): 347 - 373. [Abstract] [PDF]

				J. Grossman and J. P. Tierney The Fallibility of Comparison Groups Eval Rev, October 1, 1993; 17(5): 556 - 571. [Abstract] [PDF]

				R. Moffitt Program Evaluation With Nonexperimental Data Eval Rev, June 1, 1991; 15(3): 291 - 314. [Abstract] [PDF]

				T.D. Stanley and A. Robinson Sifting Statistical Significance From the Artifact of Regression- Discontinuity Design Eval Rev, April 1, 1990; 14(2): 166 - 181. [Abstract] [PDF]

				J. J. Heckman Causal Inference and Nonrandom Samples Journal of Educational and Behavioral Statistics, January 1, 1989; 14(2): 159 - 168. [PDF]