1. Home
  2. Students
  3. Chapter 11: Using Experiments to Evaluate Programs

Chapter 11: Using Experiments to Evaluate Programs

Chapter 11, “Using Experiments to Evaluate Programs,” addresses the important issue of when formal experiments are the appropriate choice for a given evaluation and what elements need to be addressed when planning an experiment. Common objections to conducting experiments are noted, especially given the cost in additional time and money, and important benefits are noted that justify those additional costs. Important considerations that evaluators should take seriously both in conducting experiments and in interpreting the results are explained. Appropriate statistical analyses for common experimental designs are explained, and substantial additional resources are provided online to assist with conducting those analyses. The chapter concludes with a case study illustrating both the challenge and the value of experiments in evaluations.

1

A guide to statistical analyses

Download

2

Online Power Analysis Resources

URL Comments

https://www.statisticssolutions.com/statistical-power-analysis/

This site provides helpful explanations of statistical power and related concepts. It refers to the idea of computing power and necessary sample size, but some of the other sites provide the actual calculators.

http://clincalc.com/stats/samplesize.aspx

This calculator provides relatively quick and easy calculations for some basic conditions.

http://powerandsamplesize.com/

This site provides a good range of information addressing statistical power and related issues as well as calculators specific to a variety of statistical analyses. Some of the explanations use a number of mathematical and statistical symbols, which may be less helpful for those without strong backgrounds in the area.

http://www.gpower.hhu.de/

G-Power is a useful tool with a range of options, although it generally requires a higher level of understanding issues related to power than some of the other tools.

https://www.statmethods.net/stats/power.html

For those with strong backgrounds in statistics and computers, this site provides very helpful guidance on how to use the free software R for power analyses. The section on power analysis is one of a number that provide instruction on statistical analysis using R.