Notes on Program Evaluation (based on Evaluation Methodology Basics, E. Jane Davidson, Sage Press, 2005)

  1. Identify Evaluative Criteria
    1. More than program goals
    2. Needs assessment

                                                               i.      Identify consumers/those impacted

                                                             ii.      Needs vs. wants

    1. Two-step approach

                                                               i.      Identify and document performance needs

                                                             ii.      Investigate underlying causes of performance needs

  1. Dealing with Causation
    1. Ask observers
    2. Does content match the outcome?
    3. Are there other telltale patterns that suggest one cause or another?
    4. Does timing of outcome make sense?
    5. Is “dose” logically related to “response”?
    6. Compare with “control” group
    7. Control statistically for extraneous variables
    8. Identify & check underlying causal mechanisms
  2. Determining importance
    1. Stakeholders/consumers “vote” on importance
    2. Draw on knowledge of selected stakeholders
    3. Evidence from literature
    4. Specialist judgment
    5. Evidence from needs and values assessments
    6. Evidence of causal linkages
  3. Criteria for evaluating evaluation
    1. Validity
    2. Utility
    3. Conduct
    4. Credibility
    5. Costs




2005 A.J.Filipovitch
Revised 8 June 2005