URSI 403--Doing Urban Research -- Lecture Notes 9
VIII. Reporting Results
A.. Interpretting the analysis
- What are the possible relationships?
- What are the actual relationships?
- What are the implications of this?
B. Measures of Confidence
- Reliability
- Degree to which measure generates similar response over time
and across situations (mostly affected by random error)
- Test by
- i. test-retest
- ii. alternate forms
- iii. subsample
Each of these techniques may carry validity problems.
- Validity
- Measure of extent to which measures accurately reflect what they are supposed to measure--to what extent are observed differences "true"? (mostly affected by systematic error)
- i. internal validity--effectiveness of measurement within confines of study (Campbell's issues)
- ii. external validity
- representativeness of study group to broader populations
- interaction of testing & experimental conditions
- interaction of selection & experimental conditions
- reactive arrangement
- multiple-experimental condition interference
- Confidence limits
- Estimate of error that can be expected in a measure CL=Mean + 1.96 SE of Mean (SE of M = sd/ n**.5)
- Used to estimate sample size
- Confidence limit for correlation coefficient is r2=1/n-1
- Significance
- Test to determine whether observed results could be explained by chance deviation from expected (average) value
- Assumes normal curve:
- i. Characteristics
- symmetric
- sigma measures distance on baseline
- mu measures center point
- area under curve is expressed precisely by sigma, no matter the height (kurtosis) of the curve
- ii. z-score: any set of normally distributed measures can be standardized to the normal curve
- z=x-mu sigma
- average will be 0
- all values lie between -1 and +1
- Determining significance
- i. decide level of significance (usually .05 confidence level)
- ii. chance deviation will occur up to the "tail" of the normal curve beyond the confidence level
- iii. express confidence level as standard deviation score-- .05 z-score = 1.96, so 1.96 sd=.05 confidence level
- Correlation has own significance distribution, called "F", which is a function of correlation coefficient and sample size.
- i. F=(r2/1-r2)(N-2), df=1, N-2
- ii. Table of F values lists critical values for different sample sizes.
- Sensitivity--combined effect of validity & reliability of procedures
- Constant error--distorts measures in one direction
- Random error--obscures possible effect (distorts measures in all directions)
© 1996 A.J.Filipovitch
Revised 1 September 96