Tracking Student Progress and Scoring Scales
Instructional Strategies Summary Report
"Meta-Analytic Synthesis of Studies Conducted at Marzano Research Laboratory on Instructional Strategies"
August 2009
(475k PDF)
This report synthesizes a series of action research projects that target instructional strategies Marzano Research Laboratory (MRL) has identified for meta-analysis research. The research was conducted between the fall of 2004 and the spring of 2009. The data used for analysis can be found in MRL’s Action Research Meta-Analysis Database.
"Tracking student progress and scoring scales" is a combination of two assessment strategies that involve the use of a scoring scale or guide to preview expected content and the tracking of student progress toward a learning goal.
The following tables show a summary of the findings for the independent action research studies in our Meta-Analysis Database that utilized tracking and scoring scales as a target instructional strategy. For a listing of the independent action research studies that were conducted using this strategy, click here.
Meta-analytic techniques were used to aggregate the findings from the independent action research studies using the statistical package Comprehensive Meta-Analysis (Version 2). In general, meta-analytic techniques are used when the results of independent studies on a common topic are combined. To combine studies that address the same topic in different academic content areas, the results of each study are translated into an effect size. The effect size used in these studies is the standardized mean difference. In very general terms, a standardized mean difference is the difference between the averages of the experimental group minus the control group divided by some estimate of the population standard deviation.
The first table reports the results of a meta-analysis of observed effect sizes for this strategy. The second table reports the results of a meta-analysis when the observed effect sizes have been corrected for attenuation due to a lack of reliability in the dependent measure (i.e., teacher-designed assessments of student academic achievement). When a dependent measure is not perfectly reliable it will tend to lower the strength of observed relationships between independent and dependent variables. Consequently, it is always advisable to correct an observed effect size for attenuation (i.e., decrease in observed effect size) due to unreliability of the dependent measure. Each of the observed effect sizes was corrected for attenuation using .75 as an estimate of reliability. The observed effect sizes were divided by the square root of the reliability to produce the corrected effect size.
Observed Effect Size
|
Number of Studies
|
Weighted Average Effect Size
|
Standard
Error
|
95% Confidence Interval
|
Minimum Effect Size
|
Maximum Effect Size
|
Percentile Gain
|
|
Lower Limit
|
Upper Limit
|
|
14
|
0.92
|
0.25
|
0.43
|
1.40
|
-0.39
|
3.66
|
32
|
Corrected Effect Size
|
Number of Studies
|
Weighted Average Effect Size
|
Standard
Error
|
95% Confidence Interval
|
Minimum Effect Size
|
Maximum Effect Size
|
Percentile Gain
|
|
Lower Limit
|
Upper Limit
|
|
14
|
1.05
|
0.28
|
0.51
|
1.59
|
-0.45
|
4.23
|
35
|
Consulting a table of the normal curve, the overall percentile gain associated with the corrected weighted average effect size of 1.05 is .3531. This means that on the average, the utilization of tracking and scoring scales in the independent action research studies represent a gain of 35 percentile points over what would be expected if teachers did not use tracking and scoring scales.
The effect size reported in the table is a weighted average of all the effect sizes from the 14 independent action research studies. As such, it is considered an estimate of the true effect size of the experimental condition (i.e., use of tracking and scoring scales). The 95% confidence interval includes the range of effect sizes in which one can be 95% certain the true effect size falls. When the confidence interval does not include 0.00, the weighted mean effect size is considered to be statistically significant (p < .05). In fact, the probability associated with the reported effect size is less than 0.001 indicating it is highly significant in laymen’s terms.
More Research on Tracking Student Progress
|
Synthesis Study
|
Focus
|
Number of Effect Sizes
|
Average Effect Size
|
Percentile Gain
|
|
1
|
Frequency of assessment
|
35
|
0.23
|
9
|
|
2
|
General effects of assessment
|
58
|
0.26
|
10
|
|
3
|
Providing assessment feedback to teachers
|
57
|
1.10
|
36
|
|
4
|
Frequency of assessment
|
34
|
0.28
|
11
|
|
4
|
Providing assessment feedback to teachers
|
21
|
0.70
|
26
|
|
5
|
Frequency of assessment
|
233
|
0.40
|
16
|
|
6
|
Frequency of assessment
|
107
|
0.26
|
10
|
|
7
|
Frequency of assessment
|
644
|
0.39
|
15
|
|
7
|
Frequency of assessment
|
622
|
0.39
|
15
|
|
8
|
Frequency of assessment
|
19
|
0.42
|
16
|
|
9
|
Frequency of assessment
|
55
|
0.36
|
14
|
|
10
|
General effects of assessment
|
31
|
0.44
|
17
|
|
11
|
General effects of assessment
|
16
|
0.63
|
24
|
|
12
|
General effects of assessment
|
170
|
1.12
|
37
|
|
13
|
General effects of assessment
|
17
|
0.71
|
26
|
|
14
|
General effects of assessment
|
81
|
1.15
|
37
|
Source: Formative Assessment & Standards-Based Grading. (Marzano, in press)
Synthesis Studies:
-
Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1991). Effects of classroom testing. Journal of Educational Research, 85(2), 89-99.
-
Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. (1991).The instructional effects of feedback in test-like events. Review of Educational Research, 61(2), 213-238.
-
Burns, M. K., & Symington, T. (2002). A meta-analysis of preferential intervention teams: Student and systemic outcomes. Journal of School Psychology, 40(5), 437-447.
-
Fuchs, L. S. & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53(3), 199-208.
-
Gocmen, G. (2003). Effectiveness of frequent testing. Dissertation Abstracts International, A 64/7, p. 2402 (UMI No. 3099579).
-
Hausknecht, J. P., Halpert, J. A., DiPaolo, N. T., & Gerrard, M. O. M. (2007). Restesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92(2), 373-385.
-
Kim, S. E. (2005). Effects of implementing performance assessments on student learning: Meta-analysis using HLM. Unpublished Ph.D., The Pennsylvania State University, PA.
-
Kulik, J., Kulik, C., & Bangert-Drowns, R. L. (1984, April). Effects of computer- based education on elementary school pupils. Paper presented at the Annual Meeting of American Educational Research Association, New Orleans, LA.
-
Lee, J. (2006, April). Is test driven external accountability effective? A meta-analysis of the evidence from cross-state causal-comparative and correlational studies. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.
-
Menges, R. J., & Brinko, K. T. (1986, April). Effects of student evaluation feedback: A meta-analysis of higher education research. Paper presented at the Annual Meeting of American Educational Research Association, San Francisco, CA.
-
Neubert, M. J. (1998). The value of feedback and goal setting over goal setting alone and potential moderators of this effect: A meta-analysis. Human Performance, 11(4), 321-335.
-
Swanson, H. L., & Lussier, C. M. (2001). A selective synthesis of the experimental literature on dynamic assessment. Review of Educational Research, 71(2), 321-363.
-
Travlos, A. K., & Pratt, J. (1995). Temporal locus of knowledge of results: A meta-analytic review. Perceptual and Motor Skills, 80(1), 3-14.
-
Witt, P. L., Wheeless, L. R., & Allen, M. (2006). A relationship between teacher immediacy and student learning: A meta-analysis. In B. M. Gayle, R. W. Preiss, N. Burrell & M. Allen (Eds.), Classroom communication and instructional process: Advances through meta-analysis (pp. 149-168). Mahwah, NJ: Lawrence Erlbaum Associates.