Reimagining accountability in K–12 education

by brian p. gill, jennifer s. lerner, & paul meosky
February 20, 2017

Author Note

The authors thank Phil Tetlock, Mark Dynarski, Sandy Jencks, Jenny Mansbridge, Mark Moore, Todd Rogers, Christina LiCalsi, Katie Shonk, and the participants in the seminar series of the University of Arkansas’s Department of Education Reform for helpful comments on earlier versions of this article.

Author Affiliation

Gill, Mathematica Policy Research; Lerner, Harvard Kennedy School, Harvard University; Meosky, Harvard Kennedy School, Harvard University. Corresponding author’s e-mail address: [email protected]

References

1. Furgeson, J., McCullough, M., Wolfendale, C., & Gill, B. (2014). The Equity Project Charter School: Impacts on student achievement. Cambridge, MA: Mathematica Policy Research.

2. Tucker, M. S. (2014). Fixing our national accountability system. Washington, DC: National Center on Education and the Economy.

3. Darling-Hammond, L., Wilhoit, G., & Pittenger, L. (2014). Accountability for college and career readiness: Developing a new paradigm. Education Policy Analysis Archives, 22(86).

4. Center on Reinventing Public Education. (2014). Designing the next generation of state education accountability systems: Results of a working meeting. Retrieved from http://www.crpe.org/sites/default/files/CRPE_designing-next-gen-state-ed-accountability.pdf

5. Lerner, J. S., & Tetlock, P. E. (1999). Accounting for the effects of accountability. Psychological Bulletin, 125, 255–275.

6. Hout, M., & Elliott, S. W. (Eds.). (2011). Incentives and test-based accountability in education. Washington, DC: National Academies Press.

7. Gold, M. (2010). Accountable care organizations: Will they deliver? Princeton, NJ: Mathematica Policy Research.

8. Patil, S. V., & Tetlock, P. E. (2014). Punctuated incongruity: A new approach to managing trade-offs between conformity and deviation. Research in Organizational Behavior, 34, 155–171.

9. Simonson, I., & Staw, B. M. (1992). De-escalation strategies: A comparison of techniques for reducing commitment to losing courses of action. Journal of Applied Psychology, 77, 419–426.

10. Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627–688.

11. Zhang, Y., & Mittal, V. (2005). Decision difficulty: Effects of procedural and outcome accountability. Journal of Consumer Research, 32, 465–472.

12. Siegel-Jacobs, K., & Yates, J. F. (1996). Effects of procedural and outcome accountability on judgment quality. Organizational Behavior and Human Decision Processes, 1, 1–17.

13. Mendl, M. (1999). Performing under pressure: Stress and cognitive function. Applied Animal Behaviour Science, 65, 221–244.

14. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. New York, NY: Springer.

15. Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment. Seattle, WA: Bill & Melinda Gates Foundation.

16. Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104, 2593–2632.

17. Ferguson, R. F. (with Danielson, C.). (2014). How Framework for Teaching and Tripod 7Cs evidence distinguish key components of effective teaching. In T. J. Kane, K. A. Kerr, & R. C. Pianta (Eds.), Designing teacher evaluation systems: New guidance from the Measures of Effective Teaching project (pp. 98–143). Hoboken, NJ: Jossey-Bass.

18. Dee, T. S., Jacob, B., & Schwartz, N. L. (2013). The effect of NCLB on school resources and practices. Educational Evaluation and Policy Analysis, 35, 252–279.

19. Hamilton, L. S., Stecher, B. M., & Yuan, K. (2012). Standards-based accountability in the United States: Lessons learned and future directions. Education Inquiry, 3, 149–170.

20. Jennings, J., & Sohn, H. (2014). Measure for measure: How proficiency-based accountability systems affect inequality in academic achievement. Sociology of Education, 87, 125–141.

21. Gill, B., English, B., Furgeson, J., & McCullough, M. (2014). Alternative student growth measures for teacher evaluation: Profiles of early-adopting districts. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic.

22. Campbell, D. T. (1976). Assessing the impact of planned social change (Occasional Paper No. 8). Hanover, NH: Dartmouth College Public Affairs Center.

23. Dee, T. S., & Jacob, B. (2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management, 30, 418–446.

24. Carnoy, M., & Loeb, S. (2002). Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis, 24, 305–331.

25. Ahn, T., & Vigdor, J. (2014). The impact of No Child Left Behind’s accountability sanctions on school performance: Regression discontinuity evidence from North Carolina (NBER Working Paper No. 20511). Cambridge, MA: National Bureau of Economic Research.

26. Chiang, H. (2009). How accountability pressure on failing schools affects student achievement. Journal of Public Economics, 93, 1045–1057.

27. Deming, D. J., Cohodes, S., Jennings, J., & Jencks, C. (2013). School accountability, postsecondary attainment, and earnings (NBER Working Paper No. 19444). Cambridge, MA: National Bureau of Economic Research.

28. Chiang, H., Wellington, A., Hallgren, K., Speroni, C., Herrmann, M., Glazerman, S., & Constantine, J. (2015). Evaluation of the Teacher Incentive Fund: Implementation and impacts of pay-for-performance after two years. Washington, DC: U.S. Department of Education, Institute of Education Sciences.

29. Springer, M. G., Pane, J. F., Le, V., McCaffrey, D. F., Burns, S. F., Hamilton, L. S., & Stecher, B. (2012). Team pay for performance: Experimental evidence from the Round Rock Pilot Project on Team Incentives. Educational Evaluation and Policy Analysis, 34, 367–390.

30. Glazerman, S., & Seifullah, A. (2012). An evaluation of the Chicago Teacher Advancement Program (Chicago TAP) after four years. Washington, DC: Mathematica Policy Research.

31. Tyack, D., & Cuban, L. (1995). Tinkering toward utopia: A century of public school reform. Cambridge, MA: Harvard University Press.

32. Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73, 417–458.

33. Snipes, J., Doolittle, F., & Herlihy, C. (2002). Foundations for success: Case studies of how urban school systems improve student achievement. New York, NY: MDRC.

34. Beatty, B. (2011). The dilemma of scripted instruction: Comparing teacher autonomy, fidelity, and resistance in the Froebelian kindergarten, Montessori, Direct Instruction, and Success for All. Teachers College Record, 113, 395–430.

35. Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73, 125–230.

36. Enzle, M. E., & Anderson, S. C. (1993). Surveillant intentions and intrinsic motivation. Journal of Personality and Social Psychology, 64, 257–266.

37. Ross, J., & Staw, B. M. (1993). Organizational escalation and exit: Lessons from the Shoreham nuclear power plant. Academy of Management Journal, 36, 701–732.

38. Patil, S., Vieider, F., & Tetlock, P. E. (2012). Process versus outcome accountability. In M. Bovens, R. E. Goodin, & T. Schillemans (Eds.), Oxford handbook of public accountability (pp. 69–89). New York, NY: Oxford University Press.

39. Baer, R., Hinkle, S., Smith, K., & Fenton, M. (1980). Reactance as a function of actual versus projected autonomy. Journal of Personality and Social Psychology, 35, 416–422.

40. Tyack, D., & Hansot, E. (1982). Managers of virtue: Public school leadership in America, 1820–1980. New York, NY: Basic Books.

41. Glenn, C. L., Jr. (1988). The myth of the common school. Amherst: University of Massachusetts Press.

42. Friedman, M. (1955). The role of government in education. In R. A. Solo (Ed.), Economics and the public interest. Piscataway, NJ: Rutgers University Press.

43. Chubb, J., & Moe, T. (1990). Politics, markets, and America’s schools. Washington, DC: Brookings Institution Press.

44. Coons, J. E. (1992). School choice as simple justice. First Things, 22, 193–200.

45. Gill, B., Timpane, P. M., Ross, K. E., Brewer, D. J., & Booker, K. (2007). Rhetoric versus reality: What we know and what we need to know about vouchers and charter schools. Santa Monica, CA: RAND.

46. Abdulkadiroglu, A., Angrist, J. D., Dynarski, S. M., Kane, T. J., & Pathak, P. A. (2011). Accountability and flexibility in public schools: Evidence from Boston’s charters and pilots. Quarterly Journal of Economics, 126, 699–748.

47. Booker, K., Sass, T. R., Gill, B., & Zimmer, R. (2011). The effects of charter high schools on educational attainment. Journal of Labor Economics, 29, 377–415.

48. Dobbie, W., & Fryer, R. G. (2015). The medium-term impacts of high-achieving charter schools. Journal of Political Economy, 123, 985–1037.

49. Gleason, P. M., Tuttle, C. C., Gill, B., Nichols-Barrer, I., & Teh, B. (2014). Do KIPP schools boost student achievement? Education Finance and Policy, 9, 36–58.

50. Gill, B., & Nichols-Barrer, I. (2014). Charter schools. In D. Brewer & L. Picus (Eds.), Encyclopedia of education economics and finance Thousand Oaks, CA: Sage.

51. Center for Research on Education Outcomes. (2013). National charter school study. Stanford, CA: Stanford University.

52. Wolf, P., Kisida, B., Gutmann, B., Puma, M., Eissa, N., & Rizzo, L. (2013). School vouchers and student outcomes: Experimental evidence from Washington, DC. Journal of Policy Analysis and Management, 32, 246–270.

53. Chingos, M. M., & Peterson, P. E. (2015). Experimentally estimated impacts of school vouchers on college enrollment and degree attainment. Journal of Public Economics, 122, 1–12.

54. Mills, J. N., & Wolf, P. J. (2016). The effects of the Louisiana Scholarship Program on student achievement after two years (Louisiana Scholarship Program Evaluation Report 1). Fayetteville: University of Arkansas, School Choice Demonstration Project.

55. Abdulkadiroglu, A., Pathak, P. A., & Walters, C. R. (2015). School vouchers and student achievement: Evidence from the Louisiana Scholarship Program (NBER Working Paper No. 21839). Cambridge, MA: National Bureau of Economic Research.

56. Zimmer, R. W., & Toma, E. F. (2000). Peer effects in public and private schools across countries. Journal of Policy Analysis and Management, 19, 75–92.

57. Imberman, S. (2011). The effect of charter schools on achievement and behavior of public school students. Journal of Public Economics, 95, 850–863.

58. Bifulco, R., & Ladd, H. (2006). The impacts of charter schools on student achievement: Evidence from North Carolina. Education Finance and Policy, 1, 50–90.

59. Bettinger, E. P. (2005). The effect of charter schools on charter students and public schools. Economics of Education Review, 24, 133–147.

60. Zimmer, R., Gill, B., Booker, T. K., Lavertu, S., Sass, T. R., & Witte, J. (2009). Charter schools in eight states: Effects on achievement, attainment, integration, and competition. Santa Monica, CA: RAND.

61. Winters, M. A. (2012). Measuring the competitive effect of charter schools on public school student achievement in an urban environment: Evidence from New York City. Economics of Education Review, 31, 293–301.

62. Jinnai, Y. (2014). Direct and indirect impact of charter schools’ entry on traditional public schools: New evidence from North Carolina. Economics Letters, 124, 452–456.

63. Chingos, M. M., & Peterson, P. E. (2010). It’s easier to pick a good teacher than to train one: Familiar and new results on the correlates of teacher effectiveness. Economics of Education Review, 30, 449–465.

64. Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2007). Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Economics of Education Review, 26, 673–682.

65. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915–945.

66. Hawley, W., & Valli, L. (1999). The essentials of effective professional development: A new consensus. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 151–180). San Francisco, CA: Jossey-Bass.

67. Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. New York, NY: New Teacher Project.

68. Cowan, J., & Goldhaber, D. (2015). National Board certification and teacher effectiveness: Evidence from Washington (CEDR Working Paper 2015-3). Seattle: University of Washington Bothell, Center for Education Data and Research.

69. Barnwell, P. (2015, February 18). Why schools need more ‘hybrid’ teaching roles. Education Week. Retrieved from http://www.edweek.org/tm/articles/2015/02/18/why-schools-need-more-hybrid-teaching-roles.html

70. Gawande, A. (2011, October 3). Personal best. The New Yorker. Retrieved from http://www.newyorker.com/magazine/2011/10/03/personal-best

71. Newton, L. H., Hodges, L., & Keith, S. (2013). Accountability in the professions: Accountability in journalism. Journal of Mass Media Ethics, 19, 166–190.

72. Porter, A. C., Polikoff, M. S., Goldring, E. B., Murphy, J., Elliott, S. N., & May, H. (2010). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. The Elementary School Journal, 111, 282–313.

73. Ladd, H. F. (2016, May 26). Now is the time to experiment with school inspections for accountability [Blog post]. Retrieved from https://www.brookings.edu/2016/05/26/now-is-the-time-to-experiment-with-inspections-for-school-accountability/

74. Chicago Public Schools. (2014). REACH Students educator evaluation handbook 2014–15. Chicago: Author.

75. Organisation for Economic Co-operation and Development. (2014). TALIS 2013 results: An international perspective on teaching and learning. Paris, France: Author.

76. Organisation for Economic Co-operation and Development. (2014). PISA 2012 results in focus. Paris, France: Author.

77. Pil, F. K., & Leana, C. (2009). Applying organizational research to school reform: The effects of human and social capital on student performance. Academy of Management Journal, 52, 1101–1124.

78. Bryk, A. S., & Schneider, B. (2004). Trust in schools: A core resource for improvement. New York, NY: Russell Sage Foundation.

79. Ashton, R. H. (1992). Effects of justification and a mechanical aid on judgment performance. Organizational Behavior and Human Decision Processes, 52, 292–306.

80. Lerner, J. S., Goldberg, J. H., & Tetlock, P. E. (1998). Sober second thought: The effects of accountability, anger, and authoritarianism on attributions of responsibility. Personality and Social Psychology Bulletin, 24, 563–574.

81. Bodenhausen, G. V., Kramer, G. P., & Susser, K. (1994). Happiness and stereotypic thinking in social judgment. Journal of Personality and Social Psychology, 66, 621–632.

82. Tetlock, P. E. (1985). Accountability: A social check on the fundamental attribution error. Social Psychology Quarterly, 48, 227–236.

83. Sherman, G. D., Lee, J. J., Cuddy, A. J. C., Renshon, J., Oveis, C., Gross, J. J., & Lerner, J. S. (2012). Leadership is associated with lower levels of stress. Proceedings of the National Academy of Sciences, USA, 109, 17903–17907.

84. Hancock, P. A., & Warm, J. S. (1989). A dynamic model of stress and sustained attention. Human Factors, 31, 519–537.

85. Kahneman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.

86. Thaler, R. H., & Sunstein, C. B. (2008). Nudge: Improving decisions about health, wealth, and happiness. New Haven, CT: Yale University Press.

87. Cialdini, R. B. (2007). Descriptive social norms as underappreciated sources of social control. Psychometrika, 72, 263–268.

88. Hallsworth M., List, J., Metcalfe, R., & Vlaev, I. (2014). The behavioralist as tax collector: Increasing tax compliance through natural field experiments (NBER Working Paper No. 20007). Cambridge, MA: National Bureau of Economic Research.

89. Kraft, M., & Rogers, T. (2015). The underutilized potential of teacher-to-parent communication: Evidence from a field experiment. Economics of Education Review, 47, 49–63.

90. Hallberg, K., & Green, G. (2015, March 11). How can we hire and keep high quality teachers in struggling schools? [Blog post]. Retrieved from http://educationpolicy.air.org/blog/how-can-we-hire-and-keep-high-quality-teachers-struggling-schools

91. Furgeson, J., Gill, B., Haimson, J., Killewald, A., McCullough, M., Nichols-Barrer, I., & Lake, R. (2012). Charter-school management organizations: Diverse strategies and diverse student impacts. Cambridge, MA: Mathematica Policy Research.

92. Marsh, J. A., McCombs, J. S., & Martorell, F. (2010). How instructional coaches support data-driven decision making. Educational Policy, 24, 872–907.

93. Blazar, D., & Kraft, M. A. (2015). Exploring mechanisms of effective teacher coaching: A tale of two cohorts from a randomized experiment. Educational Evaluation and Policy Analysis, 37, 542–566.

94. Kane, T. J. (2012). Capturing the dimensions of effective teaching. Education Next, 12(4), 35–41.

95. Chaplin, D., Gill, B., Thompkins, A., & Miller, H. (2014). Professional practice, student surveys, and value added: Multiple measures of teacher effectiveness in the Pittsburgh Public Schools (REL 2014-024). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic.

96. Walsh, E., & Lipscomb, S. (2013). Classroom observations from Phase 2 of the Pennsylvania Teacher Evaluation Pilot: Assessing internal consistency, score variation, and relationships with value added. Cambridge, MA: Mathematica Policy Research.

97. L’Hommedieu, R., Menges, R. J., & Brinko, K. T. (1990). Methodological explanations for the modest effects of feedback from student ratings. Journal of Educational Psychology, 82, 232–241.

98. Taylor, E. S., & Tyler, J. H. (2012). The effect of evaluation on teacher performance. American Economic Review, 102, 3628–3651.

99. Dee, T. S., & Wyckoff, J. (2015). Incentives, selection, and teacher performance: Evidence from IMPACT. Journal of Policy Analysis and Management, 34, 267–297.

100. Rockoff, J. E., Staiger, D. O., Kane, T. J., & Taylor, E. S. (2012). Information and employee evaluation: Evidence from a randomized intervention in public schools. American Economic Review, 102, 3184–3213.

101. Frank, K. (2012, February 24). Constitution for effective school governance [ID No. 16715]. Teachers College Record. Available from http://www.tcrecord.org

102. Schillemans, T., & Smulders, R. (2015). Learning from accountability? Whether, what, and when. Public Performance & Management Review, 39, 248–271.

103. Blume, H. (2016, June 13). Less test-iness over L.A. teacher evaluations. Los Angeles Times. Retrieved from http://www.latimes.com/