Education Reform Depends on Using Outcome Data to Make Hard Choices

Caroline Hoxby returns repeatedly to the same policy problem: education systems often possess—or could readily produce—useful information, but information changes outcomes only when it is allowed to affect pay, lending, and institutional decisions. The hard part is frequently not measurement. It is accepting the unequal treatment that measurement can justify.

For teacher pay, Hoxby argues that a formal value-added system is necessary not because administrators know nothing about teacher quality, but because their judgments alone cannot establish that compensation decisions are objective. An administrator may have a sound view of who is effective; without statistics, there is no reliable way to distinguish that judgment from favoritism.

In the states that have pushed furthest toward differentiated compensation, Hoxby says, a substantial portion of extra pay is generally required to rest on a statistically rigorous value-added calculation. Districts can still offer bonuses for other work or contributions. But the core pay distinction needs to be measurable enough that it can be checked.

Wisconsin is her main example of the difference between permitting a policy and making it universally adopted. Act 10, in 2010, removed the requirement that districts pay teachers through lockstep, or “staircase,” salary schedules. It did not supply additional education funding. It gave districts discretion to differentiate compensation, and some moved toward systems based largely, though rarely entirely, on value-added.

That makes Wisconsin particularly useful evidence in Hoxby’s view. Because Act 10 gave districts greater pay-setting autonomy without giving them more money, she sees the state as a long-run setting for examining what districts did with that discretion. Foundation-backed pilots can also help districts create differentiated-pay systems, but Hoxby considers Wisconsin more informative because the policy change was statewide and did not depend on an outside funding intervention.

The politics remained consequential. Hoxby says teachers’ unions strongly opposed Act 10, including through protests at the state capitol and an attempted recall of the governor. Adoption subsequently varied by district: some embraced differentiated pay quickly, while others did not, in part because of union or superintendent resistance. States without formal teachers’ unions can still have “meet and confer” arrangements that resemble collective bargaining, though Hoxby says they may leave more room for local discretion. A permissive law, in her account, does not mechanically produce performance pay. In many places, the limiting factor is willingness to make and defend differentiated decisions.

The same trade-off appears in Hoxby’s discussion of Teach for America: a school can get teachers who are relatively less trained and likely to leave after two or three years, but who may arrive with unusually high aptitude, motivation, and energy. The retention problem matters because, she says, teachers typically improve during their first years in the classroom; after the fourth year, she says, improvement is limited.

Hoxby cites research by Sally Hudson, now a professor at the University of Virginia, on whether Teach for America participants compensate for their shorter expected tenure. According to Hoxby’s description of Hudson’s findings, these teachers enter with sufficiently high value-added and improve fast enough that schools can be better off using them even when they know they will need to recruit replacements. The point is not that training and retention are irrelevant. It is that staffing decisions should weigh those costs against measured initial effectiveness rather than treating years of formal preparation or expected tenure as complete proxies for teacher quality.

For Hoxby, equitable access to higher education does not require lenders and schools to disregard whether students are making progress. It requires structuring aid so that students—particularly those who enter college with less information about how to choose and navigate it—can test the opportunity before taking on substantial debt.

Caroline Hoxby argues that low-income students may be less able to assess which college is a good fit, negotiate the process of enrollment, or arrive with strong secondary-school preparation. That uncertainty has value: students should have the option to enroll and discover whether college is productive for them. But it is a weak basis for making the largest borrowing decisions at the outset.

Current Pell Grants are distributed in the same amount in each year of college. Hoxby sees no inherent reason that aid must be flat over time. She would put more grant support at the beginning and make loans increasingly important later—a back-loaded lending structure rather than one that asks students to borrow heavily as freshmen.

By the junior year, she says, a student should have more evidence about whether they are succeeding, interested in the work, and well matched to the institution. The school knows more as well, and so can a lender. The shift need not be abrupt, but it would reduce grants and raise borrowing as the student’s college record becomes clearer.

That formulation assigns responsibility to all three parties. Students should face some incentive to make productive choices; schools should have some exposure to the consequences of enrolling students who do not complete or benefit; and lenders should not extend loans at interest rates that are inappropriate to the actual risk. Better underwriting is Hoxby’s answer to the concern that risk pricing can simply burden disadvantaged students with higher costs. Rather than ignoring risk, she would delay the major lending decision until lenders can assess it with more relevant evidence.

A lender should not treat every student, institution, academic trajectory, or degree path as equally likely to produce repayment. Hoxby says the college a student attends is probably the single most predictive factor, but underwriting should also consider aptitude, progress toward a degree, failed classes, stopping out, lost credits, field of study, and plans for graduate school.

The United States has unusually rich data for this task, she says: tests, grades, high-school course-taking, college enrollment, majors, and academic progress. In some respects, lenders have more information about students than mortgage underwriters have about borrowers and houses. Machine learning, she adds, makes sophisticated underwriting more feasible.

Hoxby connects the problem to stagnant on-time completion. Despite a substantial increase in the number of people going to college, she says, the United States’ on-time graduation rate has not risen since the early 1970s. She reads that pattern alongside students dropping out, accumulating too few credits, restarting at other institutions, and entering degree paths associated with low expected earnings. Lending that ignores those patterns, she argues, makes defaults more likely without necessarily improving students’ prospects.

Hoxby’s assessment of California community colleges begins with the state’s ability to link students’ college records to later outcomes when they remain in California. That data access, she says, has enabled economists including Scott Carrell to study the system more closely than many other two-year sectors.

Caroline Hoxby says California’s community colleges perform reasonably well by U.S. standards and better than for-profit two-year institutions. Her comparison is pointed: for-profit colleges, she says, often have low graduation rates, weak student progress, degrees with little value, and no earnings increase for many students. Community colleges do better than that, though she does not treat them as maximally productive or beyond improvement.

Their strongest role, in her description, is awarding two-year degrees. Their weaker role is the pathway sometimes imagined by prospective students: beginning at a community college and transferring reliably to a selective four-year institution such as Berkeley or UCLA. Such transfers occur, Hoxby says, but rarely.

A community-college system can therefore provide real value without functioning as an interchangeable first stage of selective four-year education. Hoxby’s claim is limited to California’s system, where linked in-state outcome data make the comparison possible.

The case for nationally uniform education policy is constrained, Hoxby argues, by who actually governs and pays for schooling. Caroline Hoxby says the federal government contributes only about 6.5 percent of total K–12 spending, while states and localities bear roughly 93 percent. Large states including California, Texas, and New York spend more on K–12 and higher education than the federal government does.

That fiscal structure makes it difficult for Washington to tell states that they cannot use different tests, teacher-evaluation systems, or school policies. Hoxby would prefer more standardized data collection and reporting; comparable information matters to researchers and to policymakers trying to assess results. The federal government already requires states to collect and report some data in common forms. But she does not conclude that all states should use the same tests or pursue identical policies.

Instead, she sees states as “little laboratories.” The variation is inconvenient for analysis, but it permits policy learning. Wisconsin’s experience with teacher-pay discretion can make differentiated compensation more plausible elsewhere. Hoxby also points to Massachusetts as influential through improvements to testing and efforts to create greater socioeconomic diversity in schools.

Texas, she says, is a plausible state for broader value-added pay because it has strong education data and can track long-term outcomes for people who stay in the state. If Texas performed the statistical calculations centrally, rather than requiring each district to build its own technical capacity, Hoxby believes many districts could implement such a system. Her caveat is that smaller, sparsely populated states may lack the infrastructure to do the same thing easily.

The distinction is important to her feasibility argument. In a state with sufficient data and administrative capacity, calculating value-added need not be costly. The harder question is whether districts will make compensation decisions that recognize the resulting differences among teachers.

Institutional autonomy matters beyond pay systems. Caroline Hoxby says the typical U.S. school year is about 178 days and may have become slightly shorter over time as professional-development days have increased. The long summer break originated in an agricultural context, she says, when children were needed for farm work during the busiest season. Other countries divide the school year differently, with shorter and more frequent breaks.

Hoxby agrees that a long summer can widen differences between students whose families can provide educational activities, museum visits, travel, and enrichment and those whose families cannot. The same disparity can exist on weekends and during other time away from school. She does not offer a single mandated calendar, but argues that the 178-day year and extended summer should be reconsidered rather than accepted as a default.

Charter schools illustrate the discretion she has in mind. They are public schools and do not receive more money merely because they are charters, Hoxby says, but they can allocate resources and set policies differently. Some operate year-round, shorten the summer break, or distribute breaks across the year. The underlying question is similar to the one she raises on teacher pay and lending: whether systems can use their freedom to depart from a uniform practice when the existing default does not serve every student equally well.