Campbell's Law: Something Every Educator Should Know

As soon as I learned Campbell's Law, how I conceived my teaching changed.

One concept I wish everyone concerned about education were familiar with is Campbell’s law. I certainly wish that I’d known about it twenty or more years ago.

Campbell’s law states that “the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

Campbell’s law was coined by Donald Campbell, a social scientist and founding figure of research methodology. Campbell’s law is perhaps best understood with an example.

Imagine researchers observed that the happiest couples kissed each other goodbye in the morning and that we were then told that the key to a lasting partnership was a peck before parting. While this may be an undeniably nice thing, it’s easy to see that if the peck is not motivated by underlying feelings of love and affection, it does not carry much meaning. If the peck is the point, it becomes meaningless as an indicator of happiness. 

Campbell’s theory was drawn directly from observations of schools and teaching. He noted that achievement tests can be useful indicators of what students are learning provided they are preceded by “normal teaching aimed at general competence.” Once scoring well on the test becomes the point, those tests lose their value as predictors of competence.

Tutoring and preparation for tests like the SAT such as what Educational Endeavors offers are only partly oriented around improving general competence in content areas like reading or algebra because in order to score as highly as possible, it is just as (or more) important to understand how the testing mechanism works. Yet that insider knowledge of what logic test writers follow, and what traps they lay, ultimately does not translate to anything beyond taking the SAT.

With test prep tutoring, at least the bargain is clear. We’re preparing students for a discrete event with potentially high stakes where knowledge of the SAT itself, not just grammar and geometry, is what’s vital. However, while the skills learned through test prep may have benefits in terms of improving focus, executive function, and personal drive, the resulting SAT score is not a strong indicator of how much someone has learned from school or how likely they are to do well in college. 

This is not just because of Campbell’s law, and not just because financial resources often dictate how much test prep a student will receive, but also because the SAT is a single event that cannot measure potential or quantify the many variables necessary for academic success.

There’s little doubt we could do better when it comes to a metric for college preparation, which is why I personally am pleased to see the trend toward eliminating its use.

The SAT is relatively benign compared to what happens when schools or states get in a vicious circle of testing and (so-called) accountability that’s driven by Campbell’s law. The results are damaging and distorting to classroom practices.

I have seen it in generations of writing instruction centered around training students to pass assessments by writing five-paragraph essays utilizing approaches that are in reality counterproductive to actually learning how to write. This is not the fault of teachers or students, but of a system that has mistaken a single indicator for a complex set of skills and knowledge.

I see it also in schools that drill behaviors like sitting straight and tracking a speaker because they view these traits as indicators of “paying attention.” But surface-level indicators of attention are not synonymous with engagement, and are definitely not always an indicator of learning. A classroom that looks to be working with military precision may be satisfying as a snapshot for the purposes of surface-level accountability, but it may also be sacrificing learning for the appearance of order. 

Demanding attention when it hasn’t been earned can also degrade the quality of teaching and instruction. The best thing I ever did for the quality of my college writing classes was to end mandatory attendance. I was now under the pressure of delivering experiences and instruction that students believed were worthwhile and related to their own learning. 

This dynamic required both me and the students to increase the rigor and discipline in our work.

Campbell’s law shows up a lot in the real world and has a lot of genuinely bad effects. Any company that relentlessly sends you a survey after completing a service or purchase is likely in the throes of a dedication to something called a “net promoter score” (NPS). When employees are subject to either reward or punishment for not keeping the NPS above a certain threshold, distortions may result. I recently received an evaluation that said if I didn’t give all 10s on the survey, they’d be following up with a call to see what they could do better next time.

Rather than giving 10s, I ignored the survey. Either way the company’s data is corrupted: either by my nonresponse or by the coercion to give all 10s.

Once you start looking for Campbell’s law at work in the world, you’ll notice it often. When you do, and it’s impacting you, take a step back and ask yourself what is truly meaningful about what you’re trying to achieve and then what kind of evidence would reflect what is meaningful.