The Fire Triangle: A tool for holding ourselves accountable for intangible outcomes

Transformative Educational Leadership Journal | ISSUE: Spring 2017

We improve that which we measure. But what if we cannot measure all that we value? In this refreshing article, Amelia Peterson introduces an alternative to individual metrics: the fire triangle. This storied approach to data provides a new take on how we can make sense of the complexity of educational change to pursue more holistic goals. By Amelia Peterson

---

What matters is what’s measured. This is an oft-heard refrain in education circles. Sometimes it is declared as a mantra to live by.

From this perspective, measurement is integral to efforts to improve educational outcomes on behalf of kids. Without measurement, advocates say, we cannot motivate progress, know our impact, or compare the benefits of different approaches. Others utter this phrase with a shake of the head, viewing the focus on measurable improvement as myopic and damaging to the range of outcomes or aspects of experience which seem impossible to capture in metrics. Sceptics of the mantra point to phenomena such as teaching to the test, homogenisation of curricula and shallow or short-term learning as unintended consequences of managing change through measurement. Both positions are based on solid ground. There is good evidence that being able to observe and track change via metrics leads to greater improvement over the long term. And from an ethical perspective, the stakeholders of public education – not least students themselves – deserve some reliable indicators that schools are providing high quality learning environments. On the other hand, it is also clear that systems which prioritize a small number of metrics see perverse consequences. Campbell’s Law – that the use of any phenomena as an indicator leads to distortion of the underlying phenomena – is alive and well in education. The simultaneous truth of these two positions requires us to look for a different way to approach measurement of learning environments and metrics of educational change. One of the central challenges that faces any effort to measure educational change is the complexity of what we mean by learning. Do we mean a brain process? Or a phenomenological one? Are we interested in what children are learning day-to-day or only what stays with them over months or years? Must we only focus on declarative or conceptual knowledge they master or can we also consider more tacit knowledge they gain or holistic ways they are developing?

In considering these questions, our inclination is probably to say “it depends”.

The fact that schools operate all around the world everyday without us being able to give a more definitive answer indicates that learning is a concept a little like “the good”: we all know what good means, but try to define it and we quickly run into trouble. We could perhaps describe the qualities of a good man, a good ink pen or a good day out, but we would not expect many qualities to apply to all three. We also wouldn’t be surprised if each of us provided slightly different answers in each case – although we would have firm views about what answers would be absurd or unexpected. Likewise, we can describe what it means to learn to be civil, learn to fish, or learn the periodic table. Again, we may have slightly different views about exactly what is involved in each case, but there are a range of answers we could all probably agree are acceptable and unacceptable. So we can all recognize learning, and its context and function-specific nature. When it comes to measurement, however, the flexibility we can readily apply in a conversation becomes more difficult. So when it comes to monitoring or measuring educational change, we need to abstract from specific learning situations to a more general level. What are the necessary features of any learning experience? What can we hold ourselves accountable for putting in place, and trying to improve on? Like learning, fire is intangible and hard to describe. Only an advanced Chemistry student would be able to tell you what constitutes it or describe its properties. Yet any girls scout could tell you the three entities necessary for its creation: fuel, heat (or ‘a spark’) and air (or oxygen). By describing fire in terms of its requisite parts, we make it tangible. We can do the same for learning. Any learning experience – whether an individual lesson with children or a larger change effort with adults – requires certain elements to make it happen

The first requirement is material.

People have to have something to work on to learn. Whether it is sitting reading a difficult book or engaging in a complex group project, the material people are working on matters. Most educators will be familiar with the notion of the instructional triangle, made up of teacher, student and task. The quality of the task can often be a key determinant of what and how much a student learns. When thinking about material, we can expand the notion of the task to think about a learning environment as a whole. There are certain kinds of knowledge students may only be able to learn when in particular environments (whether real or virtual), such as what adults do around a water cooler, what it’s like to drive a car, or what prayers sound like in a mosque. The material then is our fuel: it is what directs the learning and determines its nature. Different kinds of material support different kinds of learning. Likewise, a fire burning in coal and in kindling will last different amounts of time and have different qualities. The question, “which fuel is best?” often does not make sense. Instead we have to ask, “which fuel is best for the kind of fire I want?” We can, however, say some general things about good properties of fuel, such as that we need a decent carbon content. Likewise, there may be consistent properties of good learning materials. For example, we might ask of different learning materials: how long does the average person spend on this activity before getting bored? Or what proportion of material presented in this way is recalled after a week? Or after a year? These are things we could and sometimes already do measure when we evaluate textbooks, program or software. Usually, however, evaluations are interested only in a final impact score and miss the more granular questions we might have about the qualities of materials. These may be questions that educators can inquire into effectively themselves outside of large-scale evaluations.

The second requirement is some kind of motivation.

Humans are goal-oriented and tend to conserve energy unless something is motivating them. Just as we cannot start a fire without some activation of energy or heat, a learner requires some incentive to get going on an activity. We may then find the activity sustains its own motivation – just as a fire can sustain its own heat – but we do need to keep an eye on it. This requires seeing environments and experiences from the learner perspective: does this individual have a reason to keep going? Can they perceive that reason? Can they feel the heat? Like temperature, motivation cannot be measured from a distance. Heat is not like fuel, which can be picked up from one place and used in another. It only exists in a particular time and place. To judge whether sufficient motivation is present we have to be close enough to a context to understand it through the learner’s eyes. Human perceptions are complicated, and easy to misjudge. Anyone can probably think of a few education policy decisions which seem to be based on misperceptions of how educators or learners feel.

Finally, learning cannot happen without time.

Just as the quickest way to put out a fire is to smother it, depriving it of oxygen, the surest way to hinder learning is to pile on too many concerns and give it insufficient time to happen. The easiest way to make sure oxygen is present is not to try to measure whether it is there but just to create some space. Likewise, the element that is most often missing from learning or change plans is sufficient time. This should be the easiest entity to ensure, however, and perhaps should be the first checked in any accountability process. Our response to a dying fire is often to get out the bellows. Likewise, our first response in evaluating a learning process should be to ask whether it has had sufficient time for change to occur. For a system leader, in particular, creating real time for other adults to learn may be among the most important things they can do. Thinking carefully about the quality of our fuel (material), sustaining our heat (motivation), and leaving room for air (time) are just a few general implications of the fire triangle. Like any metaphor, some parallels may appeal more than others – some may even be misleading. But hopefully it is a flexible metaphor that can be applied to evaluate an individual learning design or a larger change effort. In either case, the questions are:

• Have I got fuel? What do I know about the quality of this fuel? Has anyone tried it before? What makes me think this is the best kind of fuel for this process? Do I know where I can get more if this runs out?
• Have I got heat? Is it sufficiently spread? Where might I have cold patches? Thinking through the process as a whole, when do I think the first heat will die down? What can I do to create more?
• Have I created sufficient time and space for air?

When we make a fire, we can plan in advance what fuel we use, what heat or ignition we have, and how we will build it to create sufficient space. If we had access to expensive measurement tools, we could calculate precisely how long a fire with specific fuel, temperature and oxygen would burn. Most of the time, however, we don’t rely on these tools but instead just keep an eye on it, adjusting each of our elements as we go along to keep the fire burning. To hold ourselves accountable for learning, we can likewise ask the final question: are my feedback loops in place so that I will know if the fire starts to die down and I should adjust something? The question is hard. For an individual teacher, depending on the type of learning process they may feel they have 30 or so individual fires to tend (or maybe hundreds across a school…) will need methods for monitoring each one. For a system leader, the problem may feel compounded. They cannot possibly observe thousands or even millions of fires across a system at once. Monitoring, measuring or evaluating in terms of the fire triangle can help to mitigate this problem of keeping an eye on every fire. The more confident educators can be in their judgments about the condition of the fuel, heat and air, the less chance there is of a fire going out unexpectedly. Thinking about each of the entities separately points to new inquiry questions: which part are we most confident about? Which part are we most unsure about? Different forms of measurement can be applied to the different entities: we may be able to carry out more systematic studies of the quality of different kind of fuel, but we should be wary about drawing generalizations about learning outcomes that might rely on context-specific heat. In this way, the fire triangle draws our attention to interaction effects that are too often ignored if we ask the question “what works?”. The fire triangle also highlights that sometimes it is necessary to really understand what is needed before acting. A fire that needs air can be extinguished by adding too much fuel, likewise excessive air can blow a dwindling fire out. It is the right combination of elements that we must hold ourselves accountable to, and consequently aiming at upping any single metric will almost always have unintended consequences elsewhere in the system. Tending to each part of the fire triangle can help maintain the balance necessary for real improvement.

---