Funders shouldn’t misinterpret WASH-B and SHINE results as “WASH doesn’t work”

There has been a lot of debate about the well-designed and well-conducted WASH-B (Kenya, Bangladesh) and SHINE (Zimbabwe) trials of rural WASH interventions in the past year or so. Most recently, researchers active in WASH epidemiology published a consensus paper. Many funders may not read it, which would be a shame, as it is easy to misinterpret the WASH-B and SHINE findings as “WASH doesn’t work”. If funders make this misinterpretation, people will miss out on life-changing WASH services. In this post, I discuss the incremental changes the interventions delivered, how epidemiologists have interpreted the results, and how funders should interpret them.

In summary, I think that funders should read the consensus paper, and draw two conclusions:

1. Most WASH programmes make small incremental improvements in service level which may not bring substantial health gains in the short-term, but do have a good chance of doing so.
2. Funders should continue to support “basic” services and the systems that sustain them, in the knowledge that later incremental improvements can build on those investments to achieve “safely-managed” services further down the line.

This is not an argument that “transformative” WASH services are not achievable or desirable. In some settings they will be. Rather, I mean to point out that sustaining safely-managed services may cost more than rural households in LMICs are able and willing to pay in the medium-term. Small incremental improvements are more realistic and affordable, both for users and funders (in which I include LMIC governments).

1. What were the interventions and results?

In understanding the WASH-B and SHINE results, two things are often overlooked: the baseline conditions, and what incremental improvement the interventions made upon them. This is a point that is similar to, but different from, discussions about external validity (which I don’t plan to get into here).

Table 1 at the bottom of this post summarises baseline conditions and interventions for each WASH component in the three trials. [I haven’t seen this analysis anywhere else, but let me know if someone else did it in more depth already]. Also at the bottom, and for completeness, Table 2 summarises the trials’ main results, though most readers will already be aware of them (no impact in the WASH arms on any key outcomes, except on diarrhoea in WASH-B Bangladesh).

The first point I want to make is that, given the baseline conditions and small incremental improvements achieved (a term I differentiate from “marginal” here – incremental improvements can be large), these interventions did not constitute a typical “WASH” programme. Let’s explore this a little.

Baseline conditions: the highest priorities under SDGs 6.1 and 6.2 should be achieving basic services for all first (as I have previously argued), by ending open defecation (OD) and ending the drinking of water from unimproved and/or far-away sources. See definitions of basic for sanitation and water. The WASH-B settings in particular were already much better than what one might think of as “poor WASH conditions”. For example, there was ~95% latrine access in both Kenya (though mostly unimproved) and Bangladesh. In Kenya there was ~75% improved water, which the time-to-source data suggests fulfilled the “basic” SDG criterion for most. In Bangladesh, it appears most people used on-plot tubewells. DHS 2014 shows that >95% of rural Bangladeshi had “basic” SDG water (and 74% on-plot water). Baseline sanitation conditions were worse in SHINE (~50% OD). Overall, then, WASH conditions in these settings were far from terrible, particularly in Bangladesh.

Incremental changes: the incremental changes to fairly good baseline service levels were small in most cases. Take sanitation: the WASH-B Kenya intervention was to replace an unimproved latrine slab with a lidded plastic one. This is an improvement, to be sure, and probably valued by the user. But not a big change – it is still a pit latrine without a water seal. In WASH-B Bangladesh, ~95% of people used an improved latrine with a concrete slab (and ~30% a water seal), so the change was the switching to a pour-flush in most cases. Again, an important but small improvement. I don’t say this to criticise the interventions, but to argue that maybe we shouldn’t expect health gains with such minor improvements. Many WASH interventions make more substantial incremental changes in WASH service levels. Turning to water, interventions in all three trials were chlorination. The most recent systematic review shows that chlorination interventions have no effect on diarrhoea once non-blinding is taken into account. So, it was not surprising that the water arms had no effect. In none of the three trials was any change made to the water supply itself, whether increasing its proximity or reducing its intermittency.

Taking these two points together, baseline conditions were already at or close to “basic”, and the intervention brought them up to “basic”. This is not the kind of incremental change one might expect to have a substantial effect – the systematic review shows that diarrhoea impacts are sensitive to the change in level of service (my comment here). I am not arguing that small incremental changes are unimportant – on the contrary. Cumulatively over time, they can have a big impact. My point is that the kind of incremental changes studied in these trials (i.e. chlorination, toilet interface upgrade) are not the kinds which most WASH programmes currently focus on, or should focus on. More important transitions to characterise as “WASH” would be OD–>basic sanitation (which SHINE did, on the whole) and off-plot unimproved or faraway water –> basic water (i.e. water quantity interventions, which none of these studies did). In short, small incremental changes may be beneficial, but evidently not the very small ones assessed in WASH-B (and SHINE, to some extent) .

2. How have epidemiologists interpreted the results?

The recent consensus paper is the most important contribution to the debate around WASH-B and SHINE. Table 3 shows its five main messages, with my appended reflections. I would summarise their argument as follows: the theory behind the impact of WASH is still sound – we might just need bigger service improvements over longer periods of time to see a measurable health effect. Don’t give up on WASH because, when we change conditions enough to see an effect, history suggests it will be a big one.

What I see as missing in these debates is the value that people get from WASH beyond health. These benefits (e.g. time savings, quality of life and psychosocial factors) are substantial contributors to the overall economic benefits of WASH interventions. We are still very bad at measuring and valuing these benefits. I am biased because measuring quality of life gains from sanitation interventions is what I work on currently… but the point stands.

Table 3: Interpretation of consensus paper messages

Five consensus messages	My reflection
1. “Despite high compliance, the evaluated WASH interventions – as delivered in these settings – had no effect on linear growth, and mixed effects on diarrhea”	Unlike some previous trials, we can’t explain away these results with interventions that didn’t do what they were supposed to. We have to face up to them.
2. “The biological plausibility of WASH interventions as public health interventions is not challenged by these findings”	Finding significant health effects in a trial depends on many environmental and social factors. Trials are important, but the underlying theory supporting an intervention is also important, especially when interventions are particularly hard to test with trials.
3. “Historically, large, population-level gains in child health have not been achieved without significant improvements in WASH services”	Big health gains will take many big cumulative environmental changes. This will take longer than you can assess in a trial, and is likely to come in several incremental steps.
4. “Current evidence suggests that basic WASH services alone are unlikely to have a large impact on childhood stunting”	OK, but basic WASH should deliver important economic benefits (time savings, quality of life). These are likely worth paying for, even in the absence of health gains, and cost-benefit analysis can help answer that question.
5. “The results of these trials do not undermine the new and ambitious SDG target of safely managed services for all”	Achieving “safely-managed” services (which these interventions did not) may be more likely to deliver health impact. However, delivering and sustaining safely-managed services may cost more than rural households in LMICs are willing and able to pay in the medium-term.

3. How should funders interpret the results?

I worry that funders (in which I include LMIC governments) may read the abstracts of these papers, or hear second-hand readings of them, and misinterpret the findings. It is easy to infer that “WASH doesn’t work”, which would be a mistake. It depends on how we define “WASH” and “work”, and on how long we are willing to wait to see cumulative gains.

I think that funders should read the consensus paper, and note that:

• Most WASH programmes make small incremental improvements in service level which may not bring substantial health gains in the short-term, but do have a good chance of doing so.
• “Transformative” WASH might be more effective, but it would also be very expensive, both for me and for the users who need to sustain the service.
• Users value the non-health benefits of WASH services too, such as time savings and quality of life benefits, and these have an economic benefit.

In terms of their actions, funders who consider the bigger picture should:

1. Continue to support “basic” services and the systems that sustain them, in the knowledge that later incremental improvements can build on those investments to achieve “safely-managed” services further down the line
2. Continue to have a broader impact by funding the sector-wide systems that are necessary for further extending and sustaining services for all.

This is not an argument that “transformative” WASH services are not achievable or desirable. Rather, I mean to say that sustaining safely-managed services may cost more than rural households in LMICs are able and willing to pay in the medium-term. In many settings, small incremental improvements will continue to be more realistic and affordable, both for users and funders.

================ 

 Table 1: baseline situations and interventions in the WASH-B and SHINE trial settings, and their incremental changes

Note. this is a summary – full details are in the papers and their supplementary info. Approximate baseline conditions are based on my eye-balling across arms in the studies’ baseline tables – SHINE had 4 arms and WASH-B had 7.

 

Table 2: results of WASH-B and SHINE trials