Skip to content

When to make strong recommendations based upon low or very low confidence in effect estimates

In general, we discourage guideline panels from making strong recommendations when their confidence in estimates of effect for critical outcomes is low or very low. We have identified five paradigmatic situations, however, in which strong recommendations may be warranted despite low or very low quality of evidence. 
These situations can be conceptualized as ones in which a panel would have a low level of regret if subsequent evidence showed that their recommendation was misguided.


Paradigmatic situation 

Confidence in effect-estimates for health outcomes 

(Quality of evidence)

Balance of benefits and harms

Values and Preferences

Resource considerations

Recommendation 



Life threatening

situation 

Benefits:

Low or very low confidence

Harms:

Immaterial (very low to high)

Intervention may reduce mortality in a life-threatening situation.  Adverse events not prohibitive

A very high value is placed on an uncertain but potentially  life-preserving benefit

Small incremental cost (or resource use) relative to the benefits justify the intervention

Strong recommendation in favor

Example

Indirect evidence from seasonal influenza suggests that patients with avian influenza may benefit from the use of oseltamivir (low confidence in effect estimates). Given the high mortality of the disease and the absence of effective alternatives, the WHO made a strong recommendation in favor of the use of Oseltamivir rather than no treatment in patients with avian influenza.43

Uncertain benefit, 

certain harm

Benefits:

Low or very low

Harms:

High or Moderate

Possible but uncertain benefit.  Substantial established harm

A much higher value is placed on the adverse events in which we are confident than in the benefit, which is uncertain 

High incremental cost (or resource use) relative to the benefits may not justify the intervention

Strong recommendation against 

(or in favor of the less harmful/less expensive alternative when two are compared) 

Example

In patients with idiopathic pulmonary fibrosis, treatment with azathioprine plus prednisone offers a possible but uncertain benefit in comparison with no treatment. The intervention, however, is associated with a substantial established harm. An international guideline made a recommendation against the combination of corticosteroids plus Azathioprine in patients with idiopathic pulmonary fibrosis.44

Potential equivalence, one option clearly less risky or costly

Benefits:

Low or very low

Harms:

High or Moderate

Magnitude of benefit apparently similar - though uncertain - for alternatives. We are confident less harm or cost for one of the competing alternatives

A high value is placed on the reduction in harm 

High incremental cost (or resource use) relative to the benefits, may not justify one of the alternatives

Strong recommendation for less harmful/less expensive

Example

H. pylori eradication in patients with early stage Extranodal marginal zone B cell (MALT) lymphoma with H. pylori positive.  Low quality evidence suggests that initial H pylori eradication results in similar rates of complete response in comparison to the alternatives of radiation therapy or gastrectomy but with high confidence of less harm/morbidity/cost. Consequently, UpToDate made a strong in favour of H. pylori eradication rather than radiotherapy in patients with MALT lymphoma.45

High confidence in similar benefits, one option potentially more risky or costly

Benefits:

High or Moderate

Harms:

Low or very low

Established that magnitude of benefit similar for alternative management strategies. Best (though uncertain) estimate is that one alternative has appreciably greater harm.

A high value is placed on avoiding the potential increase in harm

High incremental cost (or resource use) relative to the benefits, may not justify one of the alternatives

Strong recommendation against the intervention with possible greater harm

Example

In women requiring anticoagulation and planning conception or in pregnancy, high confidence estimates suggests similar effects of different anticoagulants. However, indirect evidence (low confidence in effect estimates) suggests potential harm to the unborn infant with oral direct thrombin (eg, dabigatran) and factor Xa inhibitors (eg, rivaroxaban, apixaban). The AT9 guidelines recommended against the use of such anticoagulants in women planning conception or in pregnancy.1

Potential catastrophic harm 

Benefits:

Immaterial (very low to high)

Harms:

Low or very low

Potential important harm of the intervention, magnitude of benefit is variable

A high value is placed on avoiding potential increase in harm

High incremental cost (or resource use) relative to the benefits, may not justify the intervention

Strong recommendation against the intervention 

(or in favor of the less harmful/less expensive alternative when two are compared)

Example

In males with androgen deficiency, testosterone supplementation likely improves quality of life. Low confidence evidence suggests that testeosterone increases cancer spread in patients with prostate cancer. The Endocrine Society (USA) made a recommendation against testosterone supplementation in patients with prostate cancer.46

(Table was provided by Gordon Guyatt)

Feedback and Knowledge Base