7. Finding the Best Evidence

Thunyarat Anothaisintawee

7.1 Determining the Impact of Behavior Change on NCDs Through Research

Knowledge changes constantly. For this, if for no other reason, non-communicable disease (NCD) managers must be able to find up-to-date evidence and to interpret and integrate that evidence into their local decision-making. One example is the effect on health of low-calorie sweeteners, as seen in Case Study 7.1.1. Evidence about effectiveness and cost-effectiveness usually come from research findings, so understanding the characteristics, advantages and disadvantages of various types of study design is important for NCD managers if they are to use evidence to good effect in their local contexts.

Case Study 7.1.1 Knowledge growth: A case study of low-calorie sweeteners

Sugar is one of the unhealthiest diet ingredients. Consumption of excessive amounts of sugar can cause diseases like obesity, type 2 diabetes mellitus (T2DM) and heart disease. Despite this knowledge, current intake level is very high. In 2012 the average intake among U.S. adults was 77 grams per day, equal to 19 teaspoons or 306 calories.1 For this reason, the American Heart Association Nutrition Committee recommended a decrease in added sugar in sweetened products. By contrast, low-calorie sweeteners (LCSs) contain few or even no calories while providing an intensely sweet taste. In addition, LCSs do not cause the same metabolic responses in the human body as sugars. These desirable properties make LCSs an attractive substitute for sugar from a public health perspective. LCSs are usually recommended for obese people and T2DM patients in order to reduce their weight and control their blood sugar levels.

However, after using LCSs as substitute for sugar for many years, the shocking evidence from several observational studies was that using LCSs was associated with weight gain and increased the risk of T2DM. These findings were the very opposite of the original belief that LCSs were safe. Despite this evidence, the potential harmful effect of LCSs is debated thanks to inconsistencies in the findings between observational studies and intervention trials. The American Heart Association has recommended against the consumption of LCS beverages by children and encourages the use of water (plain, carbonated and unsweetened) rather than LCSs as an alternative to sugar-sweetened beverages.2 Based on the available evidence, the potential adverse effects of LCSs are still inconclusive and further research on the association between LCSs and risk of CVDs and cardio-metabolic risk factors is needed.

7.2 Types of Study Design

There are two main types of study design: quantitative and qualitative3. A quantitative study is useful for assessing the burden of diseases, exploring the association between potential risk factors and diseases and estimating the benefit of the intervention for the prevention and treatment of diseases. A qualitative study is useful for understanding the process of implementing an intervention, how the intervention works and what the obstacles are to implementing the intervention in practice.

There are several subtypes of quantitative studies4 as presented in Figure 7.1. Two broad types are experimental and observational. Experimental studies usually allocate subjects randomly into intervention and non-intervention groups (arms of the trial). Such studies are called randomized-controlled trials (RCTs). Studies using non-random methods, as when patients are allocated to the arms of a trial according to their birth dates, are called quasi-experimental studies.

Observational studies investigate the relationship between exposures and outcomes. Common types are case-control, cohort and analytic cross-section.5 If researchers select cases (for example, subjects having interested outcomes) and controls (subjects not having interested outcomes) and compare the odds of exposure between cases and controls, we have a case-control study. If researchers select an interesting group or cohort of the population, measure its exposure or treatment and follow the subjects up until there are outcomes, we have a cohort study. A cohort study usually quantifies the effect of exposure as relative risk, or the probability of the outcome in the exposed group compared with that in the non-exposed group. In analytic cross-sectional studies, researchers select a group of the population, as in a cohort study, but with no follow-up. Exposure and outcome in a cross-sectional study are measured at the same time.

An observational study may be analytical or non-analytical. An analytical study is one that measures a relationship between two variables, like the relationship between interventions or exposures and outcomes. A non-analytical study typically describes characteristics of the population, such as the burden of disease and changes in it, by measuring prevalence and incidence. Descriptive studies commonly consist of case reports, case-series and cross-sectional studies. Analytical studies can be divided into experimental studies, in which researchers assign interventions or exposures to subjects, and observational studies, in which exposure and occurrence of disease are measured as they occur, without experimental controls.

7.3 Quality Assessment of Studies/Evidence

Different study designs have different advantages and disadvantages. RCTs are most free from bias, especially selection bias, because the subjects are assigned to intervention or control groups randomly so that other determinants (confounders) are also randomized.6 This yields greater certainty that any difference in outcomes between intervention and control groups is attributable to the intervention rather than other factors. RCT is the most appropriate design for assessing treatment efficacy. However, RCT is not suitable for investigating disease risk factors because it is unethical to allocate subjects randomly to potential harmful exposures. For such investigations, an observational study design is more appropriate. Findings from RCTs cannot always be generalized to non-experimental settings, because the confounding factors controlled for in the trial may be important determinants of the link between causes and consequences in real-world settings. The RCT is highly suited to testing hypotheses about cause and effect, but not so suited to making predictions about outcomes in normal practice. This test is sometimes termed ‘internal validity’. Studies that attempt to predict consequences in real-world settings seek ‘external validity’.

Fig. 7.1 Types of study design.

Note: Outcome = O and Exposure = E.

Observational studies are liable to suffer from confounding bias.7 This occurs when the measured association between exposure and outcome is distorted by the presence of other factors. These other factors are termed confounding factors or confounders.

Amongst observational studies, the cohort design has the highest validity, because it allows cause to precede effect, so that a temporal relationship between intervention and outcome can be claimed. These studies are, however, time-consuming because of the need to follow subjects up until the outcomes occur. This design is not suitable for rare diseases or for those with a long latent period, such as most cancers.

Case-control and cross-sectional studies can overcome some of the problems with cohort studies because measurements of intervention/exposure and outcome occur at the same time but the temporal relationship cannot then be understood. Moreover, the case-control design is prone to recall bias because participants are asked to think back to whether or not they received the intervention. People with the disease tend to remember more of the exposure than those without it.

7.4 Types of Evidence Synthesis

Due to the huge and increasing volume of evidence, its synthesis integrates types and sources of evidence into a coherent review. This is called evidence synthesis. These reviews are of two main types, narrative and systematic, as illustrated in Figure 7.2. Systematic reviews have higher validity since its review processes (scope of the review, inclusion/exclusion criteria, selection of studies, data analysis, resolution of disagreements between reviewers) are explicit, transparent and have to be reproducible by other researchers.

Not only have the number of primary researches increased hugely, but the number of systematic reviews has also risen. Many are published every day,8 so it is impossible for NCD managers or policy-makers to remain up to date with specific topics. Systematic reviews also usually focus on a specific topic to answer a specific question and so may not provide a comprehensive picture or perspective on complex conditions or problems, which is the usual situation in policy decision-making. For these reasons, umbrella reviews, or overviews of reviews, have been developed. These are tertiary researches that combine data from several systematic reviews that are relevant to a particular health problem.9 Umbrella reviews apply similar methods to those of systematic reviews but aim to provide a more comprehensive evidence synthesis, by including, for example, evidence of the effectiveness of different interventions for the same condition, or the same intervention for different conditions or populations. The umbrella review is useful for providing a general idea of research in a specific area and also for providing information when the existing evidence about a given topic is inconsistent or contradictory. An example of the umbrella review is presented in Case Study 7.4.1.

A further benefit of umbrella reviews is the speed with which they can be done. For pragmatic reasons, reviews that can synthesize the evidence quickly are likely to be most attractive to NCD managers and policy-makers. Umbrella reviews that consider previous systematic reviews rather than primary researches can save time in work and rapidly provide evidence to inform policy decision-making.

Fig. 7.2 Hierarchy of evidence synthesis.

Case Study 7.4.1 Efficacy of lifestyle interventions and effect of lifestyle factors on the risk of type 2 diabetes mellitus, cardiovascular diseases and hypertension: An umbrella review

Health-harmful behavior such as eating a poor diet, physical inactivity, inadequate sleep time, use of tobacco and alcohol, all increase the risk of NCDs. Health-harmful behavior also increases the burden of NCDs by increasing their metabolic risk factors, including being overweight/obese, abnormal blood pressure and unhealthy glucose and lipid levels. To prevent and control NCDs, these metabolic risk factors should be reduced by modification of harmful lifestyle behavior. This is an umbrella review of the efficacy of lifestyle interventions for the primary prevention of type 2 diabetes mellitus (T2DM), hypertension and cardiovascular disease (CVD), and the risk effect of harmful behavior (poor diet, physical inactivity, smoking, alcohol drinking and inadequate sleep time) on T2DM, hypertension and CVD. The methods used are in the Online Appendix 7.

Two-hundred and sixty-seven systematic reviews and meta-analyses (SRMAs) of interventions for T2DM, hypertension and CVD met our inclusion criteria and are included in the umbrella review. Of these, 70 were on T2DM, 127 on hypertension and 70 on CVD. Lifestyle interventions considered in the review were diet, physical activity, combined diet control and physical activity interventions, smoking cessation, alcohol drinking and sleep interventions. The effects of each intervention are summarized below.

• Food patterns (e.g., Mediterranean, DASH and diets with high HEI and AHEI scores) reduced the risk of T2DM, CVD and high blood pressure, while the evidence on food groups and food nutrients show conflicting results.
• The findings from this review were similar to those of other reviews and confirm the benefit of moderate and high intensity physical activity in the prevention of T2DM and CVD. However, our review found that low intensity physical activity, such as walking, could also lower the risk of each condition.
• Evidence from systematic reviews and meta-analyses (SRMAs) of RCTs strongly supports the advantage of several lifestyle interventions in the prevention of T2DM and lowering blood pressure. However, the RCTs showed no significant benefit of lifestyle interventions in the case of CVD.
• There was a J-curve association between alcohol and risk of CVD: moderate but not high alcohol intake significantly decreased the risk of CVD, when compared with non-alcohol intake. However, alcohol reduction in people who regularly drank reduced the risk of T2DM and blood pressure level.
• Sleep is one of the lifestyle factors that was associated with a risk of NCD. People who sleep less than 7 hours/day had significant higher risk of T2DM, CVD and hypertension than people who sleep 7–8 hours/day.

7.5 Role of Environmental Interventions in Changing Health Behavior

The findings from the umbrella review show that health-promoting behavior significantly reduces the risk of developing T2DM. However, encouraging people to change their long-term unhealthy habits and maintain the new behavior for months or years is challenging. Lifestyles are not determined only by individual preferences, but also by sociocultural determinants (i.e., social norms and networks) and environmental influences (e.g., workplace and school environments, city plan and public transport).10 Motivating people to change their unhealthy lifestyles using only individual-based or health-system strategies might therefore be insufficient to achieve broad success, though applying policy- or population-based approaches by modifying social and environmental factors are likely to be important.

Policy- or population-based interventions target the entire population. These interventions are usually classified into six types:

• behavior-change communication and mass media campaigns,
• front-of-pack labeling and consumer information,
• taxation subsidies and other economic incentives,
• school and workplace interventions,
• local environmental changes and
• direct restrictions and controls.11

Policy interventions that are cost-effective by the WHO-recommended cost-effectiveness ratio of ≤100 I$ per DALY averted12 include: reducing exposure to risk factors such as unhealthy diets and physical inactivity through front-of-pack labelling of salt content, establishment of a supportive environment for lower-sodium options to be provided in public workplace cafeterias and implementing wide public education and awareness of the benefits of physical activity through mass-media campaigns. The status of these measures as Best or Wasted Buys is discussed in Chapters 4 and 5, respectively. Problems in evaluating their cost-effectiveness include the limited nature of the behavior changes actually induced and the time taken for effects to emerge. Additionally, several policy interventions (sugar tax is one) cannot be randomly assigned at the population level, so experimental research designs like RCTs are inappropriate and one needs to turn to natural experimental methods.

Natural experimental studies are called for when an RCT is impractical or unethical, the intervention in question is likely to have a significant health impact but there is uncertainty about its effect size and there is the potential for replication or generalizability of the study.13 This study design is more susceptible to error through omitted variable bias, loss to follow-up and misclassification of exposure and outcomes. Since the intervention cannot be randomly assigned in the population, this study design affords less protection against selection bias or confounding resulting from selective exposure to the intervention. Explicit multivariate modelling, with accurate measurement of exposures, outcomes and potential confounders, in addition to using a large sample size to detect the expected effect, are crucial.

7.6 Conclusion

This chapter has reviewed the armory of research designs that may be called into use in understanding causes and effects in NCD prevention and treatment. The SEED Tool in Chapter 3 recommends systematic reviews or umbrella reviews as useful in answering the fundamental question concerning the theoretical basis of an intervention’s effect, which helps to identify both causative variables and potential confounders.

We used an umbrella review of systematic reviews to demonstrate the process of evidence synthesis on the efficacy of lifestyle interventions on health-harming behavior for T2DM, CVD and hypertension. The review process and data synthesis took a long time and required an enormous effort from the review team. Whenever possible, therefore, methods should be modified to accelerate the review process and provide the information to the decision-makers in a timely fashion. In addition, the umbrella review cannot replace policy monitoring and evaluation, since the evidence synthesis is used to inform policy development to identify the most effective intervention. However, monitoring and evaluation of policy implementation remains the key component for ensuring the Best Buy policy.