Machine-learning-based high-benefit approach versus conventional high-risk approach in blood pressure management

Abstract

Background

In medicine, clinicians treat individuals under an implicit assumption that high-risk patients would benefit most from the treatment (‘high-risk approach’). However, treating individuals with the highest estimated benefit using a novel machine-learning method (‘high-benefit approach’) may improve population health outcomes.

Methods

This study included 10 672 participants who were randomized to systolic blood pressure (SBP) target of either <120 mmHg (intensive treatment) or <140 mmHg (standard treatment) from two randomized controlled trials (Systolic Blood Pressure Intervention Trial, and Action to Control Cardiovascular Risk in Diabetes Blood Pressure). We applied the machine-learning causal forest to develop a prediction model of individualized treatment effect (ITE) of intensive SBP control on the reduction in cardiovascular outcomes at 3 years. We then compared the performance of high-benefit approach (treating individuals with ITE >0) versus the high-risk approach (treating individuals with SBP ≥130 mmHg). Using transportability formula, we also estimated the effect of these approaches among 14 575 US adults from National Health and Nutrition Examination Surveys (NHANES) 1999–2018.

Results

We found that 78.9% of individuals with SBP ≥130 mmHg benefited from the intensive SBP control. The high-benefit approach outperformed the high-risk approach [average treatment effect (95% CI), +9.36 (8.33–10.44) vs +1.65 (0.36–2.84) percentage point; difference between these two approaches, +7.71 (6.79–8.67) percentage points, P-value <0.001]. The results were consistent when we transported the results to the NHANES data.

Conclusions

The machine-learning-based high-benefit approach outperformed the high-risk approach with a larger treatment effect. These findings indicate that the high-benefit approach has the potential to maximize the effectiveness of treatment rather than the conventional high-risk approach, which needs to be validated in future research.

Introduction

Medicine has been prioritizing the treatment of patients with a high risk of adverse health outcomes such as cardiovascular diseases (CVD) and mortality—high-risk patients—with an implicit assumption that high-risk patients benefit most from the intervention.^1,2 This assumption has led to the development of numerous prediction models that predict the absolute risk of adverse health outcomes, such as the Framingham risk score.³ This approach has historically not been challenged, due to the inherent difficulty of estimating the actual benefit of treatment at the individual level—individualized treatment effect (ITE). However, in recent years, there has been substantial advance in the machine-learning-based approach to estimate how treatment effects vary based on observable characteristics of individuals. Estimating such heterogeneous treatment effects (HTEs) enables us to investigate whether and the extent to which the beneficial effect of intervention varies across individuals.^4–6 One such method, the causal forest, uses an ensemble of ‘trees’ or partitions optimized to detect HTEs. This approach allows us to estimate the ITE for a particular patient as a function of individual characteristics.^7–9 In theory, treating patients with a high estimated ITE (‘high-benefit approach’), instead of treating patients with a high risk of adverse health outcomes (‘high-risk approach’), has the potential to maximize the effectiveness of the treatment and improve population health outcomes. However, to our knowledge, no study to date has investigated whether the high-benefit approach leads to improved population health outcomes compared with a traditional high-risk approach using nationally representative data.

The SPRINT (Systolic Blood Pressure Intervention Trial) showed that intensive systolic blood pressure (SBP) treatment (goal SBP <120 mmHg) was, on average, associated with a lower risk of CVD events and all-cause mortality among people without diabetes,^10,11 leading to lowering the diagnostic threshold of SBP from 140 mmHg to 130 mmHg in the American College of Cardiology (ACC)/American Heart Association (AHA) hypertension guidelines.¹² This change has been controversial, given that approximately half of the American people were diagnosed as hypertensive using this threshold.¹³ Another randomized controlled trial (RCT) that focused on individuals with diabetes, ACCORD-BP (Action to Control Cardiovascular Risk in Diabetes Blood Pressure) trial, found no evidence that intensive SBP control was associated with a lower probability of CVD events.¹⁴ In clinical practice, individuals with high blood pressure are likely to receive antihypertensive treatments, despite the fact that not all patients with blood pressure above a diagnostic threshold benefit from lowering SBP. A few studies have estimated HTEs of intensive SBP treatment^15–19; however, the evidence is lacking as to whether the high-benefit approach improves the effectiveness of our blood pressure management. By providing intensive SBP treatment to those participants with a large estimated ITE, instead of participants with a high SBP, the high-benefit approach has the potential to improve the effectiveness and efficiency of the treatment, leading to a larger number of CVD events being prevented at the population level.

In this context, using data from two randomized controlled trials (SPRINT and ACCORD-BP) and one national cohort of American people [NHANES (the National Health and Nutrition Examination Survey)], we compared the population health outcomes when we selected individuals who received intensive SBP control using the high-risk approach versus the high-benefit approach.

Methods

Data sources and study participants

We used three databases: the SPRINT trial,¹⁰ the ACCORD-BP trial¹⁴ and the NHANES 1999–2018²⁰ (Figure 1). The anonymized SPRINT and ACCORD-BP data were obtained through the National Heart, Lung, and Blood Institute’s Biologic Specimen and Data Repository Information Coordinating Center. The NHANES data were available through the webpage of the National Center for Health Statistics.

To evaluate the effect of intensive SBP control among people with and without diabetes, we combined the datasets of the SPRINT (includes individuals without diabetes) and the ACCORD-BP trial (includes individuals with diabetes). The SPRINT trial was a randomized clinical trial comparing intensive versus standard SBP treatment among 9361 US adults without diabetes, conducted at 102 clinical sites between 2010 and 2013. The ACCORD-BP trial was also an RCT comparing intensive versus standard SBP treatment, which included 4733 adults with diabetes and glycated haemoglobin levels ≥7.5%, conducted at 77 clinical sites in the USA and Canada between 2003 and 2009. We selected 10 672 trial participants (6189 in the SPRINT and 4483 in the ACCORD-BP) who had an event or were followed for 3 years after the trial enrolment. Although the eligibility criteria for both trials included SBP 130 to 180 mm Hg at screening, one-third of participants showed SBP ≤132 mmHg at baseline of these trials (Supplementary Figure S1, available as Supplementary data at IJE online).^10,11,14 The detailed protocol and study design of the SPRINT trial and the ACCORD-BP trial have been previously reported.^10,14,21

Next, we used the NHANES data to transport the results from the SPRINT and the ACCORD-BP trials to the national population in the USA. The NHANES is a large-scale periodic survey of a representative sample of the civilian non-institutionalized US population, conducted by the National Center for Health Statistics (NCHS).²⁰ Structured household interview data and physical examination results at a mobile examination centre have been collected continuously and released in 2-year cycles since 1999. Across 60 776 participants who were enrolled in the NHANES between 1999 and 2018 and had SBP measurements, we selected 14 575 adults aged ≥40 years at high CVD risk [history of coronary heart disease, estimated glomerular filtration rate (eGFR) of 20–59 ml/min/1.73 m², 10-year Framingham risk score ≥15% or age ≥75 years).

Intervention

Both in the SPRINT and in the ACCORD-BP trials, the participants were randomly assigned to either the intensive SBP treatment group (goal SBP <120 mmHg) or the standard SBP treatment group (goal SBP <140 mmHg) at a 1:1 ratio.^10,14 Blood pressure was measured three times in a seated position after 5 min of quiet rest, using an automated measurement system (Model 907, Omron Healthcare, Kyoto, Japan) during each office visit in both trials. For the selection of the treatment groups, analyses adhered to the intention-to-treat approach.

Outcomes

We used the SPRINT composite primary outcome during a 3-year follow-up, which included myocardial infarction, acute coronary syndrome, stroke, acute decompensated heart failure and CVD death.²¹ Because the primary outcome in the ACCORD-BP was the composite of myocardial infarction, stroke and CVD death, we additionally included unstable angina and acute decompensated heart failure, to construct the SPRINT composite primary outcome in the ACCORD-BP subset as well.

Baseline covariates

The variables collected by self-report at enrolment in SPRINT, ACCORD-BP and NHANES include: age (years), sex (male, female), race/ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, others), education status (less than college, college or above), insurance status (insured or not), living arrangements (living alone or living with others), and smoking status (yes, no). Clinical and laboratory information [i.e. blood pressure, body mass index (BMI), total cholesterol, high-density lipoprotein (HDL) cholesterol, eGFR (<45, 45 to <60, 60 to <90 and ≥90 ml/min/1.73 m²] previous history of CVD, statin use and number of antihypertensives use] was also collected at enrolment in each study. For the purpose of this study, we labelled diabetes as 0 for the SPRINT participants and 1 for the ACCORD-BP participants according to the eligibility criteria in these trials. In NHANES, we classified diabetes as fasting glucose level ≥126 mg/dL, haemoglobin A1c ≥6.5%, use of antihyperglycaemic therapies or self-report of a physician’s diagnosis at enrolment.²² Missing data on these baseline covariates were imputed with a random forest approach within each dataset.²³

Statistical analysis

First, we confirmed that covariates were balanced between the intensive and standard treatment groups in our combined dataset of the SPRINT and the ACCORD-BP. Next, we applied the causal forest method (using grf package in R) for the dataset to develop the prediction model for ITE (on the risk difference scale) of intensive SBP treatment on CVD events during 3 years of follow-up. We constructed an ensemble of 2000 causal trees that identify subgroups with different treatment effects. When building each tree using all of the above-mentioned covariates, we employed the double-sample tree specification in which we had the following two steps. First, we randomly selected the 50% subsample without replacement from the original data to build each tree. Second, to reduce bias due to overfitting in tree predictions, we further split the fractional subsample into halves—an approach known as ‘honest’ splitting—and used the first half subsample to construct the tree and the second half subsample to make predictions. The model was built by 10-fold cross-fitting so that estimates for each fold were computed using trees that were fit without observations from that fold.²⁴ Each parameter of the causal forest model was tuned by 10-fold cross-validation. The calibration performance of our causal forest model was evaluated by computing the best linear fit of the regression of the observed treatment effect on the mean forest prediction and the out-of-bag predicted treatment effect, using the augmented inverse probability weighting and the ordinary least squares. The discrimination performance of our model was evaluated by calculating c-for-benefit.²⁵ The variable importance was calculated by a simple weighted sum of how many times each variable was split at each depth in the causal forest. More details on causal forest analysis can be found elsewhere.^6–9

Second, we applied the causal forest model built in the combined data of SPRINT and ACCORD-BP to estimate the ITE of intensive SBP treatment on CVD events for each participant. First, we examined the relationship between baseline SBP (‘risk’) and estimated ITE (‘benefit’). Next, we categorized the trial participants into two groups according to the current SBP threshold by the ACC/AHA hypertension guideline¹²: high-risk group (SBP ≥130 mmHg) vs low-risk group (SBP <130 mmHg). Based on the estimated ITE using the causal forest model that was fit with other folds, we also categorized participants into two groups as hypothetical treatment strategies: high-benefit group (estimated ITE >0) vs low-benefit group (estimated ITE ≤0). We then compared the high-benefit approach and the high-risk approach using: (i) the number of individuals treated; (ii) the estimated average treatment effect; (iii) the number needed to treat (NNT); and (iv) the estimated number of CVD events prevented. We also compared the performance of the high-benefit approach with the high-risk approaches, using 10-year CVD risk score based on the Framingham risk score (≥20%) and the 2013 ACC/AHA pooled cohort equations (≥10%)²⁶ instead of SBP.

Last, to obtain the treatment effect of intensive SBP control and compare the performance between the high-benefit approach and the high-risk approach among NHANES participants, we applied the transportability formula by calculating the inverse-odds-weight: inverse of odds of being in the trial sample instead of the target population, to emulate the target population (NHANES) from the trial sample (SPRINT/ACCORD-BP).^27–29 Detailed analytical steps and transportability formula are shown in Figure 2. All statistical analyses were performed with R version 4.0.2. The 95% confidence intervals (CIs) were calculated using 1000 bootstrapped samples. This study was exempted by the institutional review board at the University of California, Los Angeles (IRB #20–002223).

Sensitivity analyses

We conducted the following three sensitivity analyses. First, given the possible selection bias due to loss to follow-up, we also conducted a sensitivity analysis calculating inverse-probability censoring weights from baseline covariates and applying the weights in our causal forest model.³⁰ Second, to incorporate the uncertainty around estimated ITEs, we built a causal forest model for each bootstrap sample to calculate and compare the average treatment effects and 95% CI of the high-benefit approach and high-risk approach. Third, to transport the trial results to the US general population, we applied the NHANES survey weight in our transportability formula using the two-stage weighting approach.³¹ Last, we compared the performance of the high-benefit approach with the high-risk approach using a threshold of 20% for the 10-year CVD risk score based on the 2013 ACC/AHA pooled cohort equation (instead of a 10% threshold used in the main analysis).

Results

Participant characteristics

Across 10 672 trial participants in the SPRINT and ACCORD-BP, the mean (standard deviation) age was 65.5 (8.4) years, and 40.8% were female. Baseline characteristics were balanced between the intensive and standard treatment groups in our pooled data (Table 1).

Causal forest model to predict ITE of intensive SBP control from trial results

During 3 years of follow-up in the SPRINT and ACCORD trials, 905 (8.4%) patients experienced composite CVD outcomes [intensive SBP treatment, 410/5334 (7.7%); standard SBP treatment, 495/5338 (9.3%)]. The prediction based on the causal forest model was well calibrated and suggested the presence of heterogeneity of the beneficial effects of intensive SBP control to reduce CVD events (Supplementary Figures S2 and S3, available as Supplementary data at IJE online); in the best linear fit model for the observed treatment effect, the coefficient of the mean forest prediction was 1.02 (P-value <0.001, i.e. significantly greater than 0 and close to 1, indicating that the mean forest prediction was correct) and the coefficient of the out-of-bag predicted treatment effect was 0.54 (P-value = 0.045, i.e. significantly greater than 0, indicating that the forest captured heterogeneity). The forest prediction also showed high discrimination performance, as demonstrated by a high C-for-benefit of 0.90 (95% CI, 0.84 to 0.97). The variable importance calculation showed that age, BMI, HDL and total cholesterol levels and baseline SBP were frequently split when building the causal forest model (Supplementary Figure S4, available as Supplementary data at IJE online).

High-risk approach vs high-benefit approach

A total of 6270 participants (78.9% in the high-risk group and 73.2% in the high-benefit group) showed both SBP ≥130 mmHg and ITE >0 (Figure 3), suggesting that a large number of individuals who met the clinical criteria of hypertension benefited from intensive SBP control. Individuals with ITE >0 (the high-benefit group; n = 8563) were more likely to be older, female, and to have lower eGFR than those with ITE ≤ 0 (the low-benefit group; n = 2109; Table 2). They also had higher HDL cholesterol levels, lower percentage of clinical CVD, statin use and history of diabetes compared with the low-benefit group. We found a similar pattern when we stratified the participants into high-risk and low-risk groups (Table 3).

When we compared the performance between the high-risk approach and the high-benefit approach (Table 4A), we found a larger average treatment effect in the high-benefit approach than in the high-risk approach [the high-benefit approach, +9.36% age point (95% CI, 8.33 to +10.44); the high-risk approach (based on SBP ≥130 mmHg), +1.65% age point (95% CI, +0.36 to +2.84); the high-risk approach (based on 10-year Framingham CVD risk score ≥20%), +2.23% age point (95% CI, +0.68 to +3.70); the high-risk approach (based on ACC/AHA pooled cohort equation ≥10%), +1.91% age point (95% CI, +0.64 to +3.06)]. The NNT to prevent one case of the composite CVD outcomes was 11 in the high-benefit approach, 61 in the high-risk approach (based on SBP), 45 in the high-risk approach (based on Framingham score), and 52 in the high-risk approach (based on ACC/AHA pooled cohort equation), respectively.

The distribution of baseline covariates among the entire trial samples (SPRINT/ACCORD-BP) and the target population (NHANES) is shown in Supplementary Table S1 (available as Supplementary data at IJE online). These covariates were well balanced between these populations when we applied the inverse-odds weighting in the transportability formula (Supplementary Figure S5, available as Supplementary data at IJE online). Consistent with the main analysis, the NNT was 11 in the high-benefit approach and 57–64 in the high-risk approach (Table 4B).

Sensitivity analyses

We found similar results (i) when we considered right-censoring using inverse-probability censoring weights (Supplementary Table S2, available as Supplementary data at IJE online), (ii) when we built a causal forest model within each of 1000 bootstrap samples (Supplementary Table S3, available as Supplementary data at IJE online) and (iii) when we applied NHANES survey weights in our transportability analysis (Supplementary Table S4, available as Supplementary data at IJE online). The average treatment effect in the high-benefit approach was 7.44% age points larger than that in the high-risk approach for the sensitivity analysis using a threshold of 20% for the 10-year CVD risk score based on the ACC/AHA pooled cohort equation (Supplementary Table S5, available as Supplementary data at IJE online).

Discussion

Using the machine-learning causal forest model applied to two large RCTs’ data and one nationally representative sample of US adults, we found that the benefit of intensive SBP control to reduce CVD outcomes varied across individuals. We found no relationship between the risk (SBP at the baseline) and the benefit (estimated ITE). The average treatment effect of our hypothetical high-benefit approach was higher than that of the high-risk approach, leading to the 4–5 times smaller NNT to prevent one case of the composite CVD outcomes in the high-benefit approach compared with the high-risk approach. We also found consistent results when we transported the trial results to the NHANES cohort. These findings highlight the possibility that selecting individuals receiving the treatment using the high-benefit approach has the potential to substantially increase the effectiveness of the treatment and improve population health outcomes.

Given the wide heterogeneity in the impact of medical interventions on health outcomes, the identification of individuals who are expected to receive material benefit from the treatment is critically important in clinical decision making.^16,32–34 Historically, high-risk patients have been perceived to have the largest benefit from treatment,^1,2 and clinicians often decided who to treat based on their blood pressure levels. However, we found that a substantial number of trial participants with SBP <130 mmHg at baseline evaluation (despite showing SBP ≥130 mmHg at the initial evaluation to determine the eligibility of the trial participation) benefited from controlling SBP to under 120 mmHg. In addition, we could lower NNT by around one-fifth by focusing on individuals with the highest estimated benefit from the intervention using the high-benefit approach compared with the high-risk approach (based on SBP or CVD risk scores). These findings indicate that treating only individuals with high SBP or high CVD risk may not be the most efficient way to prevent future cardiovascular events through blood pressure management.

Our approach using causal forest algorithm in the trial data would maximize the utility of the RCT findings towards a tailored approach in clinical medicine, by allowing us to identify individuals who are expected to receive benefits from the treatment without the cost of throwing away patients’ information.^7,9 In clinical medicine, we ‘dichotomize’ the population into those individuals with and without hypertension—even though the risk of adverse health outcomes increases linearly without abrupt changes as blood pressure increases³⁵—and treat only those individuals with hypertension. However, our findings indicate that such an approach may be inefficient, given that not all individuals with a high SBP benefit from the treatment, whereas a substantial number of individuals who are labelled as normal (non-hypertensive) would actually benefit from the treatment had they been treated. Even in precision medicine, in which health care is ‘individually tailored on the basis of a person’s genes, lifestyle and environment’,³⁶ health care often focuses on individuals with a diagnosis of disease and leaves out those who do not meet the diagnostic criteria of the disease. Our findings indicate that the high-benefit approach has the potential to improve population health outcomes without additional resources by treating individuals with a high estimated benefit of the treatment, regardless of whether they meet the diagnostic criteria of hypertension.

Our findings extend the long-standing debate on whether the high-risk approach or the population-based approach is more effective in the prevention of CVD. The high-risk approach has been more dominant in clinical medicine, leading to the development of a large number of prediction models for estimating the risk of CVD events.^3,37,38 However, there have been several limitations to this approach. For example, the high-risk approach addresses only a small number of individuals with substantially high health risks, and it does not address a large number of individuals with a small elevated risk who contribute a large share of the burden of CVD. It may falsely reassure some people by labelling them as ‘low risk’, and disincentivize them from receiving or adhering to beneficial risk-lowering interventions.³⁹ The population-based approach assumes that decreasing risk for everyone is more effective in reducing the burden of disease than focusing on high-risk individuals, because the larger number of people with a small elevation in risk usually contributes more to the burden of disease than a small number of people exposed to high risk.⁴⁰ This approach also has limitations, including that small benefits to individuals can be outweighed by small risks associated with the intervention, and this approach may lead to poor motivation among individuals and physicians.⁴¹ The high-benefit approach proposed in this study has the potential to overcome these limitations by directly estimating ITE and treating individuals with a large estimated benefit from treatment. Given the previous cost-effective analysis of the SPRINT,⁴² these benefits are expected to be weighed against the increased implementation costs due to additional office visits, laboratory tests and medications. It is also important to note that our findings should not slow down the treatment of individuals with a high risk of CVD but low benefit from intensive SBP control. They rather suggest the need for establishing an alternative approach to intensive SBP control to effectively prevent CVD for such high-risk and low-benefit individuals.

Limitations

Our study has limitations. First, we could not rule out the possibility that unmeasured individuals’ characteristics might modify the treatment effect in the SPRINT/ACCORD trials because the causal forest model only enables us to detect HTEs through measured variables. Second, because some of the baseline characteristics in our data were self-reported, we might have a risk of measurement error in this information. Third, given the difference in the inclusion and exclusion criteria between the SPRINT and the ACCORD trials, we might have detected HTEs partially due to the study design rather than social or physiological mechanisms. Fourth, because both SPRINT and ACCORD-BP focused on middle-aged and older adults, we cannot necessarily generalize our findings to young adults aged <40 years. Fifth, although we estimated ITEs for each fold based on the algorithm fitted without observations from that fold, future research is needed to fully take account of the uncertainty of the estimated ITEs and to validate our findings using other databases.

Conclusions

Using data from two large clinical trials investigating the benefit of intensive SBP control and one nationally representative data of US adults, we found that individuals with the highest risk of adverse health outcomes were not always the ones who benefited most from the treatment. Furthermore, our hypothetical high-benefit approach outperformed the high-risk approach, with 4–5 times lower NNT. Our findings indicate that the machine-learning-based high-benefit approach could identify individuals who would benefit most from the intervention, leading to a lower number of cardiovascular outcomes at the population level.

Ethics approval

The present study was approved by the institutional review board at Kyoto University (R3069), and the study was conducted following the Declaration of Helsinki.

Data availability

Available through National Heart, Lung, and Blood Institute BioLINCC data repository at [https://biolincc.nhlbi.nih.gov/studies/sprint/], [https://biolincc.nhlbi.nih.gov/studies/accord/], and [https://wwwn.cdc.gov/nchs/nhanes/Default.aspx].

Supplementary data

Supplementary data are available at IJE online.

Author contributions

All authors had full access to all of the data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design: K.I, S.A., Y.T. Acquisition, analysis or interpretation of data: K.I., S.A., Y.T. Drafting of the manuscript: K.I. S.A., Y.T. Critical revision of the manuscript for important intellectual content: K.I., S.A., Y.T. Statistical analysis: K.I., S.A., Y.T.

Funding

K.I. was supported by Japan Agency for Medical Research and Development (AMED; JP22rea522107), the NIH/NIDDK grant F99 DK126119, the Japan Society for the Promotion of Science (21K20900 and 22K17392), the Japanese Endocrine Society, Meiji Yasuda Life Foundation of Health and Welfare, and the Program for the Development of Next-generation Leading Scientists with Global Insight (L-INSIGHT) sponsored by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan. Y.T. was supported by the National Institutes of Health (NIH)/National Institute on Minority Health and Health Disparities Grant R01MD013913 and NIH/National Institute of Aging Grant R01AG068633. S.A. was supported by the Golub Capital Social Impact Lab, Schmidt Futures, the Sloan Foundation, Office of Naval Research Grant N00014-17–1-2131, the Mercatus Center and Microsoft Research. Study sponsors were not involved in study design, data interpretation, writing, or the decision to submit the article for publication. This article does not necessarily represent the views and policies of the NIH. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgements

We thank the editors and several anonymous reviewers for their helpful comments. We also thank Stefan Wager (Stanford University) and Scott Fleming (Stanford University) for their help with discussions on methods for evaluating heterogeneity in the treatment effect.

Conflict of interest

None declared.

References