Abstract
Objective Reviews on the relationship of low-energy sweeteners (LES) with body weight (BW) have reached widely differing conclusions. To assess possible citation bias, citation analysis was used to quantify the relevant characteristics of cited articles, and explore citation patterns in relation to review conclusions.
Design A systematic search identified reviews published from January 2010 to March 2020. Different characteristics (for example, type of review or research, journal impact factor, conclusions) were extracted from the reviews and cited articles. Logistic regression was used to estimate likelihood of articles with particular characteristics being cited in reviews. A qualitative network analysis linked reviews sub-grouped by conclusions with the types of articles they cited.
Main outcome measures (OR; 95% CI) for likelihood that articles with particular characteristics were cited as evidence in reviews.
Results From 33 reviews identified, 183 different articles were cited (including other reviews). Narrative reviews were 62% less likely to be cited than systematic reviews with meta-analysis (OR 0.38; 0.16 to 0.86; p=0.03). Likelihood of being cited was higher for evidence on children than adults (OR 2.27; 1.59 to 3.25; p<0.0001), and with increased journal impact factor (OR 1.15; 1.00 to 1.31; p=0.04). No other factors were statistically significant in the main analysis, and few factors were significant in subgroup analyses. Network analysis showed that reviews concluding a beneficial relationship of LES with BW cited mainly randomised controlled trials, whereas reviews concluding an adverse relationship cited mainly observational studies.
Conclusions Overall reference to the available evidence across reviews appears largely arbitrary, making citation bias likely. Differences in the conclusions of individual reviews map onto different types of evidence cited. Overall, inconsistent and selective use of the available evidence may account for the diversity of conclusions in reviews on LES and BW.
Trial registration number Prior to data analysis, the protocol was registered with the Open Science Framework (https://osf.io/9ghws).
Summary
The main outcome of the present citation analysis was that published reviews concluding a beneficial relationship between low/no calorie sweeteners and body weight cite mainly randomised controlled trials (RCTs), whereas reviews concluding an adverse relationship cite mainly observational studies. It is widely acknowledged that a body of evidence based on RCTs is rated as being of high quality, while evidence from non-randomized trials or observational studies is rated as being of low quality.
This citation network analysis included 33 reviews and its aim was to assess the citation patterns and possible citation bias in reviews on the relationship of low/no calorie sweeteners with body weight, in an effort to understand the basis for widely differing conclusions in the published literature of systematic and narrative reviews. Systematic reviews with meta-analysis were significantly more likely to be cited compared to narrative reviews. Also, articles on children were 127% more likely to be cited than articles on adults. Higher journal impact factor also favoured citation.
About half of the reviews concluded that there was an effect or association of low/no calorie sweeteners and body weight, either beneficial, neutral or adverse, whereas the other half concluded that the evidence was insufficient to draw a conclusion. In line with the reported finding that most of the cited RCTs reported a beneficial effect of low/no calorie sweeteners on body weight whereas most cited observational studies reported a neutral/ adverse association, the analysis found that individual reviews that cited mainly RCTs concluded to a beneficial effect/ association of low/no calorie sweeteners with body weight, whereas reviews citing mostly observational studies reported an adverse effect/ association. Reviews concluding that there was insufficient evidence to draw a conclusion cited a mix of both RCTs and observational studies.
Overall, the authors conclude that, taken together, these results show a very diverse and inconsistent pattern of citations on this scientific topic, suggesting that the citation of evidence across reviews is somewhat arbitrary, which may contribute to the diversity in review conclusions.
