On this page:

 

If you’re completing a review that looks only at the effects of health interventions through Covidence and want to only consider research papers reporting on Randomised Controlled Trials (RCTs), we have two automation features to support you, so that you can focus your effort on references more likely to be included in your review:

  • Tagging references reporting on RCTs,

  • Remove references reporting on non-RCTs before screening.

The Cochrane RCT Classifier

We’ve integrated with the leading RCT classifier algorithm, developed by EPPI-Centre, to provide you with a prediction on whether studies in your review potentially report on an RCT. The Classifier has been endorsed by Cochrane and is also being used in the Cochrane Screen4Me workflows, which also includes Cochrane Crowd.

The classifier has been shown in testing to successfully identify over 99.5% of health-related references that potentially report on RCTs, only incorrectly classifying ~0.5% of RCTs as not being an RCT (Thomas et al, 2021).

Requirements

Given the RCT classifier has been trained, calibrated and validated using health-related research papers, we only allow the features to be used on Cochrane reviews or reviews in the “medical and health science” research area.

To ensure the classifier has enough context about each reference to make informed predictions, we'll only input references with titles of 14 characters or more and abstracts of 400 characters or more. This mirrors the criteria used for validating the classifier's performance.

The features

Tagging references reporting on RCTs

When the feature is enabled, all eligible references imported to the review will be run through the Cochrane RCT classifier and tagged with either “Possible RCT” or “Not RCT”, depending on the prediction made by the classifier:

The studies list can also be filtered by the RCT tags, allowing you to see a list of only “Possible RCTs” or “Not RCTs”:

Remove references reporting on non-RCTs before screening

When the feature is enabled, all eligible references imported to the review that don’t have any manual screening votes applied will be run through the Cochrane RCT classifier to identify and remove the references not reporting on RCTs.

The references automatically removed can be reviewed via the “Auto-marked as ineligible” removed before screening tab, under the import references section:

 

By clicking here, you can also review the list of auto-marked ineligible references and, if you believe any should be included, simply click “Move to screening” to return them to the screening list. This gives you full control over what’s reviewed.

Enabling the features

When creating a review, you will be shown two RCT-related settings under the “Automation options” section when the review’s research area is set to “Medical and health sciences” or it is a Cochrane review:

Disabling the features

You can disable both features at any point via the automation options section on review settings (shown above).

Tag references reporting on RCTs

Disabling the feature to tag references reporting on RCTs will retain tags already applied to references, although further references won’t be automatically tagged when imported to the review.

Remove references reporting on non-RCTs before screening

Disabling the feature will cause all references automatically removed by the classifier to be moved back to the title and abstract screening step for manual screening. You will be able to view the actions performed on each reference in their history:

 

Do you have feedback or suggestions on how we can improve this feature?

Let us know by submitting your feedback here.