When you've completed screening at Title & abstract or Full text review stage, you can export inter-rater reliability (IRR) data.

To do this, go to the Export page for your review; you'll see the section where you can choose to export a CSV file containing IRR data for Title & abstract or Full text review.


When you've selected your stage and clicked 'Export CSV', you'll shortly see an export file appear which you can download:

The export file includes not only the reviewer comparison data; you'll also get the calculations you need, including Cohen's Kappa coefficient:

In inter-rater reliability calculations from the Title and abstract screening stage, 'maybe' votes are counted as 'yes' votes.

Please note that only the final set of votes cast at each round are included for inter-rater reliability calculations. If votes have been undone and re-done at either stage, only the votes cast during the re-voting process will be included.