Some researchers use third-party AI agents, such as browser-based assistants, to screen studies, extract data or record decisions in Covidence on their behalf. Covidence does not currently block these tools, but we don't recommend using them to make review decisions. Third-party agents are not validated, tested or supported by Covidence, and their use can create challenges for transparency, accountability and the reliability of your review.
Why doesn't Covidence recommend using third-party AI agents?
Your audit trail may not reflect who made the decisions. Covidence records actions against the logged-in reviewer. If an AI agent makes decisions on your behalf, those decisions appear as yours, without indicating that AI was involved. This matters because workflows such as dual screening and conflict resolution depend on clear accountability for each decision. Your co-reviewers may also be unaware that an agent has acted on your behalf.
You need to demonstrate that the agent's decisions are reliable. Journals, peer reviewers and readers may expect you to explain how AI was used, how its performance was evaluated and how its outputs were checked. Third-party agents operating through the Covidence interface can be difficult to evaluate, reproduce and audit. This makes it harder to demonstrate that your review methods were rigorous and your findings trustworthy.
We can't support or troubleshoot third-party agents. Third-party agents are not tested or supported by Covidence. Changes to our interface may affect how they operate, potentially without warning. If an agent makes an error or disrupts your review, our support team may be unable to identify, investigate or resolve the problem.
What should I do if I choose to use an AI agent?
If you decide to use a third-party agent, we recommend that you:
Agree on its use with your review team before starting, including which tasks it will perform and how decisions will be checked.
Evaluate its performance against your review's requirements before relying on its outputs.
Document and report its use, including the tool, model and version, tasks performed, dates of use, how performance was evaluated and how humans verified its outputs.
Remember that actions performed by an agent will be recorded under your Covidence account. You'll need to maintain your own record of which decisions involve AI.
Does this guidance apply to all AI tools?
The principles of evaluation, transparency and human oversight apply to any AI used in evidence synthesis, including general-purpose AI tools, models developed by your team and AI features within Covidence.
However, the specific limitations described above relate to third-party agents taking actions within your Covidence account.
This guidance does not apply to assistive technologies, such as screen readers, or tools used separately that do not take actions within Covidence.
Where can I learn more about responsible AI?
For practical guidance on using AI responsibly in evidence synthesis, explore:
Our AI blog series: How we think about responsible AI, evaluate emerging capabilities and decide when automation is appropriate.
How to plan and report AI use: Guidance from our Knowledge Base on documenting AI-assisted work in your review.
Covidence model cards: Examples of how we evaluate AI capabilities, communicate their performance and limitations, and support informed decisions about their use.