Responsible AI for Monitoring & Evaluation

Human accountability

AI may support a task, but responsibility for analysis, interpretation, recommendations, and final outputs remains with qualified people.

Evidence connection

Every important claim should be traceable to source material, not merely to a convincing AI-generated paragraph.

Proportional safeguards

The more a task affects meaning, judgement, people, resources, or decisions, the stronger the review process should be.

Data care

Personal, confidential, political, commercial, or community-sensitive information requires explicit permission and appropriate protections.

Context awareness

AI outputs should be checked for missing cultural, political, historical, organisational, and local context.

Transparent records

Teams should be able to explain what AI was used for, what was checked, what changed, and who approved the final work.

Core review questions

• Is the output supported by the source material?
• What has been omitted, simplified, or overstated?
• Are minority, dissenting, or unexpected views still visible?
• Are causal claims justified by the evidence?
• Could the evaluator defend the final conclusion without the AI output?

Verification methods

• Compare AI outputs with original sources.
• Spot-check summaries, codes, and grouped responses.
• Map important claims to supporting evidence.
• Use peer challenge for higher-risk interpretation.
• Validate sensitive findings with appropriate stakeholders.

Sensitive information

Check consent, confidentiality, security, and organisational rules before using AI with transcripts, personal data, vulnerable groups, or protected information.

Equity review

Look for missing perspectives, dominant assumptions, biased language, and summaries that erase the experience of smaller or less powerful groups.

Stakeholder meaning

AI may support preparation, but it should not replace participatory interpretation, validation, or locally grounded sense-making.

Project start-up questions

• Which AI uses are allowed, conditional, or prohibited?
• What kinds of data must never be entered into AI tools?
• Who approves higher-risk uses?
• How will AI use be disclosed to clients, funders, or participants?

Minimum governance roles

• Practitioner: records and checks AI-supported work.
• Project lead: approves risk level and review process.
• Reviewer: challenges analysis, evidence links, and recommendations.
• Data or ethics lead: advises on sensitive information.

Escalation triggers

• The task involves participant-level or identifiable data.
• The output may influence findings or recommendations.
• The context is politically, socially, or commercially sensitive.
• The team cannot explain how the output was checked.

1. AI use register

Keep a project log of tool used, task, data type, risk level, review method, disclosure decision, and responsible person.

2. Evidence-to-claim map

Link each major finding, conclusion, or recommendation to the evidence that supports it and note any AI involvement.

3. Sensitive data decision tool

Check whether data is public, internal, confidential, personal, politically sensitive, or protected by consent conditions.

4. Professional judgement check

Confirm that the practitioner can explain the finding, uncertainty, alternative explanations, and limits without relying on AI wording.

5. Equity and voice review

Check whether smaller groups, dissenting views, local categories, and marginalised perspectives have been preserved.

6. Disclosure builder

Prepare clear language explaining what AI was used for, what it was not used for, and how human review was maintained.

When to use it

• At the start of any AI-supported project.
• Whenever AI is used to draft, summarise, classify, code, analyse, translate, or prepare evaluation material.
• Before delivery, to confirm that all AI-supported outputs have been reviewed.

Minimum fields

• Date and person responsible.
• AI tool used and task supported.
• Data type and sensitivity level.
• Risk level and review method.
• Disclosure decision and approval notes.

How to build it

• List every finding, conclusion, and recommendation that may influence decisions.
• Add the evidence source, page, transcript reference, dataset, or observation note.
• Mark whether AI contributed to summarising, grouping, drafting, or wording the claim.
• Record caveats, contradictory evidence, and confidence level.

Review questions

• Can the claim be defended without relying on AI wording?
• Has the source evidence been checked by a person?
• Are minority or dissenting views visible?
• Is uncertainty preserved rather than smoothed away?

Decision steps

• Classify the material before using AI.
• Remove identifiers where possible.
• Check consent and contractual restrictions.
• Use approved tools only for sensitive material.

Stop triggers

• You cannot explain where the data will be processed.
• You do not know whether the tool trains on user inputs.
• The data includes children, vulnerable groups, political exposure, or protected characteristics.
• The client or ethics process prohibits AI processing.

Explainability

• Can the evaluator explain the finding in their own words?
• Can they identify the evidence that supports it?
• Can they explain why alternatives were not selected?

Uncertainty

• Are caveats visible?
• Are confidence levels proportionate?
• Has AI language made weak evidence sound stronger than it is?

Accountability

• Who approved the final wording?
• Who checked the evidence?
• Would the team defend the conclusion without showing the AI output?

What to check

• Which voices are most visible in the AI-assisted output?
• Which groups, locations, languages, or roles are less visible?
• Were dissenting views merged into the majority view?
• Did AI replace local terms with generic development language?

Corrective actions

• Return to source material for underrepresented groups.
• Separate majority, minority, and divergent findings.
• Preserve local words and concepts where appropriate.
• Validate sensitive interpretations with suitable stakeholders.

Low-risk wording

AI was used to support editing, formatting, or readability. The evaluation team reviewed the final text and retained responsibility for meaning and accuracy.

Analysis-support wording

AI was used to assist with organising material and identifying possible patterns. The evaluation team checked outputs against source evidence and made all analytical judgements.

Sensitive-use wording

Where AI-supported processing involved sensitive material, this was handled only through approved systems and review procedures. Human reviewers verified outputs before use.