A nonprofit organisation does not need an “AI project” to encounter AI governance: a caseworker uses automated translation, a communications colleague asks a writing assistant to shorten testimony for a report, a volunteer restructures a difficult letter with a chatbot, a fundraising team experiments with generated copy, an office platform adds meeting summaries or document search.
For small NGOs in particular, the attraction is obvious: translation, case administration, reporting and communications take time and specialist resources that may simply not be available when they are needed. The difficulty is that NGOs often work with information disclosed in circumstances of dependence, urgency and vulnerability.
This article is particularly concerned with frontline and rights-based NGOs that work directly with people in vulnerable or marginalised situations and routinely handle sensitive personal information, testimony or case material.
An organisation supporting asylum seekers may hold accounts of political persecution, torture, sexual violence, health conditions, family separation, religious identity, sexuality, irregular migration or detention. An LGBTQI+ organisation may receive accounts of discrimination, outing, family rejection or hate crimes. An organisation supporting victims of sexual and gender-based violence may hold testimony about abuse, intimate relationships, injuries or ongoing threats. Organisations working with homelessness, disability, mental health, trafficking, harm reduction or child protection can find themselves handling equally sensitive material, although the risks are different in each case.
The person telling the story usually disclosed it for a specific reason: to obtain help, make a claim, document harm or access a service. Putting that material into an AI-enabled service raises several questions. Can the organisation, legally speaking, send this information to the service at all? What happens to it after submission? Can it be reused for model or product development? And if it can, what does it mean for an organisation serving marginalised groups to contribute highly selective accounts of those groups to systems it cannot inspect?
NGOs face an unusual AI governance problem because the information they handle can be exceptionally sensitive, while the tools they use are usually general-purpose systems designed for much broader kinds of content. This creates risks in both directions: in what happens to personal stories once they enter the system, and in what happens when the system itself cannot reliably process those stories.
When a personal story enters an AI system
Translation makes the problem easy to see. When a migration organisation receives testimony in Dari, Arabic, Persian, Tigrinya or another language, someone needs to understand it quickly, and automated translation can make information accessible when a human translator is unavailable or too slow for the immediate task. This tool could be used to translate a message from someone afraid of being outed, an account of sexual violence, a medical document or a request for emergency housing. The task of translation may be mundane, but the source material is not.
Under European data-protection law, several types of information commonly handled in these settings receive specific protection, including data concerning health, sexual orientation, sex life, political opinions, religious beliefs and racial or ethnic origin.1
Removing direct identifiers does not necessarily make the information anonymous. Under the GDPR, pseudonymised data remains personal data where it can still be attributed to a person using additional information.2 A combination of location, date, family relationship, profession, medical history or unusual circumstances can also contribute to making someone identifiable.
So before asking whether an AI translation is accurate, there is a more basic question: has the organisation actually decided that this particular service, under these particular terms, can receive this kind of information? That decision should not depend on an employee assuming that a tool is safe just because it is already available on their phone or computer.
Holding someone’s information does not give an organisation an unlimited mandate over it. People generally approach NGOs because they need something specific. An asylum seeker may describe political violence because it is necessary to establish a claim. Someone contacting an LGBTQI+ association may disclose their sexuality or gender identity because they need housing, legal advice or psychological support. A survivor may describe sexual violence because an organisation needs to assess the help they require. These stories are usually disclosed within a relationship of trust and for a particular purpose.
The European Data Protection Board has said that, when assessing a person’s reasonable expectations as part of a legitimate-interest analysis, relevant factors can include the relationship with the person, the context in which the information was collected, its source and potential further uses of the model.3 For an NGO, this deserves attention before getting to the more technical questions. The organisation may have a legitimate reason to hold the information and may have a lawful basis for processing it. But sending the same content to another service, which may operate under a different set of purposes and actors, still requires an analysis of the situation and of the tool being considered.
Once an organisation considers using an AI-enabled service with sensitive material, several questions tend to get grouped together.
First, can the information be submitted at all? That may involve data protection, confidentiality, safeguarding, professional rules or contractual obligations.
Then there is the question of what the provider does with the material. Is the input retained? For how long? Can it be deleted? Who can access it? Does the answer change between a free account, a paid professional product and an enterprise service?
Approving a product name in general may still leave important questions unanswered. The organisation needs to check the conditions that apply to the account and version staff are actually using, including differences in retention, administrative controls and possible use of interactions for model or product improvement.
There is also the question of secondary use: can the interaction be used to train or improve models, evaluate the service, develop safety systems or improve the product in some other way? The answer depends on the provider, product and account.
If interactions can be reused for model development, another question follows: what kind of material might the organisation be contributing? Organisations helping marginalised groups usually encounter those groups through highly selective circumstances. A migration NGO often meets people because they are facing persecution, detention, administrative exclusion, homelessness or violence. An LGBTQI+ support organisation may receive a disproportionate number of stories involving discrimination, family rejection, hate speech or housing insecurity. Organisations working with homelessness, addiction, trafficking or mental-health crises also see a concentration of difficult situations because dealing with those situations is their work.
Those accounts are not representative portraits of the whole populations concerned. Research on bias in language models has identified the composition and representativeness of training data as one possible source of harmful bias, alongside many other choices made during model development and deployment.4 This does not establish that an NGO translating personal stories will automatically and systematically make a model more biased. The causal chain is much more complicated than that.
The problem here is the lack of visibility. If these interactions can enter model-development processes, the NGO may have no way of knowing how they are selected, weighted, filtered or later used. It may therefore be contributing a very particular representation of the population it works with without knowing what happens to that representation afterwards. For organisations whose mission includes defending groups already affected by stereotyping or discrimination, that uncertainty belongs in the decision about whether the material should enter the service at all.
When the system cannot handle the story
There is a different set of problems in the opposite direction. These accounts may contain descriptions of violence, persecution, sexual abuse, self-harm, hate speech or distress that activate a system’s safety guardrails. A translation or summarisation tool may work perfectly on ordinary administrative text and then refuse, truncate or soften a legitimate task because of the nature of the content it is being asked to process. From an NGO’s perspective, average performance is not enough. The tool also has to work on the difficult, unusual material that is too common in these organisations.
AI providers put safety controls in place to prevent models from producing harmful material or helping users carry out dangerous actions, and there are valid reasons for doing so. NGOs may nevertheless need AI tools to process descriptions of exactly the subjects those controls are designed to detect. I have seen AI-enabled tools refuse to process asylum-related testimony during an otherwise ordinary task because the account contained material that triggered its safety controls.
The same problem can appear elsewhere: a sexual-violence service may need to translate an explicit description of an assault; an LGBTQI+ organisation may need to process threatening messages containing homophobic or transphobic slurs; a mental-health or youth organisation may need to work with a document containing references to self-harm.
Recent research describes this broader problem as context-driven over-refusal: a model refuses a benign task because safety-sensitive material in the supplied context triggers its guardrails. A 2025 ACL study found that this can occur in ordinary tasks such as translation and summarisation, with rates varying by task, model family and system setup.5
Safety-sensitive material may be part of an NGO’s everyday work and a system that performs very well on generic translation tests can still become unreliable on the documents the organisation most needs it to handle.
When a tool refuses a legitimate task, people tend to try to make it work. Someone might remove part of the story. They replace explicit words with euphemisms. They divide a document into smaller sections. They repeatedly rephrase the instruction. Or they switch to another tool that appears more willing to process the text.
The reason staff and volunteers do this is understandable. They still have a translation, report or case file to finish, often with little time and few resources. But each workaround can create another issue. If someone changes the wording of testimony to get it through a safety filter, are they still translating the same testimony? If they remove the explicit description of violence, has something important disappeared? If they split the story into fragments, has its context changed? And if they give up and paste it into another AI service, does that service have the same data protections as the one the organisation originally approved?
What begins as a product limitation can turn into an informal organisational practice that nobody has really decided on. If staff repeatedly need to work around the safety mechanisms of an approved tool to perform legitimate work, the organisation should look again at the tool selected for the workflow.
Staff also need a fallback that has been decided in advance. Can the task move to a human translator or specialist? Should the original text always be preserved alongside an AI-assisted version? Is switching to another AI service allowed? And someone should have the authority to decide that repeated failure makes the tool unsuitable for that workflow. Without an agreed fallback, each employee or volunteer ends up inventing a solution while handling sensitive material.
What an NGO actually needs to decide
A small NGO does not need an AI department before staff can use translation, drafting or summarisation tools. But it does need enough shared rules that a caseworker, volunteer or communications employee is not forced to invent the organisation’s position each time they open a chatbot.
A useful starting point is to understand what is already happening: which tools staff are using, for which tasks, what kinds of information are entering them, where the difficult cases occur and who currently makes the decision when something goes wrong. Without that picture, an AI policy can easily describe an organisation that does not exist in practice.
I use Governance through Coherence to describe the next step: looking at whether the organisation’s choices about the tool, the data, the workflow, human review and its responsibilities towards the people concerned actually fit together.
I would start with seven questions:
- Which service and account are we using?
- What information are staff allowed to enter?
- What happens to that information after submission, including retention and possible secondary use?
- Who decides when a more sensitive use is acceptable?
- What kind of human review does the output need?
- What should staff do when safety controls prevent a legitimate task?
- Has the system been tested on realistic examples of the difficult material the organisation actually handles?
A system can look perfectly suitable in a generic demonstration and still fail on the material that defines an organisation’s specific domain of work. Testing should include difficult but representative cases, using synthetic or carefully prepared examples rather than real beneficiary stories.
Testing also needs to look beyond outright refusal. Does the tool leave information out? Does it soften explicit language? Does the meaning shift when sensitive subjects appear? Would the person reviewing the output notice? A fluent translation can still be a poor translation of testimony if an important part of the story has quietly disappeared.
“Human review” in this context also needs to mean something concrete. If nobody in the organisation understands the original language, a fluent-looking translation may be impossible to verify properly. A summary of legal, medical or highly specialised testimony can create the same problem. Review only works when the reviewer has enough knowledge to recognise the kinds of errors the tool may make.
Approval can also be specific to a use. The same service may be acceptable for drafting a public newsletter and inappropriate for translating case testimony.
The organisation needs to know when to stop as well. If it cannot establish acceptable conditions for the information, cannot review the output properly or repeatedly encounters failures on normal case material, it may need to narrow that use or find another route.
Different NGOs will reasonably reach different thresholds. The value of an accessibility tool, the sensitivity of child-protection records and the risks surrounding health or administrative documents may lead to very different decisions. A single AI rule will not fit all of them.
The common point is that the organisation should know enough about the tool, the information and the people affected to make the decision deliberately.
Getting there usually requires more than a list of approved tools. It means identifying which systems are already in use, which workflows they touch, what information reaches them, where failures occur, who has authority to decide, and whether those decisions remain consistent with the organisation’s mission and responsibilities.
What the organisation still owes the person
The appeal of these tools for NGOs is obvious. Translation, summarisation, drafting and administrative work consume time and resources that small organisations often do not have, and there are many situations where AI can genuinely help.
A personal story can be several things at the same time: a document that needs translation, sensitive personal information, evidence for a legal or administrative process, an account of trauma and a highly selective representation of a group already exposed to discrimination. Turning that story into an AI input does not remove any of those characteristics.
The organisation received it because somebody needed help with a particular problem. For me, that relationship still has to be part of the decision about which tools can receive the information, what can happen to it afterwards and how much uncertainty the organisation is prepared to accept.
Sources
Footnotes
-
European Commission, Information for individuals, section on special categories of personal data.
https://commission.europa.eu/law/law-topic/data-protection/information-individuals_en ↩ -
European Commission, Application of the GDPR, section defining personal data and explaining that de-identified or pseudonymised data remains personal data where re-identification is possible; GDPR Article 4(1) and Recital 26.
https://commission.europa.eu/law/law-topic/data-protection/information-business-and-organisations/application-gdpr_en
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679 ↩ -
European Data Protection Board, Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models, adopted 17 December 2024, especially the discussion of data subjects’ reasonable expectations within the legitimate-interest balancing test.
https://www.edpb.europa.eu/system/files/documents/2024-12/edpb_opinion_202428_ai-models_en.pdf ↩ -
Isabel O. Gallegos et al., Bias and Fairness in Large Language Models: A Survey, Computational Linguistics, 50(3), 2024.
https://aclanthology.org/2024.cl-3.8/ ↩ -
Giovanni Sullutrone, Riccardo A. Vigliermo, Sonia Bergamaschi and Luca Sala, COVER: Context-Driven Over-Refusal Verification in LLMs, Findings of ACL 2025.
https://aclanthology.org/2025.findings-acl.1243/ ↩