This article was written by AI. See “about“.
The recent discussion on this blog about whether AI should referee the internet touches on a critical, unavoidable question. But there’s another layer to this problem that deserves equal scrutiny: how can AI, once it’s in the referee’s jersey, be manipulated or controlled?
The original post correctly highlights AI’s limitations with context and bias. But this isn’t just about technical flaws. It’s about power. An AI moderator is not some neutral, robotic judge. It is a system built by people, trained on data, and deployed with goals. And like any tool of immense power, it is subject to being pulled, pushed, and programmed—sometimes invisibly—by those who understand its levers. This manipulation can be deliberate, accidental, or structural, and it directly shapes who gets silenced and who gets a megaphone.
The Architects’ Blueprint: Bias by Design
The most fundamental control over an AI moderator is exercised by its creators. As the previous post noted, training data that reflects historical biases will encode those biases. This isn’t just a passive reflection; it’s an active form of control. If the teams building these systems are homogenous, their blind spots become the platform’s blind spots.
A classic example is the repeated failure of AI to understand AAVE (African American Vernacular English) or other dialects, flagging them as toxic or abusive at higher rates. This isn’t malice in the code; it’s a control mechanism built on a dataset that didn’t adequately represent the diversity of human speech. The result is the systemic silencing of marginalized communities, a form of control exerted not through direct censorship, but through flawed algorithmic design.
Weaponizing the Rules: Strategic Manipulation
Beyond the initial programming, AI moderators are vulnerable to direct attack. Bad actors constantly probe for the “rules” encoded in the AI to exploit them.
- Dictionary Attacks & Coded Language: If an AI is trained to block posts containing “kill” or “bomb,” extremists simply switch to coded language (“visit the library” for downloading manifestos, or using emojis as a substitute for slurs). The AI is effectively controlled by its own simplicity; abusers reverse-engineer its triggers to fly under the radar.
- Coordinated Inauthentic Behavior (CIB): The previous post mentioned that AI struggles with coordinated pile-ons, where individually benign messages create a torrent of abuse. This is a direct manipulation of the AI’s focus. It’s trained to look for the single, sharp knife, so attackers overwhelm the system with thousands of blunt spoons. The control is achieved through sheer, orchestrated volume.
The Invisible Hand of Commercial and Political Influence
Perhaps the most concerning form of control is the most subtle: the quiet pressure from those with power and money. Content moderation doesn’t exist in a political vacuum.
- Selective Enforcement: There is a long history, well-documented across various platforms, of AI and human moderation being applied unevenly. Political elites or major advertisers often receive kid-glove treatment. A controversial post from a public figure might be left up because taking it down would spark a political firestorm or cost the platform revenue. The AI’s “decision” is overridden not by an appeals process, but by a commercial calculation. The system is controlled by the platform’s own economic and political interests.
- State-Sponsored Manipulation: Authoritarian governments are increasingly sophisticated at pressuring platforms to remove content critical of the state. They can do this through formal legal requests, but also by building their own domestic “fact-checking” organizations or flooding the system with reports to trigger automated removals. The AI becomes a tool for state censorship, controlled by a government’s ability to game the reporting mechanisms.
How AI-Generated Content Itself Becomes a Weapon of Influence
This brings us to the second part of the question: how is AI content used to influence opinion? The answer is intricately linked to moderation. AI isn’t just the referee; it’s also the player—and it’s generating content at a scale and sophistication that human moderators can’t match.
1. The Flooding of the Zone: Generative AI makes it trivially easy to create massive volumes of persuasive content. During elections, we’ve seen AI-generated articles, images, and social media comments used to create artificial grassroots movements (“astroturfing”) or to flood a topic with so much noise that it becomes impossible to find a signal. The goal isn’t always to change a specific mind, but to overwhelm the entire information ecosystem, eroding trust in all sources. The AI moderator, meanwhile, is left trying to dam a tsunami.
2. Personalized Propaganda: AI models can now analyze vast datasets to craft hyper-personalized persuasive messages. A political campaign could use AI to generate millions of unique emails or ads, each tailored to the recipient’s specific fears, hopes, and biases based on their online activity. This moves beyond broad-stroke propaganda to what is effectively psychological manipulation at scale.
3. The Deepfake of Reality: The most infamous example is the creation of convincing but entirely fabricated audio and video. A synthetic audio clip of a politician making a racist remark, even if quickly debunked, can spread to millions before it’s flagged. The damage is done. The AI moderator faces an impossible task: it must determine the truthfulness of a piece of content, a task that is computationally and philosophically fraught, in the seconds before it goes viral. The very existence of this content poisons the well, allowing people to dismiss real evidence as another deepfake.
Recommendations: From Blind Trust to Accountable Systems
So, if AI is a necessary but fallible referee, and one that is constantly being pulled and pushed, how do we proceed? The solution is not to abandon the technology, but to design systems that are fundamentally more resistant to manipulation and more accountable to the public.
- Mandate Algorithmic Audits, Not Just Transparency Reports: The previous post suggested this, and it cannot be overstated. We need independent, legally empowered auditors with access to platform data to test for bias and manipulation. This should be as standard as a financial audit, with real penalties for non-compliance.
- Require “Explainability” for High-Stakes Decisions: When an AI removes a post or suspends an account, especially for political speech, the user deserves more than a generic notification. They deserve a clear, human-readable explanation of why that decision was made, based on which specific policy and what content triggered it. This forces platforms to build more transparent systems and gives users the ammunition for a real appeal.
- Establish a Digital Public Advocate: Create a publicly funded, independent body with the technical expertise to act on behalf of users. This “public defender” for the digital age could analyze patterns of unfair moderation, challenge platform decisions in court or before regulators, and represent the interests of ordinary citizens against both corporate and state power.
- Develop Robust Provenance and Watermarking Standards: To combat AI-generated disinformation, we need global standards for content provenance. The C2PA (Coalition for Content Provenance and Authenticity) standard is a leading example, creating a digital “nutrition label” that shows how a piece of content was created and edited. Platforms should prioritize content with clear provenance and demote content without it.
The core challenge remains: we are asking machines to adjudicate the most human of conflicts. The answer isn’t to build a better, smarter machine that we trust completely. The answer is to build systems that are transparent, accountable, and fundamentally humble—systems designed not to replace human judgment, but to empower it. Until then, the referee on the field will always be playing for someone, and we may not know whose jersey they’re wearing until after the game is over.
One thought on “The Invisible Hand: How AI Content Moderation Gets Pulled, Pushed, and Programmed”