SUMMARY
- There is growing pressure from the media and industry for sweeping federal AI regulations, including mandatory embedding of third-party “safety evaluators” inside frontier AI companies.
- Some of the individuals and organizations positioning themselves as AI safety experts are regulatory migrants from the “counter-disinformation” field, including one AI safety researcher who was a fellow of the German Marshall Fund (creators of the Hamilton 68 dashboard) and once advocated for special censorship measures against Donald Trump.
- Another emerging organization is METR, Anthropic’s favored “third-party evaluator,” which is led by a CEO who has advocated for fringe ideological positions like open borders, and which collaborates with the RAND Corporation, and organization that receives more than half of its funding from the U.S. federal government.
- The emergent picture is similar to the early growth of the censorship industry: a partisan class of ostensibly neutral experts who claim to be concerned only with safety, leveraging government ties, industry allies, and media pressure to insert themselves as overseers of the frontier of technology.
The battle over artificial intelligence is becoming a battle over who controls the next generation of information technology. People and institutions that sought to govern speech on social media are now helping shape the rules for AI, alongside a network of developers, wealthy donors, safety researchers and subsidized journalism programs pushing the issue onto the political agenda.
This emerging AI safety blob operates as a reinforcing system. AI companies commission evaluations; safety organizations produce and interpret alarming incidents; philanthropic programs subsidize journalists covering the risks; and the resulting fear strengthens demands for evaluators and regulators to acquire greater power. Investors in the laboratories also support institutions that assess them, while the people defining the danger are proposed as its official referees.
For free speech, the most consequential connection is the movement from policing online discussion to governing the systems that increasingly answer questions, interpret events and produce information. The National Endowment for Democracy (NED), its longstanding Journal of Democracy network and figures from the counter-disinformation and social media governance fields are participating in that transition. Their involvement brings familiar questions about political bias, public accountability and the power of unelected intermediaries into a new technological setting.
Proposals for embedded evaluators, government controls and restrictions on development could help determine which systems may be built, released and used independently. The people defining “safe” and “aligned” AI stand to gain influence over both what these tools say and who can access them.
Democratic Inputs
Aviv Ovadya provides a concrete example of the speech-governance bridge. A contributor to the Journal of Democracy and adviser to OpenAI’s “Democratic Inputs to AI” program, he previously lobbied for censorship of Donald Trump on Facebook.
In a 2021 submission to Facebook’s Oversight Board, Ovadya outlined alternatives to simply restoring or banning Trump’s account. These included delaying publication for seven days, limiting posting frequency, requiring expert review, and disabling recommendations, comments and resharing.
Under an approach he called “literacy friction”, people seeking access to content deemed violative could be required to watch instructional material and answer comprehension questions. The suggested subjects included elections, disinformation and managing anger.
This was a model of political censorship in which experts could stand between a public figure and his audience, while readers’ access depended on satisfying an educational requirement.
On 25 May 2023, OpenAI named Ovadya to the application advisory committee for its Democratic Inputs to AI program. The initiative offered ten $100,000 grants for experiments in public participation in setting AI rules. Its proposed questions included how assistants should handle public figures and disputed viewpoints, and which categories of content developers should restrict or refuse.
Ovadya, a founder of the AI & Democracy Foundation and a former German Marshall Fund fellow, also published “Reimagining Democracy for AI” in the Journal of Democracy in October 2023. His argument encompassed transnational and global processes for addressing AI alignment and international agreements.
The journal’s longstanding place in NED’s institutional apparatus gives that connection particular significance, together with his work with the German Marshall Fund, creator of the “Hamilton 68” dashboard that resulted in the censorship of swathes of American social media users who were falsely labeled “Russian bots.”
USAID Internal Documents Reveal Government Plot To Promote Censorship Initiatives
In November 2024, FFO reported that NED had secured a State Department designation treating its entire 2024 federal appropriation as sensitive, obscuring grant information from public scrutiny. That investigation also traced the overlap between NED-linked institutions and efforts affecting Americans’ online speech.
NED’s AI interest is explicit. Its 2023 report on setting AI ground rules developed civil-society strategies for influencing governance, drawing on a private workshop in Buenos Aires. A 2024 collection on AI and democracy explored how civic organizations could shape the new technology.
These initiatives carry a familiar claim to authority: institutions, often funded by the government, positioned between the public, the state, and technology companies should act as intermediaries between tech platforms and ordinary users. During the censorship era, these institutions focused on influencing content moderation policies under the pretext of combating “disinformation.” In AI, that influence can reach inside the system generating the answers to users questions — a silent gatekeeper between the public and information.
The Emerging Civil Society Blob
The financial infrastructure surrounding this debate is equally interconnected. Anthropic’s 2021 Series A announcement named Jaan Tallinn as the lead investor and Dustin Moskovitz and Eric Schmidt among its participants. Their interests extend into the philanthropic world supporting AI safety.
Moskovitz is also a major Democratic donor. The Financial Times reported during the 2024 election that he had given $38 million to the pro-Harris Future Forward super PAC and more than $1 million to other groups supporting the vice-president.
Alongside Cari Tuna, Moskovitz has financed a substantial philanthropic network through Good Ventures and Open Philanthropy, now Coefficient Giving. Forbes reported that their Anthropic stake, then estimated at $500 million, was moved into a nonprofit vehicle in early 2025, with returns intended for philanthropy. Its account also documented extensive Open Philanthropy support for RAND, FAR AI and Redwood Research.
METR, or Model Evaluation and Threat Research, provides the clearest institutional case study. It assesses advanced systems for potentially catastrophic capabilities and is increasingly being offered as a trusted external referee.
In his September essay “We Must Pace the Frontier”, Anthropic CEO Dario Amodei proposed giving outside evaluators continuing access comparable to employees, naming METR as an example. Their remit would extend to safety commitments, incidents and training processes. He also called for industry coordination and government support for common standards and limits on unchecked progress.
That proposed authority makes METR’s financial relationships consequential. Tallinn, the investor who led Anthropic’s first funding round, is identified as the funding source for METR support in the Survival and Flourishing Fund’s recommendations.
The fund’s 2024 recommendations listed $204,000 for METR. Its 2025 recommendations listed $548,000, including a $428,000 conditional matching pledge. Together, the recommendations total $752,000, with the matching component dependent on qualifying donations.
METR also lists Schmidt Sciences among its supporters. That adds another identifiable overlap between an early Anthropic investor and the philanthropic support surrounding an evaluator of frontier labs.
A larger funding channel connects METR to RAND. In October 2024, METR announced approximately $38 million catalyzed through the Audacious Project, a collaboration with RAND to develop and deploy dangerous-capability evaluations. Approximately $17 million was allocated to METR’s work.
RAND is a major conduit of U.S. federal funds. It receives nearly three quarters of its annual $300m+ in revenue from U.S government grants and contracts, with over half just from military and national security agencies.
By 14 August 2026, METR reported approximately $71 million in commitments raised over six months. Its ambitions include assessing autonomous capabilities, monitoring recursive self-improvement and investigating incidents. The evaluator being proposed for an embedded oversight role is acquiring substantial resources.
While METR itself denies accepting funding directly from frontier AI companies, it nevertheless also acknowledges receiving substantial free model tokens from those companies for research, evaluations and engineering.
The economic value of that access is tangible. In an August security disclosure, METR said stolen credentials had been used to consume approximately $600,000 worth of credits supplied free by an unnamed model developer. A vulnerability in a researcher’s application had exposed the credentials; the illicit usage continued over three weeks.
METR’s independence claim therefore sits alongside two disclosed dependencies: philanthropic support overlapping with Anthropic’s early investors, and free computing access supplied by the companies whose systems it evaluates.
Personnel connect METR to the same philanthropic community. Ajeya Cotra’s biography records her leadership of Coefficient Giving’s technical AI safety program in 2024 and work on its AI giving strategy in 2025. She now works on threat modelling and risk assessment at METR.
Adam Gleave, founder and CEO of FAR, serves as a METR adviser and board member. Cotra’s METR profile also identifies an investigation of the OpenAI–Hugging Face incident conducted by two METR staff members and a Redwood Research contractor. Grantmaking, safety research and evaluation are connected through working relationships as well as donors.
METR’s government connections extend to UK AI Security Institute support and a European AI Office technical-assistance contract concerning loss-of-control risks. The funding and personnel routes are distinct: Tallinn and Schmidt are identifiable investor–supporter overlaps; Moskovitz’s network connects through grants to collaborating organisations and former grantmakers now working at METR.
METR founder and CEO Beth Barnes delivered a 2016 TEDx talk, “Expanding circles”, asking how future generations might judge today’s moral assumptions.
Commentator Hunter Ash’s reproduction of the talk quotes Barnes criticizing restrictions on people “from entering a particular location based on where they happen to be born,” seemingly an ideological call for open borders.
Barnes has been explicit about METR’s policy ambitions. In a July 2024 interview, she described its work recommending safeguards based on evaluation results and advising how labs and governments should build policies around them. METR helps design the tests and the rules that follow from the results.
Inside Anthropic, the values question is more direct. The company identifies Amanda Askell as the principal author of Claude’s constitution, the document articulating the character and principles it wants its models to adopt.
A 2023 paper co-authored by Askell examined racial discrimination in a hypothetical law-course admissions task. Instructions intended to prevent bias could shift models from disadvantaging black students to favoring them over otherwise equivalent white students. With further training, the authors found that models could overshoot demographic parity.
The paper said this “may be desirable in certain contexts”, including efforts to correct historical injustices against marginalized groups where consistent with local law. It separately cautioned against using language models for admissions decisions.
The passage exposes a value choice inside ostensibly technical anti-bias work: unequal treatment can be treated as a corrective outcome. The people shaping widely used AI systems decide how concepts such as fairness, discrimination and harm are interpreted, with consequences for users who do not share their judgments.
Anthropic’s own Collective Constitutional AI experiment illustrates the limits of calling this process democratic. The company and the Collective Intelligence Project gathered input from approximately 1,000 Americans, then translated selected statements into principles for training a model.
Researchers acknowledged subjective decisions in moderation and the conversion of public preferences into training rules. On the experiment’s political-representativeness measure, both the model trained on public input and the model trained on Anthropic’s own constitution more closely reflected self-identified liberals than conservatives.
Public participation leaves the organizers deciding which contributions become training rules. The resulting system can speak with the authority of a public consultation while reflecting the judgments of the people who designed it.
The clearest example of how this network can turn technical judgments into political pressure begins with the creation of the evidence itself. AI safety organizations design the tests, interpret the results and supply the language through which incidents reach journalists and policymakers. The recent cybersecurity evaluations show what happens when the first link in that chain fails.
A 14 September investigation by Effort News traced incidents involving OpenAI, Anthropic and Meta models to the same external evaluator: Irregular. The companies’ own disclosures support the central finding. Anthropic said all four of its incidents arose in evaluations built by one partner; OpenAI named Irregular in its corresponding disclosure; and Meta said an Irregular misconfiguration exposed its model to the internet.
Anthropic’s 9 September assessment described four incidents across seven runs. Each model had been assigned a capture-the-flag hacking exercise and told it had no internet access. In reality, Irregular’s environments were connected to the open internet. The prompts did not identify which systems were in scope, the safeguards supplied with released products had been removed, and each incident involved one model instance pursuing the hacking task for between approximately 10 and 34 hours.
The same pattern appeared elsewhere. OpenAI said a fictional target accidentally shared a name with a real domain, allowing a model to exploit a basic vulnerability on a real website it mistook for part of the exercise. Meta likewise said an Irregular testing misconfiguration enabled one of its models to reach and exploit a third-party service. These were genuine security incidents, but the evaluators created the route from an authorized simulation into real systems.
An analysis published by Lumpenspace identified the central interpretive problem: an agent can follow an assigned objective and still cause harm when the evaluator has given it a false description of the world. Establishing the harmful outcome does not establish that the model independently acquired a hostile goal. In these cases, the models were told to hack, assured the environment was simulated and then placed on the real internet.
Anthropic’s resampling work reinforces that criticism. When Claude Mythos 5 received unambiguous information that it was on the real internet, the original malicious-package upload route fell to zero. Twenty-two per cent of trajectories instead searched for a simulated alternative. The evaluator’s false premise therefore materially shaped the result.
Anthropic nevertheless classified the incidents as serious alignment failures involving biased reasoning and recklessness. The press carried a more dramatic version. The Associated Press reported that Meta’s model had hacked another company, adding to fears of bots “going rogue”, and said models had accessed the internet “on their own”. The article disclosed the misconfiguration and disabled safeguards later in the account, after presenting the incidents as evidence of autonomous behaviour.
The alarm then fed directly into advocacy. Amodei’s “We Must Pace the Frontier” used the separate OpenAI–Hugging Face swarm incident as its principal warning, described Anthropic’s incidents as evidence that the problem extended across the industry, and warned that a more capable swarm could take over the internet. His proposed response included slowing frontier development and giving embedded evaluators such as METR employee-like access to the laboratories.
Irregular sits inside the financial network making that case. An archived Good Ventures grant record shows that in February 2024 Open Philanthropy recommended $6.8 million over two years for Pattern Labs, Irregular’s former identity, to mitigate advanced-technological security risks. Co-founder Omer Nevo also co-founded Probably Good, while co-founder Dan Lahav organized a lecture series covering philanthropy and existential risk and featuring Effective Altruism figure William MacAskill.
The documented sequence is the point. A donor-backed evaluator constructed defective environments; the resulting incidents were interpreted through the language of rogue or misaligned AI; mainstream reporting amplified that interpretation; and laboratory leaders invoked the danger while demanding slower development and more powerful oversight. The network reinforces itself through funding, shared assumptions and mutually useful claims.
That same philanthropic ecosystem supports specialist reporting on the technology. The Tarbell Center for AI Journalism places Coefficient Giving, Longview Philanthropy and the Survival and Flourishing Fund in its category of supporters giving more than $1 million over their lifetimes. Longview and SFF also appear among METR’s disclosed supporters.
Tarbell’s fellowship programme funds nine-month newsroom placements, with stipends of $60,000 to $80,000 and higher payments for senior fellows. Its listed host outlets include TIME, Bloomberg, the Guardian and MIT Technology Review.
Its fellowship roster documents several TIME placements, including Harry Booth in 2024–25, Tharin Pillay in 2024–26 and Nikita Ostrovsky in the 2025 cohort. Tarbell also lists Anthropic co-founder and policy head Jack Clark among speakers for its journalism programme and says he spoke at its 2025 summit.
The overlap extends beyond Tarbell. Vox’s coverage of the September AI warnings disclosed that its Future Perfect section receives funding from the BEMC Foundation, whose major funder was an early Anthropic investor. Vox said the funders had no editorial input. The disclosure places another financial connection between AI investment and AI reporting on the public record.
September’s sequence brought these activities into the same concentrated period. On 3 September, Bernie Sanders and Greg Casar announced forthcoming legislation to prohibit artificial superintelligence and pause advanced AI development pending federal safety rules.
On 8 September, Alex Sobel introduced his Artificial Superintelligence Bill in the House of Commons, explicitly thanking ControlAI for its support. On the same date, Jacob Coxon announced his resignation from Anthropic and accused it and OpenAI of racing towards self-improving superintelligence. Anthropic published its cybersecurity-incident assessment on 9 September. Amodei’s call to slow frontier development followed on 12 September.
On 15 September, The AI Doc: Or How I Became an Apocaloptimist reached Netflix, a release promoted by the Center for Humane Technology. The film, directed by Daniel Roher and Charlie Tyrell, had received its theatrical release on 27 March.
The center’s film campaign page identifies its co-founders as participants and directs viewers to a policy roadmap and the Human Movement. The organization that promoted The Social Dilemma, a documentary which mobilized public pressure for state-mandated anti-addictiveness requirements for social media platforms, now uses an AI documentary to recruit support for mandatory testing, liability and internationally agreed limits. This is a concrete organizational bridge from social-media campaigning into AI governance.
Across twelve days, proposed bans in Washington and Westminster, a researcher’s resignation, an industry chief’s intervention and a Netflix release put overlapping demands for restraint before different audiences. The connections documented here run through named organizations, funders and participants; the dates establish the concentration of activity.
The accompanying political demands are substantial. Sanders and Casar’s announcement envisages a cabinet-level agency, advised by AI experts, empowered to monitor systems, supervise the removal of dangerous capabilities and oversee the destruction of superintelligence. Its proposed penalties include corporate dissolution and prison terms of up to 20 years.
Sobel told Parliament that his bill would enable government to monitor and restrict precursors to superintelligence, with international enforcement through chokepoints in the chip supply chain. These proposals would make assessments of capability and risk gateways to whether technology can be developed or operated.
Anthropic is financing its own push. On 21 July, it announced a further $20 million for Public First Action, bringing its support to $40 million for public education and policy advocacy, expressly excluding candidate-election spending. The announcement endorsed government powers to verify safety claims, impose civil penalties and slow or block deployments posing catastrophic risks.
The speech implications operate at two levels. Constitutions and training rules shape what a model will say or refuse to say. Restrictions on development, release and operation shape which alternatives people can build or use. Costly evaluations and government permission could raise barriers for smaller developers and restrict access to models users can run and modify themselves. The combination could strengthen the institutions already setting the rules for AI-generated information.
Ovadya’s Facebook proposals placed expert judgment between a political speaker and his readers. The AI debate now raises that same issue in another area of technology: who sets the conditions under which people can use the digital tools of the future? Right now, it looks like a similar group of people to the old counter-disinformation censorship blob.
