Written and maintained by CASRAI Editorial Board
Last updated
Frontier Alignment Research (FAR AI) is an independent AI safety research nonprofit in Berkeley, California, addressing critical technical safety challenges in frontier artificial intelligence. Incorporated as a 501(c)(3) entity, FAR AI operates an integrated pipeline across foundational research, targeted grantmaking, red-teaming, and technical advisory support for public institutions.
What it is and who runs it
FAR AI — incorporated in October 2022 as Frontier Alignment Research (EIN: 92-0692207) and initially established under Players Philanthropy Fund fiscal sponsorship — is an independent 501(c)(3) research organization in Berkeley, California. The institute operates to close the research-adoption gap in frontier AI safety by identifying high-leverage challenges, incubating technical solutions, and scaling vetted safeguards before risks manifest.
The organization was co-founded by CEO Adam Gleave (formerly of UC Berkeley’s Center for Human-Compatible AI) and President Karl Berzins. Executive leadership includes COO Boston Nyer, Head of Finance Taylor Boyle, Head of Programs Lindsay Murachver, and Head of Technology (Frontier Safety) Kellin Pelrine.
FAR AI operates across five core functions:
- Foundational Research: In-house research developing novel mathematical and empirical alignment methods.
- Field Building: Convening workshops, hosting technical events like ControlConf, and funding external investigators.
- Applied Research: Validating promising ideas by testing models at near-frontier scale.
- Adoption: Pre-deployment evaluations and red-teaming with frontier developers and national institutes.
- Incentives: Releasing open benchmarks and advising standards bodies to encourage safety investments.
FAR AI works alongside evaluation groups like METR, provides technical testing to frontier AI labs, leads a consortium developing CBRN risk evaluations for the EU AI Office, and collaborates with the UK AI Security Institute.
Eligibility (summary, with the official source)
Eligibility for FAR AI initiatives depends on the specific program track. As described on the official FAR.AI Programs page on 2026-11-09, the organization runs two primary external tracks:
- FAR.Labs Visiting Residency: Open to individual researchers, academic collaborators, graduate fellows, and aligned nonprofit teams working on AI alignment, robustness, and security. Approved visitors receive short-term residency of up to four weeks free of charge.
- FAR.Labs Long-Term Membership: Open to researchers and organizations seeking ongoing workspace in Berkeley, priced on a cost-recovery basis to offset operational overhead.
- Targeted Research Grantmaking: Open to academic faculty, postdoctoral scholars, and independent technical investigators. As of 2026-11-09, due to evaluation review capacity, grant candidates are evaluated via expert nominations from established AI safety researchers, with open public RFPs planned.
What it funds or requires
FAR AI provides direct financial support and research facilities. As described on the official page on 2026-11-09, the organization funds and requires the following:
1. The $12 Million Grantmaking Fund: Funded by a $12 million grant from Coefficient Giving, FAR AI supports high-impact research neglected by commercial labs. Active funded projects include:
- Weak-to-Strong Generalization: Explaining superhuman decisions in chess and Go, led by Nicholas Tomlin at UC Berkeley.
- Automated Red-Teaming: Agent-based testing environments generating adversarial prompts, led by Wenbo Guo at UC Santa Barbara.
- Lifecycle Alignment Security: Defenses against prefill exploits and fine-tuning attacks, led by Ashwinee Panda at the University of Maryland.
- Superhuman Failure Modes: Robustness against data poisoning, model stealing, and prompt injection, led by Florian Tramèr at ETH Zurich.
2. FAR.Labs Facility Infrastructure: Researchers accepted into FAR.Labs receive Berkeley workspace resources including ergonomic height-adjustable workstations with 4K monitors, Gigabit Ethernet, bookable meeting spaces from call booths to a 50-person seminar room, and daily catered meals and snacks at no charge to visitors.
3. Institutional Compliance and Independence Safeguards: Under its consulting policy, FAR AI caps revenue from for-profit frontier developers at a maximum of 10% of annual operating revenue. It retains publication rights for non-confidential findings and provides pro bono evaluations to public bodies implementing frontier AI frameworks.
Application or compliance steps
Engaging with FAR AI programs follows structured steps as described on the official page on 2026-11-09:
- Track Determination: Choose between short-term residency at FAR.Labs, long-term membership, external grant funding, or technical collaboration.
- Residency Application: Submit a visitor request via FAR AI’s portal, outlining your alignment research agenda, timeline (up to four weeks for zero-cost visits), and expected outputs.
- Grantmaking Nomination: Connect with an established AI safety researcher for an expert nomination under the Coefficient Giving fund, or monitor announcements for open RFPs.
- Compliance and Dual-Use Review: Review disclosure policies. Projects touching severe dual-use risks (such as CBRN or cyber offense) must adhere to coordinated vulnerability disclosure.
- Facility Onboarding: Approved visitors complete orientation at the Berkeley hub, receiving security access, desk assignments, and collaboration tools.
- Dissemination: Grantees and resident scholars publish results in open venues (such as arXiv preprints or machine learning conferences) and present at technical forums like ControlConf.
Typical timeline phases (no calendar dates as facts)
Participation across FAR AI programs advances through five structured phases:
- Phase 1: Application Review: Program staff evaluate residency applications or grant nominations for alignment relevance and facility capacity.
- Phase 2: Onboarding: Scholars arrive at the Berkeley facility, establish computing environments, and join research cohorts.
- Phase 3: Empirical Investigation: Researchers execute agendas, developing white-box probes, auditing games, or adversarial exploit pipelines like STACK.
- Phase 4: Coordinated Disclosure: Vulnerabilities discovered in frontier models are shared confidentially with developers or government partners under defined patching windows.
- Phase 5: Open Dissemination: Findings are published as peer-reviewed papers, open-source code repositories, or policy briefs.
Common mistakes
Researchers and partners commonly encounter several misconceptions regarding FAR AI:
- Assuming FAR.Labs Requires Fees for Short Visits: Qualified AI safety researchers can visit FAR.Labs for up to four weeks free of charge, with workspace and daily meals included.
- Submitting Unsolicited Cold Grant Applications: The grantmaking track currently operates on expert nominations; unsolicited cold applications cannot be processed until public RFPs launch.
- Confusing FAR AI with a Commercial Consultancy: FAR AI caps commercial developer revenue at 10% of total revenue and refuses contracts that suppress public safety insights.
- Viewing FAR AI as a Regulatory Agency: FAR AI conducts technical evaluations and advises bodies like the EU AI Office, but holds no statutory regulatory enforcement powers.
- Treating Red-Teaming as Basic Prompt Engineering: FAR AI focuses on deep technical vulnerabilities, such as internal representation deception probes, prefill jailbreaks, and multi-layer pipeline evasion.
Related programs (table)
The following table compares FAR AI with related AI safety research and evaluation organizations, as described across official organizational disclosures on 2026-11-09:
| Organization / Program | Primary Focus | Governance & Funding Model | Key Facilities & Activities | Distinctive Role |
|---|---|---|---|---|
| FAR AI | Red-teaming, deception detection, adversarial robustness | US 501(c)(3) nonprofit; 10% cap on for-profit consulting; $12M grant pool | FAR.Labs Berkeley co-working hub, targeted grantmaking, ControlConf | Combines physical research residency, independent grantmaking, and multi-layer red-teaming |
| METR | Autonomous model capabilities and catastrophic risk threshold evaluations | US 501(c)(3) nonprofit; accepts no funding from AI developers | Virtual evaluation environments, standardized task suites, horizon-length testing | Third-party quantitative benchmark measurement for catastrophic autonomy |
| Center for AI Safety (CAIS) | Foundational safety benchmarks, compute provision, public advocacy | US 501(c)(3) nonprofit; philanthropic donations | Dedicated GPU compute clusters, WMDP benchmark, safety fellowships | Ecosystem compute allocation, public benchmarks, and policy communication |
| UK AI Security Institute (UK AISI) | Pre-deployment and post-deployment state-level safety testing | UK government agency; public funding | Government testing testbeds, inspectorate access to frontier model checkpoints | Statutory pre-deployment national security and systemic risk evaluations |
| Alignment Research Center (ARC) | Theoretical alignment (heuristic explanations) and catastrophic risk evals | US 501(c)(3) nonprofit; philanthropic grants | Theoretical research seminars, initial foundation of ARC-Eval team | Theoretical frameworks exploring worst-case model transparency and alignment |
FAQ
What does FAR AI stand for?
FAR AI stands for Frontier Alignment Research. Established in 2022, the organization was originally known as the Foundation for Alignment Research before adopting its current name.
Where is FAR.Labs located and how can researchers visit?
FAR.Labs is located in downtown Berkeley, California. Researchers working on technical AI safety can apply via FAR AI’s portal to visit for up to four weeks free of charge, with access to workstations, meeting rooms, and daily catering.
How is FAR AI funded and how does it maintain independence?
FAR AI is a 501(c)(3) nonprofit funded primarily by philanthropic donations, including a $12 million fund from Coefficient Giving. Its governance policy caps revenue from for-profit AI developers at 10% of annual revenue and requires independent publication rights.
What is the STACK attack developed by FAR.AI?
STACK is an adversarial attack methodology developed by FAR.AI that systematically bypasses multi-layer safety pipelines. In empirical trials, STACK achieved a 71% success rate on catastrophic risk scenarios where conventional single-layer attacks registered 0% success.
Does FAR AI conduct evaluations for government agencies?
Yes. FAR AI serves as a technical partner to government bodies, leading a consortium developing CBRN risk evaluations for the European Union AI Office and collaborating with the UK AI Security Institute.
Sources
- FAR.AI Official Website, “About FAR.AI & Team”, as described on the official page on 2026-11-09. URL: https://far.ai/about
- FAR.AI Official Website, “Programs: FAR.Labs & Grantmaking”, as described on the official page on 2026-11-09. URL: https://far.ai/programs
- FAR.AI Official Website, “Transparency, Compliance & Financials”, as described on the official page on 2026-11-09. URL: https://far.ai/transparency
- FAR.AI Official Website, “AI Safety Research Overview & Agendas”, as described on the official page on 2026-11-09. URL: https://far.ai/research
