Scaling the Good - OpenAI $180B Deployment Strategy

With OpenAI’s new raise at an $852B valuation, OpenAI Foundation’s stake is now worth $180B. Anthropic’s cofounders have pledged to donate 80% of their wealth. Nobody seems to have a concrete idea of how to deploy 100s of billions (soon trillions) of wealth productively to “make AI go well”. If you were in charge of the OpenAI Foundation right now, what exactly would you do? And when? It’s not enough to identify a cause you think is important, because that doesn’t answer the fundamental problem of how you convert money to impact. Identify the concrete strategy you recommend pursuing.¹

Uncertainty about the goodness of our ‘impact’—a metaphysical and political question—is the crucial hidden constraint in the problem. Modeling it explicitly clarifies the shape of plausible solutions.

Desiderata

Scalability: Building an apparatus that can reliably cash out each marginal $100B+ in substantive benefits with decent returns to scale.
Complementarity: Converting (other people’s) dollars into benefits in the form of useful tokens is already the core business; it will already be finding (if $850B+ valuations are to be maintained) productive economic uses. Effective giving will fund activities that people are unable, unwilling, or unaware of wanting to pay for, but which nonetheless deliver benefits. Donations must avoid crowding out the market.
Leverage: Transferring $180 billion directly to the poorest billion people is the conversion efficiency floor.
Shared Valence: Deploying the wealth of a small nation state to ‘do good’ across variegated societies requires tact, diplomacy, and statesmanship-sans-state. Our intuitions about what counts as a benefit do not translate cleanly across cultures. Avoiding moral adventurism is a first-order goal. “Half-court tennis” is as bad a strategy in international aid as it is in war-making.²

We ignore (4) at our peril. Silicon Valley knows that making something people want demands “good taste” plus feedback. At global scale, ethical taste is not culturally invariant, and our moral intuitions, mostly learned within our culture, perform poorly on out-of-sample problems. Feedback must be sought out in philanthropy, and when it is, it is often idiosyncratic; nice solutions in one place cause unforeseen problems in another. Unitized, copy-paste solutions necessary for efficient scaling often fail to deliver the expected benefits. EA works hard to identify the narrow class of problems for which copy-paste works well. But in general, technocratic schemes to ‘benefit all of humanity’³ have a nasty habit of failing for want of sensitivity to local needs and ignorance of local knowledge.⁴

Big donors have (geo)political reasons to attend to (4) as well. Large scale global ‘impact’ perceived as unwanted outside influence or violating sovereignty could find itself quickly rate-limited by political resistance or even military action. Foundations and corporations ultimately exist at the sufferance of states and must calibrate their ambitions accordingly.

Approaching (4) via negativa⁵ is the only way to make it tractable under wide and irreducible variance in ethical preferences. We know this intuitively and gravitate toward solutions like curing diseases because averting suffering doesn’t require a positive grand vision of the good. Taking away a ‘known bad’ is the best we can hope to do for people who might share next to none of our values. (How might we do good for, say, a member of the Taliban?)

The complication for donors maximizing impact is that people are ready and willing to pay to avoid and treat ‘known bads’ like diseases, so giving here fails to satisfy (2) unless it offsets underinvestment or inability to pay. Donations to make up for a shortfall are a good idea (OpenAI Foundation is already pushing here), but social and political complexity binds on (1).

Nevertheless, the disease analogue provides a useful intuition pump: we’re looking for chronically undertreated civilizational-scale maladies, technocratically diagnosable, and treatable by cutting-edge non-invasive surgery.

One concrete example of a solution with this shape is Direct Air Capture (DAC) and related drawdown solutions to climate change. While more dollar- and energy-efficient pathways exist, they require distributed political sacrifice and are therefore politically constrained. Drawdown strategies are capital intensive—expensive in terms of energy, plant capex, and materials—but can be sited in remote areas anywhere on earth. In the optimistic strong AI scenario, market forces and accelerated basic science are already driving the costs of energy, raw materials, and construction lower through process improvement, better digital modeling and simulation, and industrial automation.

The Bootstrap Phase for setting the trajectory toward a drawdown solution scalable by about 2040 could begin in 2026. It would involve:

Seed investments for promising approaches and prizes for well-specified technological and research bottlenecks;
Modular designs built as a proving ground for highly autonomous industrial systems;
Medium-scale deployments for real-world process knowledge and early supply chain scaling;
Exploratory siting analyses for storage and land acquisition plans;
Drafting international organizational and governance structures and soliciting international buy-in for monitorable and continuous reverse-geoengineering;
Soliciting conditional funding commitments from governments so that scaling is go-for-launch (possibly insurmountable political lift).

If the Bootstrap yields a well-optimized modular DAC technical stack for $O(\$1\text{T})$ research spend (assuming Wright’s-Law learning near solar-PV rates), then we’re positioned for (1) since dollars convert to modules convert to captured $\text{CO}_2$ at the $O(\$10\text{T})$ scale. Contributions toward building out and operating the modules satisfy (2) by effectively burying money in the ground with the carbon, with philanthropic dollars eating the cost of capture that no company or state wants to internalize. Leverage for (3) comes from breadth-first “exploration of the tech tree”⁶ to pluck the bridge technologies to tractable DAC early, integrate them with market-driven innovations in energy and materials, and march down cost curves. Finally, (4) works because approximately nobody is attached to $\text{CO}_2$ PPM, counting on future massive refugee flows, or excited about avoidable deaths. It’s hard to imagine stiff political resistance to controlled lowering of $\text{CO}_2$ so long as a bunch of nerds building machines in the desert are footing the bill for it. Most simply won’t care but will still benefit. Averted losses are huge, progressively distributed, and achieved in situ.

Additionally, the optimization target is well-defined, continuously monitorable, and has a clear stopping point. If drawdown somehow gets out of hand (e.g. we’re being paperclipped⁷ by superintelligent AI controlling the modules), localized kinetic strikes on modules give states potential veto power over a runaway process.

It’s worth emphasizing that working constraint (4) via negativa is what unblocks (1). Only good abstracted from particular social and political arrangements scales at the $100B+ margin.

Patel, Dwarkesh. ‘Blog Prize.’ dwarkesh.com/p/blog-prize. ↩
Paine, Sarah C. R. Sarah Paine Lecture Series. Interviewed by Dwarkesh Patel. dwarkesh.com/p/sarah-paine-lecture-series. ↩
OpenAI. ‘About.’ openai.com/about/. ↩
Scott, James C. Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. Yale University Press, 1998. ↩
Taleb, Nassim Nicholas. Antifragile: Things That Gain from Disorder. Random House, 2012. ↩
Nielsen, Michael. Interviewed by Dwarkesh Patel. dwarkesh.com/p/michael-nielsen. ↩
Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, 2014. The ‘paperclip maximizer’ thought experiment originally appears in Bostrom, “Ethical Issues in Advanced Artificial Intelligence” (2003). ↩