Executive Summary

We propose a novel approach to mitigate AI-driven market misalignments. Traditional solutions—coordination, regulation, and redistribution—centralize power and fail to address root causes. We propose a system where AI-powered market intermediaries mediate transactions by evaluating and rewarding providers based on achieved outcomes rather than direct consumer payments, generalizing outcome-based contracting to domains where it was previously intractable. These intermediaries employ assessment, matching, optimization, and payment routing mechanisms to restructure market incentives. A prototype experiment in the human labor market for flourishing is outlined, involving 200 participants. This approach offers the potential to fundamentally realign markets with human values across various domains, addressing both harmful product incentives and potential human labor displacement.

Market Misalignment and Human Flourishing

Many AI risks are driven by markets misaligned with human flourishing:

There are markets for things that are bad for us, such as AI arms races among nations and labs, the market for AI girlfriends and other hyper-stimulus, isolating distractions, and markets for political manipulation and destabilization.
There are markets that displace us entirely, such as markets to replace ethically-concerned human workers with AI workers that put business objectives first, or potentially, markets that displace all human labor, such that humans lead meaningless lives as mostly consumers, and AGI-generated wealth is potentially concentrated among a few providers.

We can summarize these as failures of markets to put human values and meaning on a par with (what should be) instrumental goals like engagement, ROI, or the efficient use of resources.

There are three common responses to these problems with markets. Each centralizes power:

One response is to keep the number of market actors low, and to coordinate between them such that they voluntarily resist market incentives for a time. This can only work when the number of players is small enough that they can meet and agree. [Ex: safety evals, AI lab pledges]
Another response is regulation, such that market actors mostly follow market incentives but are forced to stop just short of the bad outcomes. Regulation centralizes power in the state (or as is likely necessary, a global governmental body) and in the small number of large actors which can afford to comply with regulations. [Ex: AI executive order, EU AI act, Pause AI]
A third response is to let a bad outcome occur (such as human displacement) but then redistribute wealth to minimize the damage. Redistribution also centralizes power: in the hands of AI providers accumulating capital, and in the states which redistribute it. [Ex: UBI]

Not only do these approaches centralize power, they also don’t actually re-align markets: markets continue to pull the wrong way, patched by pledges, regulations, or redistributions.

A New Approach: Market Intermediaries

We believe powerful AI can be used to deeply align markets with what's actually good, eliminating both classes of market failure described above.

The idea is that buyers buy through an AI intermediary; sellers are then paid by the intermediary based on the 'goodness' (as defined by the buyers) they produced, rather than by the services rendered.[1] In other words, such a market intermediary uses non-market data about good outcomes for buyers to route resources from consumers to providers.

One way to think of market intermediaries is that they make outcome-based or performance-based contracts ubiquitous in a market, by externalizing the costs of specification and auditing, and removing principle-agent problems in contract design.

In some fields, outcome-based contracting is already the norm: aircraft maintenance is provisioned via performance-based or “power-by-the-hour” agreements, where what matters is whether the aircraft can fly at the scheduled times. In health insurance, there are value-based contracts or capitation payments: agreements between insurers and health providers that tie payment to patient or population health outcomes, not to services rendered. In cloud compute, service-level agreements tie payments to cloud service uptime.