<aside> 👉

For background on these projects, see our position paper.

</aside>

As we’ve mentioned, we want to focus the research we coordinate under the FSA banner. We’re considering organizing work under four “moonshots” and four more “foundational areas”.

Read below for more on each!

<aside> 📋

Table of Contents

</aside>

The Moonshots

(m1) a supernegotiator

Two kinds of search costs slow down conflict negotiation and settling on arrangements.

• First, the slow search for workable coalitions. Were a coalition to exist, it might be able to make it worthwhile for both parties to de-escalate or accept some terms. In geopolitics, this may be a coalition of neighboring countries. With an argument between startup founders, this might be their community of investors and advisors, or a broader community which benefits from good practice in the area. Conflicts often continue because such a coalition cannot form. The search for terms on which the surrounding countries or community could work together, and what conditions they could offer, requires information discovery which never gets done. We currently do narrow searches in this space: Will surrounding countries issue some economic sanctions? Join a pre-existing coalition? Will they pressure one side to pay the other some money? Give up some property? We believe AI can lower this search cost.

• Second, the search for as-yet-unconsidered upsides in dealmaking. Often, those forming a contract or treaty have no idea how much they could really help each other. For instance, an employer may offer an employee a job, who accepts certain pay for it; but that same employee may be worth much more in a different position. Neither side has the information required to suggest that other deal. We believe these unconsidered upsides exist in other contexts, such as economic agreements between nations, and arrangements for local property use.

  Again our current processes for treaties or contracts takes little information into account about both sides’ values, and does little exploration of the solution space. This, too, is an area where AI can help. In this case, there’s information to be collected about what really matters to the parties in question: what they value; what makes their lives worth living; what’s at stake.

Research direction: we suggest factoring these search problems into two pieces.

• First, tuning information discovery: what information is needed from both parties to build coalitions and surface hitherto unconsidered upsides?

• Second, accurately modeling the evaluation processes of both sides to guide search, finding deals or coalitional arrangements that are successively more “win-win”, such that longer searches and more options result in even more upside for the relevant parties.

  This may require figuring out things like how and whether a deal promotes a certain value that a party holds, how the values are to be represented (perhaps not just as payoffs, but as CAPs or some other “thick” representation that compiles to a preference order over deals).

Putting them back together, a researcher could create a loop where an agent, through human feedback, gets better at asking the right questions for information discovery, and better at predicting which of two deals will seem better to both sides, such that when agents ideate, passing back and forth possible deals, they can be pretty sure the humans will accept one.

<aside> 🧪

Some research ideas

• A Multi-Agent System for Expanding the Scope of Agreements: Design a multi-agent system where AI agents must find a novel agreement that human principals prefer over their own initial proposals. Optionally, try to refine the information discovery mechanism in parallel with the search mechanism. (This is somewhat related to work in open source game theory.)
• Empirical Comparison of Preference- vs. Values-Based Generative Mechanisms: Generative Social Choice is a promising approach to navigating huge option spaces, but is hobbled by its need to extrapolate from preference-profiles instead of richer/thicker values profiles of the agents involved. Implement a "Values-Based Generative Social Choice" system that uses rich, qualitative value profiles to generate potential agreements. This project would empirically test its performance against traditional mechanisms that rely on simple preference rankings, measuring outcomes based on participant acceptance rates, perceived fairness, and novelty of the solutions.
• Formal Foundations for Values-Based Mechanism Design: Develop a formal mathematical framework for designing negotiation and social choice mechanisms that operate on explicit representations of values and norms, rather than just utility functions. This theoretical paper would explore how reasoning about principles can lead to new types of cooperative guarantees and justifiable equilibria that are out of reach for preference-only models. </aside>

<aside> 💬

• From the FSA paper

• More open questions </aside>

(m2) a public-resource regulator

Imagine a small community that lives by a lake. There are a million decisions about what happens on the lake: How many motorboats can be on the lake? How loud? How big? Which events happen on the beach? Who’s invited? What are the rules for sunning yourself? When someone wants to take over the lake for a wedding or a special event, how does that work? And so on.

Currently, the citizens around the lake might elect a lake management council that meets quarterly to make these decisions. But that means cool opportunities get lost if they can't wait for this review process. In the near future, ultra-fast AI agents may be building new kinds of boats, and spinning up events that the people around the lake might like or might hate.