Our Work

• The best overview of our values-based AI work is our paper on arxiv here:

  https://arxiv.org/abs/2404.10636

• The best intro to our work in economics is to first read our blogpost draft Market Intermediaries: A Post-AGI Market Alignment Vision and then Chapter 1 of this talk of Joe’s Rebuilding Society on Meaning (a bit out of date), and this paper draft https://github.com/jxe/vpm/blob/master/vpm.pdf

• The best intro to our organizational plans and social vision is Ellie and Oliver’s interview with David Shapiro https://www.youtube.com/watch?v=bC2pQ78o754

Our Main Influences

Our work at MAI is woven from 5 academic strands.

1. A set of critiques of revealed preference as a basis for welfare metrics and social choice mechanisms, spelled out mainly by Nobel laureate, welfare econ and social choice pioneer Amartya Sen.
2. An alternate basis for an alignment target / social welfare function / etc, based on a specific definition of human values and a way to reconcile/aggregate the values of groups.
3. Related theories of human moral learning, ****which explain how a person’s values (or a machine’s) might evolve when faced with new situations of moral weight.
4. Theories about the relationship between attention, meaning, and values which inform how we collect values, disambiguate them, etc.
5. Finally, a broad theory of history where big advances in coordination come from new ways to say what matters to us (so we can coordinate around these new vocabularies of benefit).

1. Critiques of Revealed Preferences

From my “Values, Preferences, Meaningful Choice”

In his 1938 paper introducing revealed preference, @Samuelson1938 warned:

I should like to state my personal opinion that nothing said here... touches upon at any point the problem of welfare economics, except in the sense of revealing the confusion in the traditional theory of these distinct subjects.

Similar sentiments followed, in @Arrow1951, @Sen1977, @Anderson2001, etc. As an information basis for welfare, optimality, social choice, etc, revealed preference has been much critiqued.

A rich literature covers how revealed preferences---which, when summed up, are called engagement metrics---lead us astray. You can often get people to choose something without serving their real interests: you can misinform them, or leverage their misplaced hopes.

Or, you can make it so people need your thing for what once was possible without it: they need your car to get to work, your social media account to find a job, your dress to socialize with their friends, etc.

More broadly, you can manufacture social circumstances where people choose your thing to "keep up with the Joneses", to signal allegiance with their tribe, or because they've lost the ability to coordinate a real solution.[See discussions of the prisoner's dilemma in e.g., @Sen1973; @Anderson2001]

In some cases, the person will know their choice doesn't express their true interests---that they are bucking to external pressure, caught in the system, or setting aside their goals to conform to a social rule.[See @Sen1977; @Anderson1993 on 'commitment'] In other cases, options have been limited or biased behind our backs.

Specifically regarding social choice, economist Amartya Sen writes eloquently about how information on people’s preferences—even their stated preferences across a menu of options—isn’t enough for good social choices. Even richer information—about people's wellbeing or utility in different hypothetical states—isn’t good enough.

Preference-based systems like markets, voting, and recommenders don’t carry all the “demand” needed for good social choices.

Many of the failures of markets to provide for what we value (community, adventure, meaningful work, personal growth, ...) are because markets act on the preference supposedly revealed by our choices, rather than delivering what’s actually important to us;