Monday, March 4, 2024

Review of Inadequate Equilibria: Where and How Civilizations Get Stuck by Eliezer Yudkowsky

This book review was written by Eugene Kernes   

Book can be found in: 
Book Club Event = Book List (08/03/2024)
Intriguing Connections = 1) To Cooperate Or To Defect?, 2) When Intelligence Goes Wrong

Watch Short Review


“A critical analogy between an inadequate system and an efficient market is this: even systems that are horribly inadequate from our own perspective are still in a competitive equilibrium.  There’s still an equilibrium of incentives, and equilibrium of supply and demand, an equilibrium where (in the central example above) all the researchers are vigorously competing for prestigious publications and using up all available grant money in the course of doing so.  There’s no free energy anywhere in the system.” – Eliezer Yudkowsky, Chapter 2: An Equilibrium of No Free Energy, Page 31

“Then why don’t you just walk up to the decision-maker and tell them about the bias?  Because they wouldn’t have any way of knowing to trust you rather than the other five hundred people trying to influence their decisions?  Well, in that case, you’re holding information that they can’t learn from you!  So that’s an “asymmetric information problem,” in much the same way that it’s an asymmetric information problem when you’re trying to sell a used car and you know it doesn’t have any mechanical problems, but you have no way of reliably conveying this knowledge to the buyer because for all they know you could be lying.” – Eliezer Yudkowsky, Chapter 3: Moloch’s Toolbox, Page 42

“This brings me to the single most obvious notion that correct contrarians grasp, and that people who have vastly overestimated their own competence don’t realize: it takes far less work to identify the correct expert in a preexisting dispute between experts, than to make an original contribution to any field that is remotely healthy.” – Eliezer Yudkowsky, Chapter 4: Living in an Inadequate World, Page 106


Is This An Overview?

In an efficient market, in an efficient civilization, the individual cannot do better than the collective power of the many who have a lot more available information.  Even if the individual has information that others do not, the individual cannot make an improvement, gain any benefits by fixing the problem, and cannot exploit the system.  Common problems within adequate systems are supposed to be resolved by the community, as good ideas have already been tried by the community.  The collective might not get the exact answer, but no individual can predict the average value of the error, the average value of the change.   

Alternatively, there are inadequate systems in which individuals can do better that the community, as problems exist but do not get resolved.  Civilization gets stuck with inadequate equilibria as they are systemically unfixable.  There are various reasons for how an inadequate system, an inadequate civilization can develop. 

Central decision makers can prevent others from fixing the problem.  Decisions makers are not the beneficiaries.  There is asymmetric information as decision makers cannot know what or whose information to trust.  Systems might be inadequate, but that does not make them exploitable as there are many competitors trying to benefit from available opportunities, a competitive equilibrium.  To improve the system would require large scale coordination action, but they are difficult to facilitate.


How To And Not To Think About Inadequate Systems?

Wrong guesses and false cynicism do exist.  Different systems are dysfunctional in different ways.  No individual is better at everything, but individuals can be better at somethings and worse at others.  There is a lot of variation in expert views.

Although there are inadequate systems, just assuming inadequacy can make people see inadequacy in everything with a lot of arguments.  Concluding inadequacy from a problem is not an adequate rule.  Even though systems have inadequate equilibria, a blanket distrust of inadequacy arguments does not get far.  Civilization cannot be beat all the time, but its good to be skeptical and check for inadequacy. 



The explanations can be improved.  The organizational quality is mixed.  There are practical examples and abstract reasoning.  The abstract reasoning and conversations can become confusing.  There are parts that would be better understood with prerequisite knowledge.   

This book is based on the dichotomy of perfect and imperfect information theory, an improvement on them.  Tailored to reduce the strictness of perfect information.  

Questions to Consider while Reading the Book

•What is the raison d’etre of the book?  For what purpose did the author write the book?  Why do people read this book?
•What are some limitations of the book?
•To whom would you suggest this book?
•What are efficient markets / civilizations?
•What makes markets / civilizations inadequate? 
•How do civilizations get stuck in an inadequate system?
•How can inadequate systems be fixed?
•What are the benefits or consequences of decision makers trying to change the system?
•Can systems be exploited?
•How to think about being better than civilizational results? 
•What is Moloch’s Toolbox?
•Why blame Moloch?
•What is Nash equilibrium?
•How do people tell the epistemic standards of others?
•How much effort does it take to resolve a civilization inadequacy problem? 
•What is Pareto-optimal?
•What does the price represent? 
•How can SAD be cured?  How did the author approach SAD ideas?
•Why are babies dying due to nutrition problems?
•What do scientist do?  What is the purpose of subclasses of scientists, the suggesters and replicators? 
•What research gets promoted? 
•What is a two-factor market?
•What is the value of a degree?
•Why and how to certify people?
•What do people want from medicine? 
•What are wasted votes?
•Which entrepreneurs do venture capitalists take?
•Why not use Velcro for shoes?
•Which candidates do newspapers follow?
•What is the Overton window?
•How do political decisions change?
•Do people trust the newspapers? 
•Should you defer to doctors? 
•What is modest epistemology? 
•Is there a problem with theoreticism?  How does theoreticism contrast with empiricism? 
•Is it better to be a hedgehog or a fox? 
•When to test a product?
•Does majority belief makes something true?
•What is status regulation?
•What is the typical mind fallacy? 

Book Details
Publisher:               Machine Intelligence Research Institute
Edition ISBN:         9781939311191
Pages to read:          162
Publication:             2017
1st Edition:              2017
Format:                    eBook 

Ratings out of 5:
Readability    4
Content          4
Overall          3