Genre = Philosophy, Epistemology
There is no way of knowing what works best in any given situation. No matter the outcome of an experiment, there is always the possibility that there are exceptions to the outcome. If the outcome shows an indication of a cause-and-effect relationship, it is possible that it was only applicable to a certain situational environment. If the outcome is a failure, it is possible that there are certain situations in which it can be true. This book asks for humility in our knowledge base, as what we know can be proven to be wrong given time.
For Manzi, the start of the scientific method was with Francis Bacon. Bacon made two observations which narrate the development of science. The observations include that the complexity of nature is greater than mental capacities of individuals to understand the complexity, and people find more patterns in data then there are. The scientific method changed the way in which the search for knowledge is carried out. Opponents of the scientific method are the Scholastic philosophers who sought knowledge from logic and debate without seeking new information or testing the theories.
The scientific method uses trial-and-error by holding as many potential causes constant and observing changes to the outcome by changing one potential cause. The evidence is inductive and can help with generalizations but with no certainty as to whether or not they are correct. Every generalization can have a variety of hidden conditionalities, creating the Problem of Induction. Generalizations are easier when there is low causal density such as biology. Unfortunately, social sciences have high causal density which means practical generalization are extremely rare. To find more of the conditionalities, experiments should be numerous and be varied.
Paradigms are priorly accepted generalizations. Paradigms are the status quo of assumptions. Scientists use paradigms to conduct theories and interpret data without the need to reconstruct every basic assumption. New information and observations which contradict the paradigm do not necessarily change the paradigm immediately. Over time, the paradigms do change and update to be inclusive of all available information. Practically, as Manzi puts it, paradigms are temporary systems of framework to aid in the construction of the new paradigms. New paradigms take over when the guardians of old paradigms retire or die, while new paradigms gain recruits. Paradigms need to be flexible to be able to progress knowledge while preventing endless questions of basic assumptions.
Trial-and-error is currently the best methods for the progression of knowledge and can be applied to private and public issues no matter the size. Making decisions about how society is to be organized should be embedded with incentives to experiment. Experimentation will lead any institution into continually searching for better organizational methods. The experimental knowledge obtained may not lead to immediate applications, but successive iterations of experimentation and practical applications interact with each other to produce further ideas.
There are two problems within the framework of this book which include limiting the history of knowledge and logic. The general assumption in this book is that knowledge will progress, while the method of progression is key. History shows that knowledge can be destroyed via destruction of libraries or certain paradigms removing reality from theories like the Scholastic philosopher. This brings up the second problem which is limiting the discussion on logic. The way the trial-and-error experiments are built are based on logic and theories. Manzi makes great effort to express the importance of Randomized Field Tests (RFTs), but their construction is based on logic. Trial-and-error can help filter information but it is not independent of logic.
The major lesson of this book is to not to presume that assumptions of knowledge are correct. Testing theories obtains necessary, but not sufficient, information to decide how current sets of decisions and institutions can be made better. As there are varied causal reasons for every generalization, no single test can prove or disprove knowledge. Manzi mantra to keep seekers of knowledge humble can be summarized as: “The model is never the system”.
Pages to read: 269
1st Edition: 2012
Ratings out of 5: