Recipe for Reductionist Analytic Modeling

The reductionist analytic modeling recipe goes like this:

  1. Pick a system to study, choosing a boundary around a region in the world and mentally sectioning it off - everything inside that boundary is the system and everything outside is the environment.
  2. Simplify the system by picking a few details to pay attention to, ignoring everything else. Divide the system into components with defined states and define interactions between components. Some parts have defined interactions that reach through the boundary defined in step 1, and interact with the environment.
  3. Assemble those parts back into the whole, building up complexity by computing how the various combinations of states of the various parts interact with each other.

Step 2 simplifies while step 3 complexifies. Step 2 is semantic while step 3 is syntactical. If the results from step 3 successfully predict the behavior of the system, then the simplifications in step 2 are said to capture the essential details of the problem.

By thinking of the work accomplished by analysis as due to these three types of work, you could find the limiting reagent and spend more time on that. The analytic engine you are improving could be personal, collective, flow or even batch-oriented.

The nature of thought

Your strength as a rationalist is your ability to be more confused by fiction than by reality.  If you are equally good at explaining any outcome, you have zero knowledge.

-- Eliezer

Logic is equally valid in all universes, and hence cannot tell you which of those universes you are in. Applying logic does not give you new information, it merely aligns your decisions better to the information you already have, by either reducing bias or increasing the efficiency of your estimators.

It is good to distinguish when you are getting new information and when you are processing existing information better, because knowing explicitly how the validity of your ideas flows allows you to make corrections faster and easier.

Long tails: a semi-technical explanation

Long tails in distributions are troublesome for 2 reasons:

  1. They are hard to test for empirically because they represent rare events. How they look like in any given model is more model-driven than data-driven.
  2. Models which are modular, and construct the distribution of interest from many independent component distributions, tend to underestimate long tails in the distribution of interest. This is a problem of degree, not a black and white issue - theoretical proofs use absolutely independent component distributions, and using those proofs for real work requires an assessment of whether components are independent enough in reality. That assessment is non-trivial and all too often skipped.