sarkology

conceptual vs. predictive models

Posted in Uncategorized by sarkology on May 24, 2010

It is generally acknowledged that models involve a trade-off between tractability and faithfulness. The more accurate your model is, the more unwieldy it gets. The more you try to make it easier to handle by simplifying, the more you lose in terms of faithfulness with respect to the phenomena being modeled.

That is true for predictive models, but not necessarily for conceptual models.

In predictive models, the more factors you take into account that influence the outcome, the better your predictions get. But in conceptual models, we aren’t after predictions. We build conceptual models to gain insight into phenomena. The more factors we add the less clear what each factor is doing to affect the outcome. We want to understand, for instance, how X affects Y, all else being equal. This will gives us insight into how X and Y are related to each other. This of course, is not help at all in prediction. In the real world, X and Y aren’t the only things that are going to be varying. Unlike in conceptual modeling, in prediction we already know how X affects Y, so we want to take that for granted, and include it in our model to improve its predictive accuracy.

There have been concerns raised in computational neuroscience about how modeling the brain with ever powerful computers is no help at all. This is because as the models get more and more faithful, they become as hard to understand as the brain itself. This leads us to how simulation can help provide understanding.

There is no intrinsic scale to phenomena. Even so, it is helpful for us to impose such scales in our effort to understand them. It is useful, say, to think of bodies as composed of organs being composed of cells composed of organelles. This is possible when there is ‘scale separation‘ in the phenomena. Consider a phenomena with 3 levels. Scale separation exists when modeling Level 3 phenomena, we can take Level 2 phenomena as basic, instead of also having to include Level 1 phenomena. Say, when considering what happens to to the organs (Level 3) when its cells undergo mass cell deaths (Level 2), we don’t have to consider the behavior of organelles (Level 1). This is not the case, for example, in turbulence. There is no hope of summing up the behavior of eddies to predict the behavior of the flow they contribute to, without also considering fluid behavior at even smaller scales.

Simulation can provide understanding when we want to analyze a system at a specific level (say, Level 3), by modeling Level 2 elements, without bothering with the details of Level 1. But, if we can those at Level 3 only depend on Level 2 phenomena, why do we need to bother with simulating Level 1 at all? Aren’t we back to dealing with the good old conceptual model where we can just choose to vary some Level 2 variables and watch what happens? Most of the time, this is wouldn’t be realistic. Scale separation is never complete, some influence still ‘leaks’ from lower levels up to higher levels. If we exclude them from a simulation, we risk changing the behavior of the system at higher levels. So we need to start from Level 1, not because we want accurate predictions at Level 4, but because we want to look out for interesting phenomena at Level 3.

An example: We simulate neurons (Level 1), which form a modular network (Level 3) containing certain specialized sub-networks (Level 2). Now this has no pretensions whatsoever to realistically portray what is going on in the brain. What we are trying to do here is observing how neurons might give rise to emergent phenomena at Level 3, even though we already have a rough idea of how sub-networks contribute to the behavior of the whole network.

In a conceptual model, we isolate key variables, and/or hold others constant. In a simulation, we let the computer take care of low level phenomena, so that we can pay attention to higher level ones. Both share the common strategy of only allowing enough detail to advance understanding.

Prediction has a different strategy. Ideally the best way to predict is to stick with what you know for sure (say, particle interactions), and just crunch the numbers. But this is often infeasible due to computational resource limitations. In prediction, we include higher level phenomena as computational shortcuts. We sacrifice potential micro-accuracy for computational efficiency, hoping that this does not affect macro accuracy too much. This is done for example in modeling planets as point masses in celestial mechanics.

But wasn’t I talking about how in a predictive model you want to include as many factors as possible? And that this would increase accuracy? Yes, because a computational shortcut usually doesn’t coexist within the predictive model with lower level features. Lower level features are usually just too expensive to bother being dealt with. So what we have instead is a bunch of computational shortcuts modeling the phenomena at a higher level. Of course, when you do this you neglect subtler effects that manage to leak up from lower levels. Which leads to the strategy of adding more computational shortcuts to capture/compensate for the limitations of existing shortcuts.

Summary: Conceptual models include details to advance understanding. Predictive models include details because, having oversimplified to save on computational resources, it has to include sub-models of higher level phenomena in order to reattain predictive accuracy.

Advertisements

One Response

Subscribe to comments with RSS.

  1. Michael Bishop said, on June 7, 2010 at 8:29 pm

    Great post! I wrote some sorta related posts at my blog last fall: http://permut.wordpress.com/tag/model/


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: