From the Ground Up

Despite living in Canada, I always keep an eye on what’s going on south of the 49th. I’ve been finding the run up to the 2008 presidential election to be very interesting, both in the ongoing primary battle between Obama and Clinton and the larger battles between the Republicans and Democrats. One site that I’ve found particularly interesting is www.fivethirtyeight.com. This is a site primarily geared towards pollster info, primary election predictions, and the gritty details of the US electoral college. The reason I find it so engaging is because the main writer for the site, Poblano, has developed a vast and complex model to predict election outcomes. He draws data from a wide variety of sources, including polls and results for primaries, senate races, and past presidential races, as well as census data for ethnic breakdowns, per capita income, demographics, religious affiliations and other trends.

The result is a very interesting model that accounts for and explains data in a much more comprehensive way than you can get from TV and radio pundits. It’s not 100% accurate, but I can certainly appreciate the scientific principles and math behind it. I’m sympathetic to the challenges of modeling a complex system. Seeking to explain the facts is always more difficult than simply presenting them.

538.com
538.com FAQ on the model
Detailed explanation of pollster weighting

The last month or so has found me hard at work modeling the results of my thesis experiments. For the most part I’ve been doing this in MS Excel with its associated VBA suite. While I’ve made some very large strides in my understanding of both of these programs, I still regularly want to bang my head against the wall. I’m an engineer, not a computer programmer. As such on a number of occasions I’ve spent several days on a problem only to find that someone else has already solved it far more thoroughly than I could hope. In the interest of preventing damage to the walls and heads of other modelers, here’s a few things I’ve learned recently.

(more…)