Friday, November 10, 2006

Wow. Even Tradesports got that wrong

Now that the Virginia Senate race between George "Macaca" Allen and James "Thunder Thighs" Webb is finally decided, the Democrats have taken control of not just the U.S. House of Representatives, but also the Senate. When you do the political math, this is pretty amazing. Thanks to Republican House Whip Tom DeLay, Congressional districts have become so highly gerrymandered that the 2004 election had barely 30 to 40 truly competitive seats, out of 435. Over the past several elections, House incumbents have tended to be reelected 95 percent of the time -- with a record 99 percent of incumbents winning in 2002.

Senate seats aren't able to be gerrymandered, of course. Nonetheless, incumbents historically win 90 percent of Senate races. Since only 33 out of 100 Senate seats were up for election this week and the Democrats needed to win 6 of those to take control, the historical odds were not good. Even if the Democrats took all the Senate seats with no incumbents (Maryland, Minnesota, Tennessee, and Vermont), that would have given them one additional seat (Tennessee, which in fact has remained Republican).* Under a "normal" election cycle, we could then expect the 90 percent incumbent victory trend to mean the Democrats would have picked up 1.4 seats from the 14 Republican incumbents facing reelection, while losing 1.5 seats to the Republican. In other words, even if Senators are rarely, if ever, actually physically partitioned, we would still expect Democratic gains to be offset by nearly identical losses.

Instead, six incumbent Republican senators lost their reelection bids -- a 57 percent reelection ratio, versus the 100 percent reelection ratio for the 15 Democrats facing reelection. While those in the financial industry are fond of saying "past performance is no guarantee of future performance," this is still amazing. So amazing that the "futures" market Tradesports had been giving odds in the 70 to 80 percent range throughout October that the Republicans would maintain control of the Senate. Since such markets are supposed to reflect the "wisdom of crowds", it might be a good time to reflect that markets still aren't crystal balls.

What are my predictions? I'm glad you asked!

First, both major political parties in the United States tend to draw their leaders from their most senior members. Most of these members got to be senior by coming from very "safe" states or Congressional districts. And safe states and districts, in practice, tend to be very strongly liberal or conservative -- in other words, statistical outliers in a country that for the past several elections has been evenly divided between Republicans and Democrats.

Political party leaders are not stupid. Well, some of them aren't. Still, by temperament and necessity they reflect the voters who elected them. This means that the voters who elected the Democratic Party leaders probably are not very similar to the voters who this past Tuesday brought the Democratic Party to power in Congress. This means the Democrats now face the very real threat the Republicans faced in 1994; that the less moderate faction of the party will set Congress' agenda for the next two years, against the wishes of the majority of "swing" voters who just put the party in power. The litmus test to see whether this is happening will be the reactions of those Democrats seriously vying to be President in 2008 -- Hillary Clinton, Barack Obama, John Edwards, and a few others. Most of these potential candidates have not been in office as long or come from as safe seats as Nancy Pelosi, David Obey, Barney Frank, or Charlie Rangel. If we see these presidential contenders begin rebelling, it will be a sure sign that the new Democratic Congress is heading in an unpopular direction.

* While Jeffords in Vermont was nominally an Independent, he caucused with the Democrats.

1 comment:

SmoothB said...

A couple things. First, the markets got most (if not all) of the actual races right -- even the Webb race. The problem here is that the bets on whether the Democrats would take the Senate seem to have assumed that the odds on each race were independent of the odds on others, which may have been a dubious assumption.

Second, if I say that there's a 65% chance that the Republicans keep the Senate, and they don't, that doesn't mean my prediction was wrong. Things with 35% chances of happening happen fairly often. In fact, even if Tradesports was perfect, then 5% of their predictions will be rejected at the 5% level, right?

Third, (and now I'm switching teams and going to defend democracy of all things), it doesn't mean anything to say incumbents usually win. We should expect that. Does Lance Armstrong demonstrate that the Tour de France isn't competitive? Does Microsoft demonstrate that the computer industry isn't competitive? So why should "the advantages of incumbency" tell you politics isn't competitive?