Rankings: Solutions

This is part 6 in a 6-part series.  Here are 1, 2, 3, 4 and 5.

In this last installment of Skyd’s series on the rankings and bid allocation, I’d like to propose a couple solutions to help make the system work a bit better.  As I have said before, I don’t think the rankings do a bad job of assigning bids, but things can always be better.

24!

March Madness is on my mind as I write.  The bracket just came out and a stack are sitting on my printer.  When I think about our Nationals, March Madness is something we should aspire to.  It is an incredible athletic event and each stage offers an incredible reward.  For some teams, it is just making the dance.  For the big time programs, the NCAA tournament is an assumption and the reward hinges on performance.  But right now at our Nationals, the dance floor is too small.  We need to expand the field to 24 teams.

As I argued yesterday, there is no ‘right’ order to rank the teams.  Because of this, wherever you draw the line will create injustice because the teams on either side of that line will have contradictory results.  I found Bryan’s analysis of Whitman, Tufts and Michigan State fascinating example of this twisted mess.  Pick almost any two teams ranked side-by-side and you can generate an argument about whether or not the order is correct.  Teams ranked between 16 and 24 are especially tricky because they have so many competing results.  Moving the line down to 24 teams means four more teams get in and there is that much less chance of a deserving team getting left out.

Another reason to expand the tournament is the relatively short shelf life of many college teams.  Joe Kershner and Arizona lived the dream of every small-time program, taking the world by storm and making it all the way to quarterfinals in 2008.  But for every Arizona, there is a Western Washington or UW-Eau Claire that almost, almost made it, only to fall short by a game.  With Nationals sitting at a mere 20 teams, there’s no room for Cinderella.

Well why not 26 or 30 or 32, then?  Over the years, I have had several conversations with Will Deaver about the number of teams at Nationals.  He has consistently maintained that increasing the number of teams would greatly tax the available resources and reduce the overall quality of the event.  (Remember, Nationals is still run almost entirely on the back of volunteers.)  24 is possible, though.  Every year in the past five, the production of Nationals has gotten bigger and better.  Keep production levels flat for a year or two and those resources could be dedicated to bringing more teams.  Format is another issue.  24-team and 32-team formats exist that preserve the existing structure of the tournament (4 pool play games into prequarters, stand alone finals, etc), but don’t really exist for tournaments from 25-31 teams.  So with 25-31 teams really awkward and 32 too big to be realistic, 24 is right in the sweet spot: practical expansion.

So go and email your USAU board member.

Tinker with the algorithm

Any changes to the algorithm need to be carefully examined and run through a number of simulations to make sure they aren’t going to create more problems than they solve.  With that in mind, I have two that I think would go a long way toward helping improve the rankings and how we deal with them.

Use a step-wise function to assign the time decay and divide the season into three tiers corresponding roughly to Jan, Feb and March.  The Jan tournaments should show significant decay in comparison to the later time tiers.  The first positive effect this would have is to make the workings of the ranking a bit easier to understand.  The second effect would be to help captains and coaches structure their season and team development.  If you knew that Jan tournaments were of very minimal weight they could be used to much greater effect for team and player development.

Void blow-out wins that hurt the winner and help the loser.  More than any other aspect of the rankings, these games (as discussed yesterday in the Corvegas Problem) bother people.  You can play great and your ranking suffers?!  You can get beat 0-15 and your ranking improves?!  Even beyond the emotional impact, do we really want a ranking system that takes these kinds of games into account?  No, we want to use games against roughly equal opponents to make decisions – that is were quality judgements can be made.

Steve Wang has made a couple mathematically oriented suggestions that have some real merit.  The first is to ‘smooth the curve’ so that the jump in ranking points more consistently matches the change in game scores.  Currently, the difference between 15-9 and 15-8 is 119 points, but from 15-14 to 15-11 is only 114.  Clearly, these two jumps are equivalent.  His second is to add a regression to the mean to limit the effect of a team with a small number of valid games.  This would be particularly important if some blow-out wins started to get voided.  A team could go to a couple easy tournaments, have a few games voided and their entire ranking would hinge on a handful of games.  Regression to the mean would add ‘extra games’ rated at 1000 (the average ranking).  This would pull everyone back toward the middle, but really pull those teams that had only played a few games.

All of these suggestions are quite technical in theory and implementation.  USAU should revive the defunct rankings committee and address these and any other suggestions in time for the 2013 season.

The Billy Burruss Proposal

After all the seriousness of these articles series, a fun little idea to end on.

Billy Burruss is my dad and since the idea was his, his name is on it.  Anyway, the other day he and I were talking about the rankings and bids and he mentioned that when he was rowing in high school and college, Nationals (at least back then) was open.  Why not, he suggested, have an open tournament the day or two days before Nationals with the winner taking the last bid.

I can see it.  Every team within driving distance dropping their $50 entry fee and getting a spot in a giant single elimination tournament to see who will be playing for real the next day.  How many would show?  Ten?  Twenty?  The atmosphere for the tournament, the expectation, the increased number of fans and spectators would be awesome, not to mention an incredible evening before Nationals as two teams battle it out for the chance to go to Nationals for real – a Nationals that will happen the next day!

And no one can complain they didn’t have a shot.

Feature photo by Andrew Davis