Flaws with the recruiting rankings

Skeptics and believers alike should read SMQ’s deep dive on recruiting rankings. [Relevant BC note -- we continue to outperform our recruiting rankings.] He concludes that "the rankings are a serviceable baseline for expectation." I am still in the nonbeliever club.

Although I think there is some value to following recruiting, I remain unemotional about the process. The payoffs are just too far away and there are too many outside factors that impact a program from signing day to the time a player hits the field.

As a public skeptic of the recruiting rankings, let me explain myself a bit. Talent matters. The team with the better talent will win the majority of the time. But I don’t think the rankings are a perfect (or even a very good) measure of talent or a good forecast for on the field results. The pillars of my arguments undermining the rankings follow. As always, let me know what you think in the comments section.


Recent history
The major sites use their consistently high rankings of programs like USC, Oklahoma and LSU as examples of how their system works. This boast ignores the actual timelines of these programs accents. UCS, Oklahoma and LSU all won championships and moved into or returned to elite status with respective coaching changes. Carroll, Stoops and Saban all won early in their tenures with less regarded talent. Once the programs established themselves their recruiting took on positive momentum and the recruiting sites followed. Now, nearly a decade later, it is easy to say “we [Rivals, Scout, Lemming] had USC class of 2003 ranked highly and it produced XX Pac 10 titles and XX first round draft picks.” These same sites never wipe the egg of their face when they could have said “our rankings of the late 90s USC classes never would have forecasted their first Championship under Carroll.” Florida provides another example of coaching versus recruiting rankings and how these sites can luck into their forecasts. Ron Zook was hailed by the recruiting sites for the classes he brought to Florida. His performance on the field left the Gator fans disappointed. Enter Urban Meyer. He wins a national championship in his second season. Recruiting sites boast: “see we knew Florida had talent!” Yet how do they explain Urban Meyer going undefeated at Utah with a bunch of two stars recruits? Or turning around a lightly-regarded Bowling Green earlier in his career? Coaching is the easy explanation. Unfortunately for Rivals and Scout, it doesn’t sell subscriptions. Recruiting news does.

Real life
This is something I didn’t appreciate until I had to make and witness personnel decisions in the business world. Although athletic evaluation is less subjective, when trying to fill roles or hire companies determine criteria for a position and then go and find people for those roles…just like sports (the subjectivity comes when placing a greater weight on personality versus tangible things like size and strength). Even the best companies make mistakes. I am sure anyone reading this who has hired someone has seen the perfect candidate not live up to expectations, while the guy or girl you had doubts about rises to the occasion. Heck, there is even something to be said of coaching someone into his or her role. I am sure readers have seen or developed people to become better than they were or outperform their peers who had better degrees or more experience, etc. Why should we expect things do be different in college sports? Bringing in the people with the best resumes in the business world is a start but doesn’t assure success. Bringing in the most sought after recruits in college football is a start but doesn’t assure success. It is about making them better and fitting the pieces together.

NFL
The NFL has a much more qualified applicant pool from which to draft. They have hundreds of hours of footage of likely draftees playing at a very high level. Collectively they are the best talent evaluators in the sport – yet they get things wrong too. It’s the nature of the process. If the best miss when identifying prospects 30% of the time, why should I expect talent evaluators from the national sites to get it right even 50% of the time. They are dealing with a much wider sample size and trying to forecast a process that has many more variables.

I think the rankings become a self-fulfilling process not a baseline. The sites try to stay ahead of the curve with the supposed measurable but there is a lot of luck, hot air and betting on the right programs and coaches. Any evaluation process is going to be filled with with hits and misses. What Tom Lemming or Mike Farrell thinks doesn’t really matter. What matters is that the coaching staff has a good eye for talent, knows how to sell kids on the school and style of play, makes them better once they get on campus and then gives them the plays to beat the guys across the line of scrimmage.