Tuesday, January 22, 2008

Flaws with the recruiting rankings

Skeptics and believers alike should read SMQ’s deep dive on recruiting rankings. [Relevant BC note -- we continue to outperform our recruiting rankings.] He concludes that "the rankings are a serviceable baseline for expectation." I am still in the nonbeliever club.

Although I think there is some value to following recruiting, I remain unemotional about the process. The payoffs are just too far away and there are too many outside factors that impact a program from signing day to the time a player hits the field.

As a public skeptic of the recruiting rankings, let me explain myself a bit. Talent matters. The team with the better talent will win the majority of the time. But I don’t think the rankings are a perfect (or even a very good) measure of talent or a good forecast for on the field results. The pillars of my arguments undermining the rankings follow. As always, let me know what you think in the comments section.

Recent history
The major sites use their consistently high rankings of programs like USC, Oklahoma and LSU as examples of how their system works. This boast ignores the actual timelines of these programs accents. UCS, Oklahoma and LSU all won championships and moved into or returned to elite status with respective coaching changes. Carroll, Stoops and Saban all won early in their tenures with less regarded talent. Once the programs established themselves their recruiting took on positive momentum and the recruiting sites followed. Now, nearly a decade later, it is easy to say “we [Rivals, Scout, Lemming] had USC class of 2003 ranked highly and it produced XX Pac 10 titles and XX first round draft picks.” These same sites never wipe the egg of their face when they could have said “our rankings of the late 90s USC classes never would have forecasted their first Championship under Carroll.” Florida provides another example of coaching versus recruiting rankings and how these sites can luck into their forecasts. Ron Zook was hailed by the recruiting sites for the classes he brought to Florida. His performance on the field left the Gator fans disappointed. Enter Urban Meyer. He wins a national championship in his second season. Recruiting sites boast: “see we knew Florida had talent!” Yet how do they explain Urban Meyer going undefeated at Utah with a bunch of two stars recruits? Or turning around a lightly-regarded Bowling Green earlier in his career? Coaching is the easy explanation. Unfortunately for Rivals and Scout, it doesn’t sell subscriptions. Recruiting news does.

Real life
This is something I didn’t appreciate until I had to make and witness personnel decisions in the business world. Although athletic evaluation is less subjective, when trying to fill roles or hire companies determine criteria for a position and then go and find people for those roles…just like sports (the subjectivity comes when placing a greater weight on personality versus tangible things like size and strength). Even the best companies make mistakes. I am sure anyone reading this who has hired someone has seen the perfect candidate not live up to expectations, while the guy or girl you had doubts about rises to the occasion. Heck, there is even something to be said of coaching someone into his or her role. I am sure readers have seen or developed people to become better than they were or outperform their peers who had better degrees or more experience, etc. Why should we expect things do be different in college sports? Bringing in the people with the best resumes in the business world is a start but doesn’t assure success. Bringing in the most sought after recruits in college football is a start but doesn’t assure success. It is about making them better and fitting the pieces together.

The NFL has a much more qualified applicant pool from which to draft. They have hundreds of hours of footage of likely draftees playing at a very high level. Collectively they are the best talent evaluators in the sport – yet they get things wrong too. It’s the nature of the process. If the best miss when identifying prospects 30% of the time, why should I expect talent evaluators from the national sites to get it right even 50% of the time. They are dealing with a much wider sample size and trying to forecast a process that has many more variables.

I think the rankings become a self-fulfilling process not a baseline. The sites try to stay ahead of the curve with the supposed measurable but there is a lot of luck, hot air and betting on the right programs and coaches. Any evaluation process is going to be filled with with hits and misses. What Tom Lemming or Mike Farrell thinks doesn’t really matter. What matters is that the coaching staff has a good eye for talent, knows how to sell kids on the school and style of play, makes them better once they get on campus and then gives them the plays to beat the guys across the line of scrimmage.


bceagle08 said...

Very well said... I was never skeptical of scout/rivals.com before I started following this blog but now I'm completely sold. It's always exciting to see the 4 and 5 star recruits coming to BC but a lot of the most productive players over the past few seasons were basically ignored by these sites. (Kiwanuka, Silva, Derrick Knight to name a few)

What I think is most encouraging is that we are recruiting kids in areas that we have had very little presence in the past. As much as I complained about TOB in the past he was great at uncovering talent from the midwest, new england, new jersey etc. Lets hope Jags and the staff continue this trend on a national scale.

Alan said...

I haven't been following the recruiting news for these very reasons, but frankly I'd be worried if Jags was going after only the top star recruits, regardless of if he got them to join or not. Basically I'd be worried they were looking at the same broken criteria as the recruiting sites.

BCDoubleEagle said...

I stopped paying attention to recruiting rankings when I learned that they often award an extra star to a player when he commits to a big time program -- as if the player's talent level suddenly increased overnight.

That being said, I think recruiting rankings can have some value in the sense that they can generate positive hype around the program. If we land lots of 4/5 star recruits, we look like a powerhouse and perhaps that grabs the interest of other 4/5 star recruits, and so forth.

I'm M said...

So...the media seems to going nuts over the fact that Ryan ditched the Senior bowl and he is being criticized pretty heavily. Do you think that Ryan could really hurt his draft stock by not playing? I would hate to see him end up like Brady Quinn.

Eagle in Brighton said...

What really would have hurt Matty would have been if he had played, and looked lackluster with "all that talent on the field."

Personally, you have all these CFB writers and pundits essentially in their offseason (ie: less to write/talk about) and latch onto these exhibitions as if legit competition to provide their empty content.

The only thing that matters are the opinions of scouts and GM's of the Falcon's, Dolphins, etc.

BCDoubleEagle said...

"What really would have hurt Matty would have been if he had played, and looked lackluster..."

Or worse, if he got injured.

Ryan proved himself in dozens of games over two years against top quality competition in some of the toughest venues in college football. No need for a meaningless exhibition game now.

Lally said...

A good take on the recruiting rankings system. It even has some fun digs at yukon in there!

I think the important thing to see with Jags' recruiting isn't so much that he's going after and landing 2 and 3 stars, but he's landing guys that Big 12, SEC and Big 10 schools are going after as well. If the "big boys" are interested in a player, I think it says more about his ability than any star rating assigned to him.

Eagle in Brighton said...

It is interesting to note how Colt Brennan and Woodson are getting picked apart at the pre-Senior Bowl practices...

Not much good can from that experience, unless you truly are an up-and-comer/underexposed.

Good for Ryan to not succumb to the whole dog and pony show phenomenon.