The trouble with scouting

Since the publication of Moneyball, a book that challenged traditional scouting beliefs, there has been an extensive debate in the world of baseball about the role of scouting. As statistics curtail the more obvious need for scouts, traditionalists argue for the importance of scouting still, as a more accurate indicator of a player's true ability than bland statistics.
My basic stand on this issue is to argue on behalf of both. Statistics are, to my way of thinking, the primary tool, as I will explain later. More traditional scouting is still important, although I must point out that it is less so than it was in the past. Even over the past 15 years, statistics for college and high school players have become more easily accessible, rendering largely useless the most basic function of scouting, which was finding out who was good and who wasn't. Now you can tell the basic facts of a player without even leaving your seat.
However, statistics do not tell everything. I mean this in the traditional sense, yes, but also especially for high school players. Statistics are only really meaningful under certain circumstances. Paul DePodesta, in Moneyball, argues on behalf of college stats, because college players play more games against tougher competition than other amateur players. So stats from college players (and the minor leagues) are highly useful and predictive of a player's future performance.
Outside of college, though, is a different story. For high school players (and amateurs in Latin America and the Pacific Rim), there are no truly meaningful stats. Stats are useful, but not to the extent that they are in college and the minors. So scouting is necessary.
Not only that, scouting tells you what stats don't. This is true, of course, in the major league environment as well. Pro-scouting advocates believe that Moneyball scoffs at the idea of a player's "makeup." But this simply isn't true; the discussion in the book about whether a player is a "rockhead" or if he was caught selling porn in college could have occurred in any big league boardroom. It's simply a matter of how much importance is placed upon "makeup" and other aspects that scouting can tell you, but statistics don't.
Scouting is great at getting at the underlying aspects of a player's performance that stats don't reveal. I can look at a player's college stats and give you a basic rundown on what kind of player he is. But there are many very important facts I can't tell you: if he's a hitter, I can't tell you anything about his batting stance, I can't tell you anything about his methods of practice and preparation, and I can't give you any personal insight as to whether or not this player can make it in the big leagues. If he's a pitcher, I can't even tell you what type of pitches he throws. I can't tell you whether he's a lefty or a righty. I can't tell you how tall he is, or how often he works out, or whether he has a serious drug problem. These are all important bits of context that help you understand a player in his real-world environment.
But ...

The central problem with scouting as it exists today is that it tells you how good a player could be, rather than how good he realistically will be. Scouting-intensive organizations tend to draft athletic players with the potential to become superstars. Stats-intensive organizations tend to draft people who already are good players. What you end up with is, in the example of many failed traditionalist teams, is a bunch of very athletic people -- who can't play baseball. The discussion of Jeremy Brown in Moneyball is instructive. The scouting people obsess about the fact that Brown is out-of-shape, and a very poor athlete. Billy Beane could give a damn what kind of athlete Brown is or what kinds of tools he has. He knows that Jeremy Brown is already a very good hitter, and that is what matters.
It's so wonderful to see a player who is a natural athlete and dream about what he could become. It's also wonderful to buy a lottery ticket and dream about how you can spend $34 million. The reality is that the best way to find a good major league ballplayer is to try and find the best college and high school players and draft them. It is not a good idea to draft people who might become good in the future. And hey -- when these great athletes hit .220 in the majors, it's not the scout's fault -- it just wasn't meant to be.
Baseball, even more than other sports, shows the separation between pure athletic ability and the ability to succeed in a specific sport. Baseball is littered with the stories of great pure athletes who didn't succeed at playing baseball. Deion Sanders was supposed to be a great two-sport threat. And he was, as far as I know, really good at football. But Deion was not a good baseball player. His pure athletic ability didn't make him much of a hitter (336/413/268 lifetime). He was a lively baserunner, who stole a career-high 56 bases (with 13 CS) in 1997 with Cincinnati. Which makes him about as valuable as a really good backup outfielder. The funny thing is that people didn't really seem to realize that he wasn't a good baseball player. Many baseball observers and fans are scouts at heart, and are much better at recognizing pure athletes rather than determing who actually plays well.
The other 2-sport legend in baseball was Bo Jackson. Everyone felt that Bo would have been a legendary baseball player if he hadn't gotten injured. I did too, until I looked at what Bo actually did -- not how he looked, but what he did. Bo's first full season was with Kansas City in 1987. He hit 22 HR and even stole 10 bases. But did you know that his hitting line was 296/455/235? In 1988 Bo moved up to 27 SB, but hit a still-unimpressive 287/472/246. For his career, Bo hit 309/474/250. He stole 82 bases and was caught 32 times (for a fair 72% success rate). So the great Bo Jackson was actually about as legendary as Dave Kingman. With more stolen bases. Bo hit homers, some of which were hit long distances, and he was pretty fast, as well. But he was an all-or-nothing hitter; he hit some home runs -- but that's it. If you're going to have a sub-.300 OBP and hit below .240, you have to hit about 40 or 50 homers to be really valuable. Bo did not.
And did I mention strikeouts? Bo really was a new Dave Kingman. He played 4 seasons of 100 or more games and struck out more than 100 times in 5 seasons. Wait -- Bo had more 100-strikeout seasons than he had 100-game seasons? Yep. His 3 most-prolific offensive seasons (1987-1989) saw him strike out 158, 146, and 172 times. Bo was a classic example of a great athlete who wasn't able to refine those athletic skills into a more complex discipline, like hitting a baseball.
A historic example of the same player would be Jim Thorpe. Thorpe was the famous Olympian who was considered to be the world's greatest athlete; a multi-talented athletic wonder. Thorpe played major league baseball for parts of 6 seasons from 1913-1919 -- mostly with the New York Giants. And he didn't hit very well at all -- a 286/362/252 lifetime hitter that was poor even for the dead-ball era. (It's worth pointing out, as someone whose name escapes me did before, that Thorpe got progressively better as his career wore on.)

So what exactly am I trying to say? I'll boil it down into bullet points:

1. It's always more valuable to look at what a player has done, rather than what you think he might do. A good bet in the draft is not someone who could learn to play, but someone who already has. This means that a lot of "lottery tickets" will have to be ignored, disappointing many scouts and other hopeless romantics.
2. The separation of pure athletic ability and baseball ability is not insignificant. The presence of great athletes who failed at baseball (Neon Deion, Bo Jackson) as well as non-athletes who became baseball stars (Babe Ruth, David Wells, John Kruk) makes baseball an even more specialized discipline than other sports (imagine Ruth or Wells on a basketball court). Obviously, a great athlete is more likely to succeed in baseball than a non-athlete. But this is a very broad generalization. Pure athleticism is not, in and of itself, an argument in favor of someone's baseball ability.
3. Scouting is a necessary part of the game, but only as a supplement to a basic understanding of baseball, of which statistics are our most important tool. The two can co-exist (as in Boston), but only when realism -- not traditionalism and mindless perpetuation of the past -- is the guiding light.