THE BOOK--Playing The Percentages In Baseball

Tuesday, December 29, 2009

Mike Silva Chronicles - Part 1: UZR

As you may remember these past few days, I rightfully accused Mike Silva of farting his opinion on sabermetrics.  He made his summary opinions without evidence, or in other words, bullsh!t.  I called him out to ask questions and gather evidence, before having an opinion.  He said he’d send me questions, and he did.  Ten of them. They are excellent questions, each one deserving of a comprehensive answer.  Even though most, if not all, of you think of Mike as a troll or a tool, at the very least, he is offering us a window into how he thinks, and based on his questions, I would say he is representative of those fans who are not knee deep into sabermetrics.  So, I say that the ten questions that he posed me, fairly long and involved, is a way for him to show sabermetric redemption.  At the moment, I have no choice than to see Mike as two people: the gasbag that we’ve come to ignore and dismiss, and the inquisitive person who has dipped his toe in sabermetrics, and wants to learn more.  And I have to believe that the gasbag is all for show, a character he plays, much like Stephen Colbert plays a character called Stephen Colbert.

Therefore, this will be the first thread in a series of ten.  I sincerly thank Mike for allowing the comments on his blog to be so unmoderated, and for giving me a chance to answer these questions.  Take it away Mike:

You admitted in the comments section at Inside the Book that UZR plots are subjective. People say it takes 3 years worth of UZR data to establish trends.  Why would anyone want to give this more than a cursory look, especially when a great defender, like Mark Teixeira, ranks so low? I have seen other outstanding defenders display “up and downâ€? performances. Doesn’t this tell you there are huge holes in this metric and perhaps old school scouting would better be served in defensive evaluations?

To refresh people’s minds, this is what I said:

If you are asking how can we use…
- the location of the batted ball,
- the trajectory of the batted ball,
- the speed of the batted ball,
- whether the batter was LH or RH,
- whether the pitcher has a tendency to give up FB or not,
- what park the player is in,
- the base/out situation
...I’ll ask: how can we NOT use that information?

So, yes, some things are subjective.  And some are not.  The batter being a lefty or righty, the park he plays in, the base/out situation are objective, they require no interpretation.  And, sometimes the bias in the subjective parts of the recording is so huge that Andruw Jones shows a 100 run difference based on whether BIS or STATS is recording the data, over a period of 7 years.  That’s ridiculously huge.  So, yes, we have bias issues to contend with.  There is an uncertainty level.

And yes, you also need more data with fielding than you do with hitting.  Roughly speaking, 200 PA as a batter (say 50 games) tells you as much as 400 balls in play (BIP) as a fielder (say 100 games).  So, if you can make a judgement on a hitter’s batting stats after one year, you can have that same level of uncertainty using UZR after 2 years.

As for giving more than a cursory look, if you look at the UZR leaders for more than one season, you will see how well it actually confirms the observers.  As I noted when the Teixeira issue became mainstream

Best-fielding 1B over the last 3 years
By Tangotiger, 03:06 PM

As of today:

+27 Pujols
+16 Kotchman
+16 Youkilis
+12 Helton
...
-10 Giambi
-11 Nomar
-18 Jacobs
-19 Sexson
-21 Fielder

That’s UZR, and that’s their totals.  You can do it as per 150 games if you like, and you’ll see Giambi at -8 per season, and the top fielders at around +5 or +7 per season.  Whatever, it’s not that important.  You look at the names in the best/worst list, and really, what is there to complain about?  The Fans’ eyes pretty much agree.  Joining those top 4 guys are Derrick Lee, Adrian Gonzalez, Mark Teixeira, Lyle Overbay, Lance Berkman.  The important point is this: the 4 fielders with highest UZR were also in the top 10 among the Fans’ picks.

Not only that, but Giambi, Jacobs, Fielder, and Sexson were 4 of the 8 worst fielding 1B last year according to the Fans.  Talk about two completely independent systems matching up.

Maybe UZR isn’t as high on Teixeira as it should be or as the Fans have him, but it’s one of those misses that we simply have to deal with.  It doesn’t invalidate the entire UZR methodology.  It is hard to find those guys that are highly or lowly ranked in UZR who don’t deserve to be there.  And please, don’t quote 4 months of data.  Don’t talk to me unless you are talking about at least two years of data.

So, every now and then, UZR misses one.  We all see Teixeira.  He’s played on several teams already (Angels, Braves, Rangers, Yankees), and the Fans of each think he’s a well above-average fielder.  Unless he’s happening to hoodwink everyone, I’m fairly comfortable calling him an above-average fielder, and that UZR is missing something on him.  As with every thing, a metric might miss every now and then.  Such is life when dealing with samples and uncertainties, and less than ideal data recording systems.  But overall, UZR is a net plus.  It adds value. 

The alternative is what?  To rely on scouts.  But, who are these scouts, and who’s recording their thoughts in a systematic manner?  All we get is some reporter cherry picking some scout’s observation to fit whatever the reporters wants to say.  If a reporter really wanted to, he would find the one scout that thinks Teixeira is an average fielder. 

Now, I offer the alternative of doing something systematic with the Fans’ Scouting Report:
http://tangotiger.net/scout/index6.php?prim_fld_cd=3

One could conceivably say that UZR is more accurate, but UZR has more huge misses than the Fans.  And therefore, rather than taking the chance on some impossible to justify UZR value, that you might as well rely solely on the Fans (or some other SYSTEMATIC collection of opinions).  I would not be in favor of that, as it’s better to have more data than less data.  What I have to dismiss is the cherry-picked scouting data.  What I would include if it was available, is all the scouting data of a group of pro scouts.  And I think most saberists would do so as well.

All data provides value, as long as you can pick out the biases.