Sports commentatary is often bad, but when the topic dips into areas where people have trouble thinking straight in general it can get terrible. Anyone who gave a good or even average grade to the Bills trading up to get Sammy Watkins is, to my thinking, wrong. Part of it is discounting the future and the abstract, probably, but I’m not going to give the topic of what they’re thinking too much thought. That way lies madness. The best sports journalist grade I’ve seen is some version of “well he’s a great prospect and even though they gave up too much I think a C- gets the point across”. Worst is “A+, BOLD MOVE”. Where were they when I drunkenly hit on 17 playing blackjack in Vegas? Of course, many people who don’t give draft grades on network websites think the Bills bombed this one. That’s the idea I’m going to flesh out here.

Giving up Pick 9, Pick 109, and next year’s estimated Pick 16 (depending on how well the Bills do this year) is trading away an estimated 104 Weighted Career Approximate Value/CAV. Here are the wideouts with careers in or above that neighborhood drafted 1980-2006:

Name	CAV
Jerry Rice HOF	159
Marvin Harrison	124
Randy Moss	123
Terrell Owens	119
Reggie Wayne	112
Michael Irvin HOF	106
Tim Brown	104
Isaac Bruce	102
Torry Holt	100
Andre Reed HOF	98
Henry Ellard	96
Jimmy Smith	95
Art Monk HOF	93

It’s a short list. Another way to put it: The Bills are betting that Watkins, like Tim Brown, performs better than have 95% of wide receivers drafted in the first round from 1980-2006.

There are some caveats and clarifications. It’s generally slightly better to have one great player than two good ones (you can only play eleven at a time, after all), and just adding CAV numbers together ignores that. There is the merchandise/ticket sales/marketing of having an elite player on a team that doesn’t have many. And that’s about the end of the good news caveats. Somebody could claim that Watkins is the missing piece that will let them make a Super Bowl run this season, but for that a lot is going to have to go right for the Bills. Finally, I suppose it could be a decision that makes more sense in the short term to people who want to make sure they’re around for more than just the short term, though of course that’s not a great thing if you’re a fan.

The bad news is that they’re not just betting that Watkins is “as good as” Tim Brown was. They’re betting he is as good as Tim Brown AND has the same durability and productivity. More bad news: this draft had a lot of good wideouts. Twelve were taken in the first 64 picks / two rounds, which is the most taken going back to at least 1980. If you like the Bills’ move, you should also like a hypothetical trade for Watkins where you give up Odell Beckham Jr. AND next year’s round 1 pick AND this year’s round 4 pick.

The real bad news, though: even if Watkins IS Tim Brown take two, the Bills only broke even, here. It’s an awful pick.

The NFL draft starts this Thursday and ends sometime in 2015 I think, so I finally cranked out some graphics that I have been wanting to look at for awhile. This work depends on the player valuation that Doug Drinen came up with: Career Approximate Value/CAV. I should also mention that Kevin Meers did a pick valuation chart I found really illuminating here.

Data are from the 1980-2006 drafts. Average CAV starts falling off after 2006 because players haven’t been in the league long enough to assess their career (though you could fudge in a couple/few more recent years by scaling up the CAV of players who are still in the league if you had to). For most of these I dumped everything after pick# 250, so we’re looking at 7 rounds + a little into round 8 pre-1993. Data are from Pro football reference.

Finally, for some reason the vast majority of CBs and FS/SSs were all called “DBs” in the dataset. I’ll do a lookup and post an updated version of this for part 2.

Let’s start with what kind of Career Approximate Value you see by draft pick:

That QB at pick 199 is, of course, Tom Brady. The distribution is very bottom-heavy no matter where you slice it, but the first pick average CAV is ~67, your average pick CAV~=16, and your median pick CAV~=6.

Breaking that down by position:

Drinen wisely didn’t let any K or P or FB have a stratospheric CAV.  TE probably looks different in recent years/will look different going forward.  First thing that jumps out at me is that historically your best bet, late in the draft, are Guards and Centers, and as they’re similar positions the higher valley might not be a fluke.

So when have teams tended to pick which position?

This isn’t a great graphic, and a lot of it is “no duh” stuff.  Most of the action is in the first line segment (difference between picks 1-25 and 26-50), and this would be my summary:
-Positions that were disproportionately selected in early/mid round 1: QB, T, RB, DE
-Positions that were disproportionately NOT selected in early/mid round 1: LB, DB, G, C, TE
-Positions that were rarely selected early and slightly less rarely selected later in the draft: P, K, FB
-Positions that were selected steadily through draft: WR, DT
-I’m not sure if that plunge in late-round-7 flyers on LBs and DBs is due to a pre/post-1993 difference or what.

Let’s take a look at picks that boom (overachieve) or bust (underachieve). Here is a plot with the players at CAV “deciles” (e.g., 20% = better CAV than 20% of players at that pick and worse CAV than the other 80% at that pick), with lines fitted to each decile.

One could make a case for different definition of a bust pick or boom pick, but I went with:
Boom: Above the “70%” curve AND CAV AT LEAST 15 (Late in the draft less than 30% of picks are CAV>=15)
Bust: Below the “30%” curve OR CAV <=1. (Late in the draft more than 30% of picks are CAV <=1)
Middling: Everything else

Note that this means an average player picked in round 7 is a boom pick, while an average player picked in round 1 is a bust. The overall picture is the following:

Finally, let’s look at the proportion of boom/busts by draft pick window and position:

-QBs were the most prone to boom/bust (have the thinnest green ribbon), unsurprisingly.
-Early picks on TEs and RBs have been prone to bust.
-OLs drafted early (relatively) rarely bust.

Here’s a quick look at the quality of draft classes:

I knew 1983 was considered a great class. I did not know the year before was so bad! 2014-05-11 edit: it occurs to me that this class was disrupted by the strike, but for another take see the 538 piece pingpack in the comments below

For evaluating team draft performance it makes sense to consider how good their picks were relative to how good an average pick at those positions are to control for their draft position, whether they had traded picks away, etc.:

This is mostly a descriptive post but: there was no statistically significant effect of team on draft performance. (If I had asked ONLY “did Pittsburgh draft reliably better than other teams 1980-2006?”, the answer would be p~=0.01 “yes”, but with alpha inflation you can’t say this wasn’t a fluke because we’re looking at 32 teams). That said, if you are a Bengals fan you are probably not surprised that they are anchoring the scale, here.

The draft looks more like a crap shoot when you break it down by year:

Finally, let’s use that actual-expected CAV measure to see the biggest booms and busts drafted 1980-2006. LIST MODE (at least I’m just putting them in a table and not in a slideshow):
Biggest Booms

Rank	Name	Team	Year	Pick	Act-Exp CAV
1	Tom Brady	NWE	2000	199	130.81
2	Ray Lewis	BAL	1996	26	121.18
3	Brett Favre	ATL	1991	33	120.22
4	Jerry Rice HOF	SFO	1985	16	116.53
5	Dan Marino HOF	MIA	1983	27	107.63
6	Zach Thomas	MIA	1996	154	105.66
7	Derrick Brooks HOF	TAM	1995	28	103.07
8	Terrell Owens	SFO	1996	89	102.11
9	Tom Nalen	DEN	1994	218	96.57
10	Jason Taylor	MIA	1997	73	95.08
11	Rod Woodson HOF	PIT	1987	10	94.57
12	Will Shields	KAN	1993	74	92.35
13	Ronde Barber	TAM	1997	66	91.06
14	Mike Singletary HOF	CHI	1981	38	90.28
15	Michael Strahan HOF	NYG	1993	40	89.08
16	Richard Dent HOF	CHI	1983	203	88.96
17	Karl Mecklenburg	DEN	1983	310	88.5
18	Trent Green	SDG	1993	222	87.74
19	Peyton Manning	IND	1998	1	87.67
20	Emmitt Smith HOF	DAL	1990	17	87.01
21	Aeneas Williams HOF	CRD	1991	59	86.88
22	Bruce Matthews HOF	HOU	1983	9	86.06
23	Rickey Jackson HOF	NOR	1981	51	85.18
24	Hardy Nickerson	PIT	1987	122	84.09
25	Rich Gannon	NWE	1987	98	83.83
26	Drew Brees	SDG	2001	32	83.8
27	Mark Brunell	GNB	1993	118	83.61
28	Greg Lloyd	PIT	1987	150	83.46
29	Marvin Harrison	IND	1996	19	82.96
30	Randy Moss	MIN	1998	21	82.9
31	Clyde Simmons	PHI	1986	233	81.16
32	Kevin Greene	RAM	1985	113	80.99
33	Keenan McCardell	WAS	1991	326	80.84
34	Andre Reed HOF	BUF	1985	86	80.44
35	Curtis Martin HOF	NWE	1995	74	80.35
36	La'Roi Glover	RAI	1996	166	80.31
37	Thurman Thomas HOF	BUF	1988	40	80.08
38	Bruce Smith HOF	BUF	1985	1	79.67
39	Lawrence Taylor HOF	NYG	1981	2	78.43
40	Seth Joyner	PHI	1986	208	78.15
41	Steve McMichael	NWE	1980	73	78.08
42	Randall McDaniel HOF	MIN	1988	19	77.96
43	Earnest Byner	CLE	1984	280	77.73
44	Jared Allen	KAN	2004	126	77.57
45	Matt Hasselbeck	GNB	1998	187	77.38
46	Brian Dawkins	PHI	1996	61	76.52
47	Anthony Munoz HOF	CIN	1980	3	75.96
48	Reggie Wayne	IND	2001	30	75.94
49	Matt Birk	MIN	1998	173	75.74
50	Kevin Mawae	SEA	1994	36	75.47
51	LaDainian Tomlinson	SDG	2001	5	75
52	Marshall Faulk HOF	IND	1994	2	74.43
53	Pat Swilling	NOR	1986	60	74.2
54	Adam Timmerman	GNB	1995	230	74.05
55	Jesse Sapolu	SFO	1983	289	73.98
56	Randall Cunningham	PHI	1985	37	73.88
57	Boomer Esiason	CIN	1984	38	73.28
58	Alan Faneca	PIT	1998	26	73.18
59	Junior Seau	SDG	1990	5	73
60	Ray Brown	CRD	1986	201	72.88
61	Lance Briggs	CHI	2003	68	72.65
62	Warren Sapp HOF	TAM	1995	12	72.57
63	Shannon Sharpe HOF	DEN	1990	192	72.57
64	Jessie Armstead	NYG	1993	207	72.11
65	Ricky Watters	SFO	1991	45	72.01
66	Derrick Mason	TEN	1997	98	71.83
67	Mark Clayton	MIA	1983	223	71.78
68	Brian Urlacher	CHI	2000	9	71.06
69	Hines Ward	PIT	1998	92	70.73
70	John Elway HOF	BAL	1983	1	70.67

(It’s admittedly a little weird to have 1st overall picks be considered “biggest booms”)

Biggest Busts

Rank	Name	Team	Year	Pick	Exp-Act CAV
1	Ryan Leaf	SDG	1998	2	-57.57
2	Akili Smith	CIN	1999	3	-56.04
3	Ki-Jana Carter	CIN	1995	1	-55.33
4	Charles Rogers	DET	2003	2	-54.57
5	Steve Emtman	IND	1992	1	-54.33
6	Bruce Pickens	ATL	1991	3	-53.04
7	Art Schlichter	BAL	1982	4	-51.52
8	Heath Shuler	WAS	1994	3	-51.04
9	Mike Junkin	CLE	1987	5	-51
10	Rich Campbell	GNB	1981	6	-50.48
11	Kelly Stouffer	CRD	1987	6	-50.48
12	Trev Alberts	IND	1994	5	-50
13	Reggie Rogers	DET	1987	7	-49.96
14	Brian Jozwiak	KAN	1986	7	-47.96
15	Andre Ware	DET	1990	7	-45.96
16	Bo Jackson	TAM	1986	1	-45.33
17	Alonzo Highsmith	HOU	1987	3	-45.04
18	Andre Wadsworth	CRD	1998	3	-45.04
19	Kevin Allen	PHI	1985	9	-44.94
20	Jamal Reynolds	GNB	2001	10	-44.43
21	Curtis Enis	CHI	1998	5	-44
22	Mossy Cade	SDG	1984	6	-43.48
23	Troy Williamson	MIN	2005	7	-42.96
24	Brent Fullwood	GNB	1987	4	-42.52
25	David Klingler	CIN	1992	6	-42.48
26	Johnathan Sullivan	NOR	2003	6	-42.48
27	Aundray Bruce	ATL	1988	1	-42.33
28	Clyde Duncan	CRD	1984	17	-41.99
29	Huey Richardson	PIT	1991	15	-41.96
30	Shawn Knight	NOR	1987	11	-41.93
31	Lawrence Phillips	RAM	1996	6	-41.48
32	Yatil Green	MIA	1997	15	-40.96
33	Kenny Jackson	PHI	1984	4	-40.52
34	Dan McGwire	SEA	1991	16	-40.47
35	Leonard Coleman	IND	1984	8	-40.45
36	David Verser	CIN	1981	10	-40.43
37	Wendell Bryant	CRD	2002	12	-40.43
38	Rickey Dixon	CIN	1988	5	-40
39	Jerome McDougle	PHI	2003	15	-39.96
40	David Overstreet	MIA	1981	13	-39.94
41	Derek Brown	NYG	1992	14	-39.45
42	Courtney Brown	CLE	2000	1	-39.33
43	Gabe Rivera	PIT	1983	21	-39.1
44	David Pollack	CIN	2005	17	-38.99
45	Todd Blackledge	KAN	1983	7	-38.96
46	Blair Thomas	NYJ	1990	2	-38.57
47	Eric Kumerow	MIA	1988	16	-38.47
48	David Terrell	CHI	2001	8	-38.45
49	Tim Worley	PIT	1989	7	-37.96
50	John Clay	RAI	1987	15	-37.96
51	Ted Gregory	DEN	1988	26	-37.82
52	Reggie McGrew	SFO	1999	24	-37.72
53	Lam Jones	NYJ	1980	2	-37.57
54	Desmond Howard	WAS	1992	4	-37.52
55	Willie Scott	KAN	1981	14	-37.45
56	Cade McNown	CHI	1999	12	-37.43
57	Tim Couch	CLE	1999	1	-37.33
58	Billy Cannon	DAL	1984	25	-37.27
59	Clifford Charlton	CLE	1988	21	-37.1
60	Perry Tuttle	BUF	1982	19	-37.04
61	Tony Smith	ATL	1992	19	-37.04
62	Junior Miller	ATL	1980	7	-36.96
63	Ron Faurot	NYJ	1984	15	-36.96
64	Michael Booker	ATL	1997	11	-36.93
65	Jim Druckenmiller	SFO	1997	26	-36.82
66	Patrick Bates	RAI	1993	12	-36.43
67	Andre Johnson	WAS	1996	30	-36.06
68	Bob Buczkowski	RAI	1986	24	-35.72
69	Todd Marinovich	RAI	1991	24	-35.72
70	Leonard Renfro	PHI	1993	24	-35.72

(JaMarcus Russell was drafted in 2007, but unless he has an improbable comeback he will edge out Leaf by a few points, here, just because he was 1st pick overall)

Part 2 will look at the relationship between draft success, which teams bet on which positions, and team success (in terms of winning actual NFL games).