OFR Analytics: Who Qualifies as MLB’s “Ace” Pitchers?

“Ace” is a term thrown around a lot in baseball discussions. But what defines an ace? And who are the current aces in MLB? Join me as I define the term and objectively determine which pitchers rightly deserve the title.

Defining an Ace

Every team wants an ace to lead its staff. Looking back from a historical perspective, it’s easy to name a few players who qualify. For instance, in the 1990’s, there was the Braves trio of Greg Maddux, Tom Glavine, and John Smoltz leading the world in pitching.  Others, such as Pedro Martinez, Randy Johnson, and Roger Clemens, made low scoring games as exciting as offensive displays. When they faced each other, the atmosphere was electric.

Perspective helps us to see some obvious pitchers who stood above the rest. Without that perspective, though, the lines are much more blurred. The “ace” moniker is thrown around too flippantly for current pitchers. Regularly in Atlanta Braves discussions, people will refer to Julio Teheran as the “ace” of the Braves staff. Teheran is a good pitcher, but he’s not one of the elite pitchers in baseball.

So what vaults a pitcher from “really good pitcher” to “ace”?

Stuart Wallace took an analytical approach at Beyond the Box Score to define an ace, but it included having three above average pitches, lots of swinging strikes, and going deep into games. Are these skills often possessed by an ace? Sure. But I question if these should be required skills. It’s nice to have a lot of strikeouts or a third awesome pitch, but if a guy is getting a lot of outs on ground balls or can control the game with just two great pitches, I don’t think that precludes him from being considered an ace. An ace, in my opinion, is based on outcome, not a standard skill set. It doesn’t matter how you dominate the game as long as you dominate the game.

Matt Snyder argued a couple of years ago that James Shields was an ace because an ace is a top 30 starter in baseball and he couldn’t think of 29 starting pitchers who were better. It sounds nice to say there should be a certain number of aces, and that number matches the number of teams in Major League Baseball, but does the 30^th pitcher automatically qualify as an ace? To answer that question, let me ask you another: With even the small bit of perspective we have since Snyder made that claim, does James Shields come up in discussions of aces?

In 2016 the Diamondbacks Robbie Ray was the 30^th best pitcher by fWAR and the best pitcher on the team. Would you call him an ace with his 4.90 ERA/3.76 FIP? That’s a hard no.  The “ace” label is too lofty to be used so loosely.

Others are even more restrictive, such as Jacob Shafer of Bleacher Report, who included in his criteria that the pitcher must be “unequivocally the best pitcher on his team.” But think this through for a moment and it quickly falls apart. If this is true, Tom Glavine and John Smoltz would automatically be disqualified as aces because for a good portion of the 1990’s they were pitching in the same rotation as Greg Maddux. Glavine and Smoltz were the sixth and seventh best pitchers of the 1990s by fWAR, but Maddux was the top fWAR pitcher of the decade. Let’s take it a step further, was Maddux even “unequivocally the best” enough to be considered an ace? Smoltz and Glavine were neck-and-neck with him through the second half of the decade, after all. Consider another example, Randy Johnson is one of the best pitchers of all time. In 2002, he was the second best pitcher in baseball. But the first best pitcher was his teammate, Curt Shilling. So was Johnson not an ace that year? These questions are absurd to even ask, which makes this definition of ace just as absurd.

After research and reflection, there are three key criteria that determine if a pitcher can rightly be called “ace”:

1. Dominating performance: Regardless of how the pitcher leverages his skill set, he shuts down the opponent.
2. Consistency over time: An ace isn’t dominant in some games; dominance is a distinguishing characteristic of his performance through multiple seasons.
3. Separation from the pack: An ace isn’t just a really good pitcher; his performance outcomes separate him from the pack. There are really good pitchers towards the front of the pack, and then there are aces whose performance separates them entirely from the pack.

What is an ace? When you chart pitching performance over a series of years, and then step back and look at the resulting blob, the aces are those one or two dots clearly separated from the rest of the blob.

Nailing Down the Methodology

Warning: The rest of this section is technical details. Feel free to skim your way down to actual results if not a data nerd like me, but make sure to catch a few of the important highlights so the results make sense.

Now we have a much more stringent and clear definition of “ace,” but it’s still subjective. How do we define “dominating”? How many years are required for that performance to be considered “consistent over time”? How are we determining if a player’s performance “separates him from the pack”?

When it comes down to it, there is only one variable that matters in determining if a pitcher is “dominating,” and that is run suppression. The pitcher who is exceptional at limiting runs is the pitcher who is exceptional at doing his job.

As for consistency, the ace label shouldn’t be used for a single season’s dominant performance. A real ace is not just exceptional one year, but sustains that excellence from year to year. To me, the best way to determine consistency over time is to take a set time frame, and then determine dominant performances in terms of run suppression for each individual year within that time frame. The pitcher gets credit for each year he qualifies as dominating. That way, the best pitchers in each year within that time frame will rise to the top by having multiple years of “dominance” rather than just one.

Also, the data should determine the number of aces rather than prescribing in advance a set number. Forget “top 30,” if the data finds no one stood above the pack, then there were zero aces that year. There should be no thoughts of, “Well, this guy’s not really an ace, but we need a fifth person/tenth person/thirtieth person.” That said, at the end of the analysis I’m going to go against what I just said and give a number: five. Why? Because it’s easy and fun to conceptualize an ace as the answer to the following question: If you could pick any five pitchers in baseball to form a starting rotation, which five pitchers would you pick? Just remember the number is not as important as the performance.

Now down to the nitty-gritty details. How, objectively, should “run suppression” be defined? This takes the actual data out of the realm of subjectivity. You can disagree with the definition of ace, but you can’t disagree with the pitchers labeled aces based on the definition.

To objectively determine the pitcher’s ability to suppress runs, there are a number of metrics to consider. There is also one obvious divide, and that is people in the “traditional” statistics camp and people who appreciate the nuance of the more advanced stats, even if harder to understand. If I go with something too newfangled, I might lose the group who seek the old ways, but if I go with those old-timey stats, the stat geeks will never stop going on about all the flaws of the old stats. To resolve this conflict, I did what any good researcher should do: I went with one traditional and one advanced, and we can compare lists at the end.

Here are the run suppression metrics we will use:

• Earned Run Average (ERA) – this is a traditional rate stat that – for better or worse – is a staple measure used to judge the ability of a pitcher to suppress runs. The issue with ERA is the many factors impacting runs scored that are included in ERA but are outside of the control of the pitcher, such as park factors and defense. It also doesn’t consider the role a pitcher plays in unearned runs. However, it is easy to understand, it is commonly understood by the masses, and it’s a reasonable measure of run suppression. Whatever hundreds of variables go into runs scoring, at the end of the day ERA tells us how many of the runs scored while Pitcher X was on the mound (if overlooking that a relief pitcher can allow runs that are charged against the starter after he leaves the mound).

• Skill-Interactive ERA (SIERA) – this is an advanced rate stat that is tracked by Fangraphs. See a more detailed explanation here. As a quick overview, SIERA doesn’t just consider how many runs were scored, but tries to determine how much the pitcher’s skill level impacted the runs scoring by considering factors such as strikeout rate, ground ball rate, and home run per fly ball rate. It also considers context, like park factors and if a home run was hit when leading by one run or when leading by 15 runs. SIERA is similar to, but more much more inclusive than, Fielding Independent Pitching (FIP), which only considers strikeouts, walks, and HR/FB rates. SIERA is not perfect, but it acts as a good counter for ERA because it attempts to consider a pitcher’s skill in run suppression rather than just number of earned runs scored. (I also like it because Fangraphs data is sortable and downloadable into excel, which makes for easier analysis.)

Both ERA and SIERA have weaknesses, but to an extent, the strength of each helps cover the weakness of the other.

Now that we have the statistical measures we will use for our key “dominance” variable of run suppression, we have a few other details to sort out before getting to the results.

First, some inclusion criteria. We need to define the pool of starting pitchers we will use to determine both the comparison average and the pitchers who set themselves apart from that average.

Here are the two inclusion criteria we will use:

• Limit the data to the last 10 years (2007 through 2016) to determine who in the last decade could be considered an ace. This time frame is arbitrary, and we might catch the last year of a great pitcher, but tough for him. He’ll just have to be considered an ace from the 10 years prior to this one. I’m sure this hypothetical person can handle that. I’m interested in current pitchers who can rightly wear the label.

• All pitchers who pitched at least 150 innings in a season were included. This is slightly more relaxed than “qualified” starters, but keeps the innings high enough to only include legitimate starters. We don’t want 2017 Bartolo Colon level collapses or replacement player fill-ins who don’t accumulate many innings muddying up the results and weighing down the mean. Because these are rate stats, we also don’t want flash in the pans to qualify with very low ERAs over very few innings. The bottom line is that we are looking for aces, a pitcher who is consistently dominant, and you shouldn’t even be in the discussion of elite starting pitchers if you pitch fewer than 150 innings in a given season.

There is one final question before we move on: How will we will objectively determine if a pitcher’s dominant run suppression separated him from the pack?

There are statistical tests that can be used, but I simplified the process by deciding if a pitcher’s ERA/SIERA was at least two standard deviations below the mean ERA/SIERA of all starting pitchers with at least 150 innings for that year, then they were exceptional run suppressors compared to their peers. Now let me explain that briefly. With a normal bell curve (and there was a slight skew for full disclosure, but it was pretty close), roughly 95% of the pitchers should fall within two standard deviations of the mean, which includes about 2.5% on the high end and 2.5% on the low end. That 2.5% of pitchers on the low end are far enough from the mean to say with some confidence that they have separated themselves from the pack. For a more recognizable example, this is the same threshold used to determine “genius” from an IQ test. By setting the threshold for ace so far from the mean, I limit it to those who truly distinguish themselves as dominant run suppressors compared to their peers.

Since each pitcher with 150 innings was duplicated for every season they reached that minimum threshold across the 10 years, every time they surpassed the two standard deviation below the mean threshold for either ERA or SIERA, they would get one point. The goal here is to reward pitchers who consistently exceed this lofty threshold as the real aces. If they exceeded the traditional and advanced metric to satisfy both groups in the same season, they get two points. At the end, all points will be summed to see who had the highest total out of the possible 20 points (10 years x 2 metrics = 20 possible points).

The Aces Emerge

After selecting pitchers who met the criteria, the total data set included 958 entries ranging from 84 starting pitchers in 2016 to 107 in 2011. Running a quick scatter plot graph by year, we see that there are some dots on the left side of each line that do separate themselves from the pack. These are the pitchers we are looking for. (Note: And now I’m curious about the dots on the right side of the graph representing pitchers who separated themselves for the wrong reasons, but that will have to wait for another day):

It’s nice that the plot shows some dots standing apart from all the overlapping dots, but these are just dots! What we really want to know is the dominant pitchers each dot represents! Let’s put some names on these dots.

WP DataTables WP DataTables

After running the data, a couple of things immediately pop out. First, there were a lot more who met the “dominance” threshold for SIERA (24) than there were for ERA (10). Second, there was not a lot of overlap between the two lists. Clayton Kershaw appears on both lists in the same year twice. That’s it.

Determining the Aces

Now for the point totals. How many times did each pitcher appear on each list?

Pitcher	ERA Points
Clayton Kershaw	3
Zach Greinke	2
Jake Arrieta	1
Chris Carpenter	1
Felix Hernandez	1
Kyle Hendricks	1
Jake Peavy	1

 

Pitcher	SIERA Points
Clayton Kershaw	3
Max Scherzer	3
Chris Sale	2
Roy Halladay	2
Felix Hernandez	2
Cliff Lee	2
Erik Bedard	1
Yu Darvish	1
Jose Fernandez	1
Zack Greinke	1
Matt Harvey	1
Corey Kluber	1
Tim Lincecum	1
Stephen Strasburg	1
Noah Syndergaard	1
Javier Vazquez	1

The take aways:

Clayton Kershaw is a stud. He is the ace of aces. He appears on both list three times: in 2011, 2014, and 2015 for SIERA, and in 2013, 2014, and 2015 for ERA. He would have a fourth appearance on both lists in 2016, but he fell exactly one inning short of the 150 inning threshold due to injury. His ERA of 1.69 and his SIERRA of 2.41 both would have qualified his performance as a dominant season. In fact, each would easily have led all pitchers.

SIERA likes Max Sherzer’s ability to dominate more than ERA does. While he doesn’t show up on the ERA list at all, he matches Kershaw on the SIERA list, appearing in 2012, 2015, and 2016.

ERA favors Zach Grienke, while SIERA favors Felix Hernandez and Chris Sale. Grienke made the ERA list twice (2009, 2015) but the SIERA list only once (2011). Sale did not make the ERA list, but twice appears on the SIERA list (2014, 2015). Hernandez appears on the ERA list in 2010, but the SIERA list for both 2013 and 2014. Unfortunately for Hernandez, his performance dropped off more recently.

One dominant season does not an ace make. Most of those making the lists once either struggled with injuries, or were one-hit wonders, or were simply really good pitchers who set themselves apart one year, but weren’t consistently head-and-shoulders above the rest often enough to earn the title of ace based on the my criteria requiring multiple seasons. These pitchers would include Cliff Lee, Chris Carpenter, Erik Bedard, Tim Lincecum, Javier Vazquez, Yu Darvish, Matt Harvey, Jake Arrieta, and the unfortunate loss of a young Jose Fernandez. Also, Roy Halladay and Jake Peavy might have earned more points if the inclusion dates were pushed further back in the past, but that is a question for another time as I’m more interested in active pitchers.

Some young guns might be emerging aces. Another category of more relevance to the discussion of current aces is those pitchers who might be emerging aces based on age and recent performances. I include Kyle Hendricks (27), Stephen Strasburg (28), and Noah Syndergaard (24) on this list. Corey Kluber is an interesting case, mostly because he’s a little older (31) and only has one appearance on the list (2014 SIERA), but he could potentially make that list again this year.

My Ace Rotation

Now is the time to finalize the list. Like I said earlier, I’m looking for pitchers with multiple years of dominant run suppression. It turns out, only five active pitchers had at least two appearances on the SIERA and ERA lists between 2007 and 2016, which makes it easy to answer my question from earlier: If you could pick any five pitchers in baseball to form a starting rotation, which five pitchers would you pick?

As determined by data showing these pitchers to be statistically ahead of the pack at suppressing runs for multiple seasons (with total appearances on ERA and SIERA lists in parenthesis) here is my rotation of current aces:

1. Clayton Kershaw (6)
2. Max Sherzer (3)
3. Zach Grienke (3)
4. Felix Hernandez (3)
5. Chris Sale (2)

In context of the question’s focus on current players, if you wanted to switch Hernandez or Grienke for one of the emerging pitchers, I wouldn’t argue, even though Grienke isn’t far off the pace for another dominant season in 2017. Sale, obviously, might be moving up from number 5 starter as well.

In fact, I plan to follow this article up with two shorter articles:

1. A look at who would make the list at the halfway point this year.
2. A different way of considering dominance based on SIERA and ERA that will draw in a few more pitchers by lowering the threshold slightly and “scoring” the results differently: Now available by clicking here.

So how would you answer the question based on this data and your own knowledge? If you could pick any five starters from the last ten years to build a rotation, who would they be? Let us know in the comments…