Speed Ratings by Reference (Tully Runners Article)
[back to Home Page] [back to Articles Page]
Reference Runners and Cross Country Speed Ratings
by Bill Meylan (Tully Runners Webmaster)
(draft: July 5, 2002)
This article describes the "Reference Runner" method of determining speed ratings. I should have written this article before writing the "Graphical" method description because the graphical method was confusing to many readers. The whole point of deriving "speed ratings" is to see how fast runners are running relative to each other ... when you determine how fast runners are running relative to each other, you can sort them into a list ... this is exactly the same as "track & field" leaderboards where the runners are listed from fastest to slowest in each event. In XC, these lists are usually called "rankings" (not the best term) ... my "ranking lists" are simply sorted lists of how fast runners have run races relative to each other (that's all they are).
"You can't rank cross country" ... I have been told this by some very good runners and coaches whom I respect. I agree with them in a generic sense ... but part of the confusion is my fault - I haven't explained what I'm doing well enough ... I will try to explain it better here. I know some people will still disagree with the concept of an "XC leaderboard", but I have converted a few critics when I explained it to them one-on-one. Many coaches are already familiar with the "Reference Runner" method because they use it themselves to evaluate their own teams.
Every team ranks themselves. By this I mean every team has a #1 runner, a #2 runner, a #3 runner and so on ... this is determined by where they finish races relative to each other. Occasionally, runners are close in ability and they switch positions, but in general, runners on the same team usually finish in the same order in most races. So sorting them by finishing position is easy ... since each finishing position has a time, sorting them by race time yields the same result.
Coaches evaluate their own runners by their finishing positions and times relative to each other. Some coaches track the time difference between their #1 and #5 runners (it's even listed in some results) and some coaches also track the time difference between each runner ... for example, they may know from experience their #1 runner usually beats their #2 runner by 30 seconds and their #2 runner usually beats the #3 runner by 15 seconds and so on ... by knowing the "typical" time difference between each runner, a coach can determine which runners had good, average or bad races. Each runner is used as a "reference" to each other runner ... the time differences can indicate which runners are improving.
Here is a simple example that occurs ... the #1 runner usually beats the #2 runner by 30-40 seconds, but in today's race, #1 beat #2 by only 10 seconds ... There are three possible explanations, (1) the #2 runner ran a big race and improved, (2) the #1 runner had an off-day and ran slower, or (3) the #2 runner ran somewhat faster and #1 ran somewhat slower. How do you know which possibility is correct?? ... Answer: by examining how all other runners in the race finished the race relative to each other because most runners are reasonably consistent ... if most other runners finished 20-30 seconds closer to the #1 runner, then it is most likely the #1 runner just ran slower than normal.
The actual final race times are not important ... what matters is the time difference between runners.
For now, let's consider only a single team as a "reference" ... as a real life example, the table below shows two races for the Beaver River girls:
Tully Invite Baldwinsville Invite Final Time difference
(9-29-02) (10-6-02) between Tully & Bville
-------------- -------------------- ----------------------
Lauren Moser 19:32 18:05 19:32 - 18:05 = 1:27
Julie Emery 19:42 (+0:10) 18:29 (+0:24) 19:42 - 18:29 = 1:13
Adrienne Gagnier 20:15 (+0:43) 18:48 (+0:43) 20:15 - 18:48 = 1:27
Kendra Chamberlain 21:15 (+1:43) 19:41 (+1:36) 21:15 - 19:41 = 1:34
Kisa Ruiz 21:43 (+2:11) 20:09 (+2:04) 21:43 - 20:09 = 1:32
** the time in parentheses is the time difference between
the #1 runner and each other runner
|
The time differences between each Beaver River runner are easily calculated (this can be done for any team) ... how fast they ran each race "relative to each other" is simply the time differences between runners. The time differences between the Beaver River runners was fairly consistent ... the one noticeable discrepancy occurs for Julie Emery who was 14-21 seconds slower at Baldwinsville compared to the other four runners ... the final time differences between Tully and Baldwinsville also show the discrepancy ... the conclusion is apparent (Julie simply had a slower race at Baldwinsville and the other runners had fairly typical races).
Now, extending this concept to more than one team ... The complete results of any single race can be used to determine the time difference between any individual and any other individual in the race ... The complete results of any single race is already a sorted list "ranking" the runners in that race by time. In addition to their own team, some coaches follow other teams (usually a rival) and keep track of finishing times and time differences "relative" to his (or her) team ... this is easily done when teams race in the same race ... if they race in different races on the same day at the same course, most coaches will "merge" the times of the two races to make the comparison.
The next extension causes most of the controversy ... How fast is one race relative to another race at a different race course?? ... I make this calculation by two totally different methods ... the first method is the Graphical method previous described and the second is the Reference Runner method (I use the Reference Runner method more often because I have large database of results for individual runners). A simple example of the Reference Runner method is shown in the table above using the Beaver River girls ... look at the final time difference between the Tully and Baldwinsville races ... based on just these five runners, the Baldwinsville race was about 1:29-1:30 faster than the Tully race (I usually use a median difference (not the average) ... median is where most values occur) ... Note: for deriving my speed ratings, I use as many runners as possible from a race to determine the time difference between different races (not just runners from one team which may be insufficient) ... more data is better ... some teams will show overall team improvement compared to other teams, so I want as many team results as possible to see who is improving, and factor this into the speed ratings.
An important point: ... I am a crazy person
who has a computer 
... for
every race, I keep track of how fast every runner runs relative to every
other runner on the same day at the same course ... and then I extend it
(both separately and combined) with results from other race courses so I
can compare how fast any runner has run relative to any other runner on a
single day or throughout the season ... it is just keeping track of time
differences - that's all ... I convert time differences into a number
called a "speed rating".
The essential point - my speed ratings are just numbers that compare how fast runners have run relative to each other ... the actual final times and which race course are not the important factors ... the time difference between runners is the important factor. The Graphical and Reference Runner methods approximate how fast one race is compared to another race ... by adding or subtracting the approximate time difference allows me to make speed ratings for each runner in each race.
The Graphical method and the Reference Runner method are completely different because the Reference Runner method uses individual performances to determine "how fast one race was relative to another race" ... the Graphical method doesn't care about individuals - it considers everybody in a race as a single "group" and "groups" are compared, not individuals ... the Graphical method requires "groups" to be of approximate equal ability or you can not make a direct comparison ... this makes the Reference Runner method generally better when a sufficient database exists ... But in most cases, the two methods arrive at approximately the same answer.
Here is an example where the Graphical method fails in a direct comparison with my standard race course ... the Graphical method can not be used directly with the Section 2 (Class A) girls (especially at sectionals) ... the quality of the Class A girls in Section 2 is too high for a direct comparison (if you exclude them and just use Class B, C and D, the Graphical method works). Likewise, you can not directly compare graphs from the Section 3 sectionals and the NY State Meet ... the quality of the State Meet is higher ... experience has also shown that the slope intercept of the boy's State meet graph is typically 60 to 70 seconds faster than sectionals (this means that "average" runners at States are about 60 to 70 seconds faster than "average" runners at sectionals).
The hardest times to make good speed ratings are the first races of the season ... a database of recent time differences doesn't exist ... some runners enter the season completely fit and ready to race, while many others are "not as fit as they will be" in coming weeks, so the time differences can be large ... some estimation is required. But it gets easier as the season progresses and more data is available (re-adjustment of earlier speed ratings is sometimes required). Most runners are reasonably consistent in how fast they run races ... speed ratings can not be calculated without this consistency.
Major complaints of adjusting the time of one race compared to another race are this ... "It doesn't make any sense if one course is hard with lots of hills and the other course is flat and easy ... runners are different and run differently on different courses ... the pace of the races (how the races are run) may be very different and the final times could mean nothing". .... My response for the hard course versus easy course: some runners are better on hard courses (hills, etc), but it should be reflected in the time differences between them and others (and that's what I evaluate) .... How races are run is an interesting topic - this is really only important for the better above average and elite runners who have the ability to implement race tactics such as surges, negative splits, etc ... since they are only a small percentage of the runners in most races, they have little or no effect on calculating speed differences between races (the Graphical method ignores them totally) ... I do not like using (and almost never use) race times of elite runners such as Molly Huddle, Tracey Brauksieck or Nicole Blood to determine how fast different races compare ... elite runners are capable of large speed variations ranging from setting course records to "coasting" (saving energy for the next race) and still winning by big margins.
Speed ratings are not precise numbers ... they are approximations because they are derived from approximations ... they indicate how fast runners have run relative to each other (approximately) ... However, I believe they are very adequate in making XC leaderboards and evaluating the overall ability of teams.