Problems with Data Analysis

As an engineer, acquiring and analyzing data has always been a big part of my job. I’m always amazed at how the same basic mistakes are made over and over again by otherwise intelligent people. Having witnessed all of these mistakes many times, I shouldn’t be surprised when I see the exact same mistakes being made by State fans on internet message boards. Here are a few examples of these mistakes:

Ignoring Data That Is Not Understood
After the spring game, there was a good deal of discussion about Bobby Washington’s rushing yardage and what conclusions should be drawn. Was Bobby going to be the man? Were there problems brewing with the defense? Of course, anyone who questioned State’s defense was soundly trounced from all sides.

These discussions immediately sprang to my mind when Barrington Edwards ripped off 62 yards on the second play of the UNC game. For the first two games this year, UNC averaged less than 60 yards net rushing for the entire game…..and that was bested on just two plays against the Pack. (I wish I knew where the defense that played against FSU was during the early part of the season.)

LESSON LEARNED: Never discard data just because you don’t understand it. Sometimes things not immediately understood become much clearer once more data is obtained.

Drawing Conclusions with Insufficient Data
My least favorite posts and/or threads are the ones that start with “Why isn’t ______ playing more? I would guess that at least 99% of the time, a player isn’t playing because:

– The guy starting is practicing better.
– Undisclosed medical issues
– Undisclosed discipline issues

What are the odds of a coach intentionally keeping a better player on the bench? The coaches see the players at every practice since they first stepped foot on campus and the fans only get to see the players during the games. Can we give the coaches just a little credit and quit bringing up the same stupid discussions over and over again?

LESSONS LEARNED: Know when you don’t have enough information to start drawing conclusions. If the information is available, then make some reasonable effort to get it. If you don’t have any facts, then exactly what are you talking about?

Shooting the Messenger
Unpopular opinions are often attacked without even bothering to provide evidence to rebut the claim. After the GT game, someone started a thread on the premium board at Pack Pride with the title, “What is Everyone so Happy About?� The poster explained that he was happy with the win, but the problems evidenced during the UNC game were still evident and the team wasn’t playing well enough to win many more games.

Many people lined up to jump all over the poster. At the same time, there was a poll started to predict what State’s final record would be. When I last looked at the poll, 581 votes were cast. 59% of the votes were predicting that State would finish at 7-4 or better. The poster that recognized problems was obviously far more accurate than his critics.

LESSON LEARNED: Sometimes unpopular evaluations turn out to be accurate. Offering or rebutting evaluations should always be supported with facts….not happy thoughts.

Lies, Damn Lies, and Statistics (or Looking for Chicken Salad in Chicken %$#!)
How many times over the last 12 months was the passing yardage from the 2004 season used to draw conclusions about the 2005 season…without bothering to include the fact that there were more interceptions thrown than TD passes? It should come as no surprise that the predictions that often accompanied this statistical travesty did not turn out to be even close to the truth.

LESSON LEARNED: The only statistic that matters is the final score. All other statistics should be used to explain the outcome, not to spin the observed results into some favorable light.

Ignoring Data That You Don’t Like (or Assuming the Ostrich Position)
My hopes and expectations for the 2005 football season were flushed during the UNC game. Note that I said during….not after. Long before the clock wound down to 00:00, State’s performance against its biggest rival convinced me that this was going to be a long year. The offensive line couldn’t protect the QB or open even a small crack for the running backs. The defense couldn’t stop an anemic rushing attack. When you throw in where UNC was expected to finish in the ACC, all of these things added up to some very unpleasant conclusions.

However, after nearly every win, opinions and projections soared. I’ve already mentioned how nearly 60% of the posters taking a survey expected a 7-4 record (or better) after the GT win. After the FSU win, many posters were expecting to roll over BC. In nearly every case, the problems and deficiencies of the team were ignored in favor of concentrating on one “positive� fact.

LESSON LEARNED: Ignoring facts that you don’t like will not improve them or make them go away. Concentrating on only the “happy stuff� does not provide the entire picture. Include all available data before making projections about future performance.

CONCLUSION
As everyone knows, “fan� is short for fanatic. The term implies raw emotion and energy, not cold, calculating logic. There is absolutely nothing wrong with screaming wildly for your team from the opening kickoff to the last tick of the clock. Fans are supposed to be happy when their team wins and sad when they lose. However, fans should be able to discuss their favorite teams like adults, not like children that can’t handle the truth. No one has a crystal ball that foretells the future…..but ignoring or misusing the information that is available will usually lead to poor projections.

About VaWolf82

Engineer living in Central Va. and senior curmudgeon amongst SFN authors One wife, two kids, one dog, four vehicles on insurance, and four phones on cell plan...looking forward to empty nest status. Graduated 1982

General NCS Basketball NCS Football

20 Responses to Problems with Data Analysis

  1. Alpha Wolf 12/05/2005 at 8:04 PM #

    An excellent analysis, VaWolf.

    One of my least favorite paper-thin analyses is that Marcus Stone’s success is attributable solely to the two big games that Dre’ Brown had. Maybe so — or maybe not. People tend to forget that the two players faced defenses that were “selling out” at times, and not at others. Stone was able to complete long passes (one for a TD) against Wake Forest because Wake had stuffed the box and were leaving their safeties on an island, and when they did, Stone burned them. Result? For a while, anyway, the defense had to play a more conventional defense that also slowed passing, and that in turn allowed the running backs a better chance to find open space and yardage. The same thing happened against Southern Miss, and to an extent, against Maryland in the second half.

    In other words, taking advantage of one defense led to a vulnerability when the defense subsequently adjusted. And, vice versa.

    What I am trying to say is that in that argument, too many relevant facts are ignored, and the issue is muddled by oversimplification. If one is going to make an argument for a cause and effect, they must include all of the causes and all of the effects for their line of logic to hold water.

    Anyway, thanks for writing that piece. I really enjoyed it.

  2. VaWolf82 12/05/2005 at 10:54 PM #

    Thanks for the kind words. You just mentioned one mistake that I should have included……oversimplification. In the engineering world, this is most often practiced by managers who won’t take the time (or don’t have the intelligence) to understand a complicated issue. In the sports world, it is practiced by fans who:
    – Want to blame every loss on one person (player or coach)
    – Want to find a quick fix for the problems
    – Don’t have a clue what they are talking about

    Another thing that I don’t like about message boards is the juvenile and idiotic meltdown threads after every loss. What is the point? Maybe I should have worked that in under the title of “Over-Reaction”.

  3. Cardiac95 12/06/2005 at 10:13 AM #

    Good piece! And its also a great way to explain Six Sigma Data Analysis to the non-technical managers many of us work with.

  4. site admin 12/06/2005 at 1:28 PM #

    best blog article I have read in a long, long time

  5. Class of 04 12/06/2005 at 4:31 PM #

    VaWolf82…. Not sure if you are overly pessimistic or just a beacon of realism. Possibly both. In any case, nice write-up. I hope this is added to “Absolute & Recent Best”.

    The lack of comments thus far is perplexing to me. Possibly it’s too difficult for people to get their minds around?

  6. SaccoV 12/06/2005 at 5:17 PM #

    Nice piece. Although I’m probably one of the ones who rambles on too long and doesn’t look enough at the data available, I’m savvy enough to read along and follow the logic of those who do. It is sad when emotion becomes too involved in our discussions but emotion is exactly what arises when we try to wrap our brains around an inconceivable loss or a well-played game against a tough team that we couldn’t pull out.

  7. VaWolf82 12/06/2005 at 8:52 PM #

    Once during a performance appraisal that was rapidly heading south, my manager told me that I needed to be more positive. I told him that I didn’t understand what he was talking about, because I am always positive that I am right when I speak out about something (being intentionally obtuse). He clarified his original remark by saying that I was too pessimistic. I told him that I didn’t think so…..Chicken Little was pessimistic as evidenced by taking one piece of data and drawing the worst possible conclusion. I asked him to give me an example where I drew a conclusion without sufficient evidence to support my position. Of course, he had no examples. (This whole episode was the clue that I needed to find a new job….and I transferred fairly quickly.)

    I don’t think that I am negative….just realistic. I have been extremely disappointed with the level of discussion on NC State message boards. I am afraid that the Great Herb Debate has polarized the fan base such that intelligent and civil discourse is nearly impossible about any basketball topic. On football topics, the optimistically delusional haven’t learned anything since they were predicting a BCS bowl during PR’s senior year. I don’t know how to change it, but at least it gives me something to write about.

    Thanks for all your kind comments.

  8. Trout 12/06/2005 at 9:49 PM #

    Wow. What a great read. Kudos to VaWolf for writing it, and kudos to statefansnation for bringing it to us.

  9. JeremyHyatt 12/07/2005 at 12:52 AM #

    I just hope that in his career at NC State Cedric Simmons will be as good Bryant Feggins and Damon Thornton (hold up which was shot and which was arrested several times?) Oh and i heard Kevin Thompson is having an good career in Europe, that a$*&*%&!

  10. Rick 12/07/2005 at 9:15 AM #

    “I don’t think that I am negative….just realistic. I have been extremely disappointed with the level of discussion on NC State message boards.”

    That makes two of us.

    Good read.

  11. Alpha Wolf 12/07/2005 at 11:17 AM #

    The problem with message boards is that anyone at all, regardless of their age, their knowledge or their emotional stability is allowed to participate in a discussion, often ruining it. Heck, usually ruining it.

    Over at Pack Pride, the premium forums were the escape from that, and there you could red/participate in rational discussions from long-term fans who usually knew a thing or two about the sport and NC State’s history in it. Then, inevitably, the cretins started arriving last year and I cancelled my subscription, because dang if I was going to pay for the same crap I could read for free. Maybe that has changed, don’t know, because I subsribed instead to the Wolfpacker which has a lot tighter leash on their board and it is lower traffic anyway. Plus, to be honest, I prefer their writers to Pack Pride’s.

    Bottom line is, I don’t think you will ever get a reasonable discussion group going on a free discussion board, because there is always going to be rotten apples spoiling the barrels. They happen from time to time on the PP free forum, but after about three pages, it sinks into the usual sophmoric rants. I feel sorry for their mods, considering that they are tasked with herding cats.

  12. Trout 12/07/2005 at 1:13 PM #

    The moderators at PackPride suck big time.

  13. VaWolf82 12/07/2005 at 1:15 PM #

    ^ That’s known as sarcasm for those that are not PP regulars.

  14. choppack 12/07/2005 at 8:39 PM #

    VaWolf – Good post.

    I do think it’s a huge expectation to expect message board fans to be reasonable and assess their team’s performance or lack thereof. Heck, I don’t see how they can analyze data when some of them can’t grasp basic truths. Part of the problem is our current situation.

    We have 2 coaches who haven’t won the big one and constantly come short of our hopes and dreams. But yet, we don’t ever outright suck – so we’ve always got some hope.

  15. SaccoV 12/07/2005 at 10:11 PM #

    Although you’re right for the current status, both Amato and Sendek are not newcomers; each has had his own way for several seasons (including at least four seasons for recruiting purposes), and not sucking is just as bad as sucking after four or five seasons have gone by. All we have to show for us staying the course is a lot of refracted lens imagery. To some, both coaches look good (the media, those outside the fan base)–to the knowledgeable fan, both coaches appear to be mirror images of one another. One won’t speak, the other won’t shut up. Neither has produced to the fans expectations. All things equal, most other colleges would have made some changes but we haven’t; thus, we sit idly by hoping for the best when our hopes have dashed so vehemently in the past. Hopes are mere glimmers after these last six years.

  16. Class of '74 12/08/2005 at 7:05 AM #

    Should we go ahead and order two gold watches now?

  17. BJD95 12/08/2005 at 9:55 AM #

    I was out of town when this came out, but it is a fantastic article. Like VaWolf, I get chagrined when people confuse “realism” for “pessimism.” Generally speaking, there’s no poster/author on NC State sports that I agree with more often than VaWolf. His posts/articles always make you think.

  18. Alpha Wolf 12/08/2005 at 10:08 AM #

    I’ve always wanted to start an NC State message board that was by invitation only — but with little moderation other than to kick someone out of they were a jerk. In other words, the only people that could post would be those that actually had a brain.

    This site does a good job of that, so the idea is likely moot.

  19. Waxhaw 12/09/2005 at 3:35 PM #

    Great read!

    Every preseason (football and basketball), I am reminded of the infamous thread several years ago: “Where would our second team finish?”. Only State fans could have several pages of replies predicting their second team could finish in the middle of the ACC when their first team wasn’t even making the NCAA’s.

    A splash of realism is very welcome.

Trackbacks/Pingbacks

  1. StateFans Nation » Blog Archive » What Went Wrong? - 04/16/2007

    ObeUbL Thank you for your blog.Really looking forward to read more. Want more.

Leave a Reply