Small Samples and the Trap of Modern Basketball Conclusions

In the data room of the newsroom, I am usually the last to turn off the lights. That night, I stayed until nearly two in the morning just to check one number that had sent the entire feed into a frenzy. A young player had averaged 24 points over his last seven games, and the headlines had already crowned him the future star of the league. I reopened all the footage, rebuilt the positional heat maps, and recalculated his efficiency by shot type. The real story looked very different: most of those points came from situations where the league average conversion rate sits at roughly one in three. Seven games are not the truth. Seven games are only seven games.

That is the first lesson anyone working in basketball data analysis must learn, and it is also the easiest to forget when the season heats up. We live in an age where every metric is available, every chart can be drawn in seconds, and every conclusion can be produced faster than the flight of a three-point shot. That very convenience creates a trap: it makes us believe that having data means having the truth.

I used to think that way. Years ago, as a young data editor, I wrote that a team deserved a heavy win rather than a lucky one, based only on expected goals and possession share. I was right about that match. But I was wrong about how I reached the conclusion, because I believed a good metric could replace the entire context. Only years later did I understand that a metric is only valuable when we know exactly what it fails to measure.

Basketball is a sport of discrete event chains. Each game has hundreds of possessions, each possession has dozens of decisions, and each decision depends on a chain of variables no model fully captures. A made or missed three-pointer can hinge on a few millimetres of wrist angle. When we aggregate those random events into a single number and call it the truth, we are performing a statistical miracle that even statisticians treat with caution.

The problem lies in the sample size. A season has 82 games, yet many of the things we want to measure happen only across a few dozen situations. How many three-pointers does a player attempt while tightly contested? How many times does a defence find itself in transition after a turnover? Those samples are usually too small to separate signal from noise. When the sample is small, a lucky streak looks exactly like a real skill, and an unlucky streak looks exactly like a decline.

That is why I set a discipline for myself: never judge a player or a team without at least three advanced metrics, and never conclude from a sample under ten games. That discipline has saved me many times from dazzling headlines that later collapsed. But it also taught me the opposite: there are moments when the crowd is right, and my model is the blind one.

In the summer of 2026, when the entire league had to move into a centralized isolation zone in Orlando because of the pandemic, every model based on home-court advantage that I had built over years suddenly became meaningless. No fans, no roaring crowd, no road pressure. When the arena was empty, my model collapsed. I realized I had forgotten the human factor. Teams considered weaker played on equal footing, and teams that relied on the power of their home crowd lost a not-insignificant edge. The data was not wrong, but it was only true in the world that produced it.

The same thing happened again in another major tournament, where I confidently predicted a strong team would advance from the group stage simply because it held the highest cumulative attacking metric. That team was eliminated. In hindsight, I realized my model was missing data on the opponent's pressing intensity, a variable outside the dataset I had collected before the tournament. That failure cost me weeks of morale, but it also forced me to build a system that integrates more unconventional data sources.

Since then, every analysis of mine includes a section many colleagues consider redundant: risks and gaps. In it, I list what the model cannot measure — injuries, psychology, a congested schedule, the chemistry between players. That section does not weaken the article. On the contrary, it makes readers trust it more, because they know I am being honest about my own limits.

But I also do not want to fall into the opposite trap: using humility as an excuse never to conclude anything. There are moments when the data is dense enough to speak with certainty. When a player sustains a three-point efficiency far above average across more than two hundred attempts, that is no longer luck. When a team holds the league's top defensive rating for three straight months, that is no longer coincidence. The line between signal and noise is not about whether we believe in the number, but about the sample size and the number of controlled variables.

This is where I frequently clash with my own colleagues. The media loves emotional stories, while I am pulled toward the spreadsheet. But I have learned that the two are not mutually exclusive. A dry metric, placed correctly, can tell a more emotional story than any commentary. Conversely, an emotional story without data backing will collapse the moment the next game is played.

I do not believe in gut feeling. But I do believe in what gut feeling confirmed by data tells me. The intuition of someone who has watched thousands of games is a form of uncoded data. The point is that we must know how to test it, not sanctify it. When intuition and numbers agree, that is when we can be most confident. When they conflict, that is when we must work hardest.

In the transfer market, the small-sample trap is far more dangerous. A young player who shines for half a season can be priced like an entire career. Teams pay for potential based on a sample too small to verify. I have written many times that a contract is only truly right when the number signs alongside the signature, meaning when on-court performance confirms the price on paper. Until then, every number is merely an expensive hypothesis.

What is worrying is that this valuation bubble does not deflate on its own. It only waits for one disappointing season to trigger a whole chain of consequences: a locked salary cap, narrowed opportunities for other players, and eroded fan trust. Data can see it coming if we read it honestly, instead of reading it the way we want to.

So every time a number excites me, I ask myself one simple question: what is the sample size? If the answer is a few games, I close the spreadsheet and go to sleep. If the answer is a few hundred situations, then I allow myself to write. That is not timidity. It is the only way an analyst avoids deceiving himself.

Numbers never need us to defend them. Rather, we need them so we do not deceive ourselves. But numbers also never tell the whole truth. They only tell the part of the truth we have learned to measure. The rest — the fatigue in a player's eyes in the fourth quarter, a coach's sigh after a loss, the moment a whole team understands each other without speaking — remains outside every spreadsheet. My job is to try to measure that part another way, not to pretend it does not exist.

Looking ahead, I believe basketball analytics will grow ever stronger, but also ever more humble. When every team has data, the advantage no longer lies in owning the numbers, but in knowing which numbers are trustworthy and which are just noise. The winner in the coming seasons will not be the one with the most data, but the one who asks the right questions of their data.

And perhaps that is what I want to say to anyone holding a spreadsheet: do not ask what the number says. Ask how many observations produced it, by whom, and in what context. The answers to those questions are what separate an analyst from someone who merely reads headlines.