In a now famous research article from 1985, the Israeli psychologist Amos Tversky and two colleagues exploded the myth of the “hot streak.” When a basketball player sinks four or five baskets in a row, we think he has moved into a higher zone of concentration and achievement, but in fact those baskets are just a random series of events. Over a career of, say, 25,000 attempts, any player will, now and then, manage to drill four in a row simply as part of the normal fluctuation of thousands of imponderables­—from a mote of dust positioned in just the right way on the hoop to a slight gust of wind provided by another player’s swoosh. There is no such thing as a hot streak.

Tversky’s findings, as his fellow Israeli and long-time collaborator Daniel Kahneman tells it in his book Thinking Fast and Slow (2011), were not well received. Red Auerbach, the coach of the Boston Celtics who fancied himself capable of eyeing hot talent, dismissed them out of hand. Yet, Kahneman explained, human beings are psychologically wired to misperceive the world. Whenever we see a pattern, as with the player’s four successive baskets, we think there must be some underlying cause, even though there often is not.

To analyze the hot streak phenomenon rationally, Tversky tells us, we first ought to denominate each of the four baskets with its probability. Assuming that the probability of a particular player sinking any given basket is one in two, we would then take the chances of getting four baskets in a row—one in two multiplied by itself four times, or one in sixteen—to get a sense of the likelihood of a streak. This is not all that unlikely, especially over the lifetime of a player’s career. But, Kahneman says, our minds are built to ignore the denominator. Seeing four baskets in a row, we feel instinctively that such a streak cannot possibly be the outcome of chance.

Has behavioral economics now become vulnerable to the same kind of attacks it mounted on orthodox economics?

This mistake is just the tip of the iceberg. If we frequently fail to properly analyze events in terms of their probabilities, Kahneman and Tversky showed, we just as often overweight their probabilities. Buying a single lottery ticket increases the probability of our winning the jackpot by a very small amount: from zero to one in ten million. But because doing so takes us from the realm of complete impossibility into the seductive realm of possibility, that minuscule increase seems much more significant to us than it actually is. It gives us dreams to savor—buying that beachfront home, retiring and writing our long-deferred novel. The more vividly we envisage something, the more real we think it is likely to be. Thus we inflate the probability of winning.

Kahneman and Tversky spent decades cataloging a rogues’ gallery of such human psychological quirks, many of them evocatively chronicled in Michael Lewis’s lively new history of their collaboration, The Undoing Project. Each paper the duo published chipped away at the foundations of mainstream economics and its notion that people, as a whole, behave in predictably rational ways. After all, orthodox economics assumes, the market will punish us if we do not behave rationally. People who buy lottery tickets will find that they have insufficient money for groceries. Sports teams that pay enormous sums for seemingly hot players will find themselves deserted by fans. Sure, mainstream economists concede, we are not perfect. People, from time to time, unintentionally do things that are not in their own best interest. But as soon as they suffer the consequences, they right themselves.

What Kahneman and Tversky showed is that our irrationalities are not random but systemic, rooted in deep-seated psychological tendencies—such as failing to take probabilities into account in some situations and overweighting them in others. These ingrained biases defeat countervailing market incentives. Over three decades the academic field that the duo spawned, “behavioral economics,” has produced more and more data demonstrating that market behavior, unaided by constraining norms or regulation, does not operate optimally.

‘We were shaking trees and challenging the establishment. Now we are the establishment.’

Our neglect and misuse of probabilistic thinking is only half the story. We also habitually miscalculate whenever we gauge the utility—the benefit—that we will reap from various goods. ­­­­­Suppose, for example, that we are offered the opportunity to gamble on a coin toss with an equal chance of winning $500 or losing $300. The expected value of such a bet is $100 (.5 ´ 500 minus .5 ´ 300). Does this mean we should always take the gamble? The odds of making money are in our favor, but most of us will pass. Why?

Because, as Lewis explains, instead of simply assessing the bet’s utility based on the positive expected value of $100, we in fact subtract the disutility—the unhappiness—we would feel on having to fork over $300, and that calculation gives us a negative result. After all, the prospect of losing something we already possess (or expect to possess) gives us more pain often than the pleasure we would derive from gaining something even more substantial. Our mistake—Kahneman calls it a “foolish mistake” and a “failure of rationality”—is to look at the gamble by weighing the disutility of the loss against the utility of the gain, and not simply the amount of the loss against the amount of the gain.

Not only do we often consider utility in ways we should not, but on other occasions—as the work of another prominent behavioral economist, Richard Thaler, has shown—we fail to properly take utility into account when we should. Suppose you have purchased a theater ticket and now, on the night of the show, there is a wicked storm. You really would be much happier staying home. Still, because you bought the ticket, you are now likely to choose the course that will bring you less utility. You will head out into the ice and snow. In rational terms this makes no sense; it is a “costly mistake,” as Kahneman says. The money you spent on the ticket has become a “sunk cost”—irredeemably in the past and irrelevant now to the proper decision. Along the same lines, if your job makes you miserable, it would be better to quit it even if you had spent a year training for it: making yourself even more unhappy by hanging on is no way to redeem that sunk cost. Rational actors should not choose the option that brings them less utility, but we do so every day.

• • •

Lewis offers a lucid and perceptive account of the model of humanity the duo developed, and the multiple and inveterate irrationalities that we sublunary creatures harbor. But he does something more. The Undoing Project is a study not just of the psychological model of human behavior that Kahneman and Tversky mapped out, but also of the psychological dynamics of their own human relationship.

Amos Tversky was an extroverted, self-confident, in-your-face academic combatant. Once, on hearing a mainstream economist talk about how so many people he knew were dolts and fools, Tversky nailed him: “All your economic models are premised on people being smart and rational, and yet all the people you know are idiots.” Daniel Kahneman, for his part, was shy and profoundly self-doubting, a character trait that was caused and compounded by the fact that Tversky, who was more gifted mathematically, landed a far larger share of academic accolades, securing a tenured chair at Stanford in the 1970s while Kahneman went to the University of British Columbia in Vancouver. Reading The Undoing Project, and especially the details of Kahneman’s biography and deep insecurities, I couldn’t help but wonder whether the double bind I have always experienced as a Jewish Canadian—what other demographic group in the world comes from both a religion steeped in Woody Allenish self-loathing and a nation that defines itself by apologetic self-deprecation?—might have rubbed off on him.

Who else comes from both a religion steeped in self-loathing and a nation that defines itself by apologetic self-deprecation?

As Lewis tells it, one colleague summed up the duo’s relationship this way: “Danny wasn’t so easy to access, Amos was all out there. . . . that they did interact and that they could interact was a miracle.” They did, until they didn’t. First academic honors separated them—Tversky alone won a MacArthur Fellowship for their joint work—and then, finally, death did. Tversky died of cancer in 1996, and this time it was Kahneman alone who won a prize for the fruits of their collaboration, the 2002 Nobel Prize in Economics.

“We started this unknown field,” Lewis reports Tversky saying in 1983. “We were shaking trees and challenging the establishment. Now we are the establishment.” That raises the question as to whether behavioral economics itself is now vulnerable to the same kind of attacks it mounted on orthodox economics. After all, while behavioral and orthodox economics differ over the extent to which human beings behave irrationally, they both agree that it is irrational to do such things as overweighting the probabilities of winning the lottery or the disutility of losing money in a bet we have an on-balance chance of winning. True, unlike many other behavioral economists such as Dan Ariely—who straightforwardly brands human beings “predictably irrational”—Kahneman himself backs away from suggesting we are an irrational species: we are obviously creatures who respond to reasoned argument. But even as he shrinks from using the term “irrational,” Kahneman says that the syndromes he and Tversky identified make us choose “delusional options” and “inferior outcomes”; and, yes, that we often exhibit a “failure of rationality.” Perhaps the best way to understand Kahneman is that we are indeed irrational insofar as we habitually make choices that make no sense for us given the kinds of creatures we are, but reasonable insofar as we can at least come to understand this and perhaps even mitigate it.

But is this true? Or is it possible that the behaviors Kahneman and Tversky diagnosed can equally be understood as fundamentally rational responses to the world?

• • •

It is not clear that Kahneman and Tversky’s model of man can adequately capture key features of our relationship to the past and the future. Think of the bet in which we face an even chance of winning $500 or losing $300. When the duo tell us that we habitually overweight the disutility of having to give up the $300, they are right only from a past point of view. In the past, before we possessed that $300 and thought of it as ours, we did not experience very much disutility from its absence in our life. From that perspective we do behave irrationally when we refuse now to risk that amount. We should compare apples to apples: how we felt about the $300 at that point in the past when we did not yet have it, against how we feel now about the $500 when we are yet to have it: and if you do not have either, $500 trumps. The problem, though, is that we can never put ourselves back into that past frame of mind. We cannot return to the farm once we have seen Paree. And so loss of something—whether a house, a person, or a sum of money—will always strike us as more painful than simply never having had it in the first place. From our perspective in the present, it is not necessarily irrational to refuse an even/odds bet of getting paid more than we would lose.

Our prevailing explanations of human decision making are far from complete.

By the same token, when it comes to the future—as far as we can make it out from our perspective in the present—we are behaving irrationally if we buy the lottery ticket. And yet we cannot say with certainty that we are throwing money down the drain. As President Obama observed in the wake of Trump’s victory, in life there “is always the possibility of surprise. . . . It doesn’t matter what the odds are. Weird stuff happens.” Suppose that, in fact, we harbored no irrational compulsion to believe in our vivid fantasies of buying a beachfront mansion when we win the jackpot. Still, the fact that the future is objectively unknowable even if subjectively predictable would at some level pull us toward buying the ticket. We know that for the winners the extremely unlikely will, in fact, turn out to have occurred.

We are thus divided souls. As creatures for whom the future is subjectively knowable, of course it is irrational to bet on lotteries. But as creatures for whom the future is ultimately objectively unknowable, lottery betting cannot finally be stamped as irrational. In the same way, as creatures for whom our past is objectively knowable, it seems irrational for us to harbor a substantial aversion to losing what we were, once upon a time, indifferent about not even having. But as creatures who can no longer access our past subjectively, that aversion is not irrational.

This fault line pervades the duo’s findings. Kahneman and Tversky tell us, for example, that a player’s run of four baskets in a row is a random occurrence. And certainly from a subjective perspective—for all our limited minds allow us to perceive—many events in life have no discernible causes. But while from a subjective perspective much happens that must be deemed random, objectively nothing in fact does. Even if we cannot always see them from our subjective vantage point, we know it to be an objective truth about our world that every event—including the four baskets in a row—has preceding causes that explain it. We might be mistaken, ex post, in the factors we settle on to account for a hot streak. But we are not being irrational, ex ante, in treating the streak as caused by some thing or things, and not as a random series of events.

In the final analysis, then, the four baskets in a row is not a “truly random” event and, as knowledge-seeking creatures, we will always be drawn to find the prior causes. Kahneman and Tversky themselves triumphed precisely because they relentlessly pushed to find underlying causes in what mainstream economists viewed as mere random irrationalities. And just as everything that happens in the present has a cause, so we can change the past by what we do in the present—at least in a subjective sense. That is why going out and seeing the show on a cold wintry night is not irrational. It vindicates our previous decision to buy the ticket, making us feel as if our purchase was not a waste.

Our prevailing explanations of human decision making are far from complete. What Kahneman and Tversky have missed remains an ongoing story for future social scientists, philosophers, and creative writers to tell.