There’s a story told about Joe McCarthy—not the right-wing senator from Wisconsin, but the manager of the great New York Yankees teams of the 1930s and ’40s. McCarthy dreamed that he had died and gone to heaven, where St. Peter told him to assemble an all-star team. McCarthy was excited: he’d have Christy Mathewson and Walter Johnson, Honus Wagner, Lou Gehrig. Just then, the phone rang. It was Satan challenging McCarthy to a game. “You haven’t got a chance of winning,” McCarthy exclaimed. “I’ve got all the players.” “Oh, I know that,” Satan answered. “But I’ve got all the umpires.”

The umpire can have a huge impact on how the game turns out, as the Supreme Court’s self-described umpire-in-chief, John Roberts, has shown. But the chief justice ignores this when he celebrates the fact that today’s Court is the first without even a single justice who has served in elective office, claiming that drawing on the lower courts for justices—as opposed to drawing on Congress, governors’ offices, and the Cabinet—has shifted the Court from “fluid” considerations of “policy” to the “more solid grounds of legal argument.” In the Affordable Care Act cases, where the Court narrowly upheld the individual mandate while imposing new and potentially significant limitations on Congress’s commerce and spending clause powers, he wrote that the Justices

possess neither the expertise nor the prerogative to make policy judgments. Those decisions are entrusted to our nation’s elected leaders, who can be thrown out of office if the people disagree with them. It is not our job to protect the people from the consequences of their political choices.

But given how much judges influence policymaking, one of the most important consequences of the people’s political choices is the composition of the courts themselves—and not just the Supreme Court. This past term, the Supreme Court issued opinions in 65 cases after oral argument, the fewest since 1953–4, when it decided Brown v. Board of Education. But the federal courts of appeals disposed of more than 57,000 cases in the year ending last September, and the federal district courts of more than 300,000. The judges serving on these lower courts have tremendous leeway on a range of matters, from sentencing individuals in federal criminal cases to determining whether a plaintiff’s claim of discrimination is plausible enough to proceed to discovery and trial. Federal laws and policies concerning everything from consumers’ rights to privacy to labor protections are enacted by our nation’s elected leaders, yet they often are only as effective as the judges who apply them, case by case.

Thus one of the most important legacies a president leaves behind is the judges who continue to shape the law for decades. Federal judges serve for life, and Justice John Paul Stevens’s distinguished career—he was nominated by President Gerald Ford and served from 1975 until shortly after his 90th birthday in 2010—is a reminder that judges can live a very long time. Earlier this year, Judge Wesley Brown, nominated to the federal district court in Wichita, Kansas by President John F. Kennedy in 1962, died while still serving at the age of 104.

The nomination and confirmation process that put Brown on the bench 50 years ago has become increasingly politicized and dysfunctional in the quarter-century since the failed nomination of Robert Bork turned filling the federal bench into a blood sport. Though public attention naturally focuses on Supreme Court seats, much of the real action concerns seats on the thirteen federal courts of appeals and the 94 district courts.

Over the last 30 years, as American politics has become polarized to a degree not seen since the nineteenth century, a conservative legal movement has self-consciously sought to move the federal courts to the right as part of a campaign to roll back key pieces of existing legal doctrine, such as federal power to regulate the environment, broad protections against discrimination, and procedural rights for individuals accused and convicted of crimes. Recent Republican administrations have pursued a judicial nomination strategy that seeks to appoint young, deeply committed conservative lawyers to the federal district courts and courts of appeals. Republican activists pressed judicial nominations as a priority, and conservative presidents and senators worked hard to get these nominees confirmed. The result is not just a deep farm team of potential Supreme Court nominees, but also a host of conservative judges inscribing their views of constitutional law and their interpretations of statutes in thousands of cases that never reach the Supreme Court. If their cases are appealed to the top, then those judges’ opinions shape the terms of the debate in the high court.

When Obama was sworn into office, there were 55 vacancies on the federal bench. There are now more than 75.

By contrast, the Obama administration has done relatively little to bring the courts back into balance. When Obama was sworn into office, there were 55 vacancies on the federal bench. There are now more than 75, and Obama will likely “become the first president since Reagan, and possibly much earlier, to finish his first term with more vacancies than he inherited,” according to Alliance for Justice. (Vacancies weren’t tracked before the Reagan years.)

To be sure, much of the problem is conservative obstructionism. Several of the president’s most prominent nominees have been filibustered. Many others have been victims of “blue slips”—an internal Senate procedure that allows either senator from a nominee’s home state essentially to block a nomination. Even candidates who ultimately are confirmed by overwhelming votes have faced a series of procedural roadblocks. And the prospect of having to put their careers on hold has probably dissuaded many qualified candidates even from seeking nominations.

But conservative obstruction is not the only problem. During the period when Democrats held a filibuster-proof senate, the administration moved far too slowly to make nominations. Granted, it was a busy time, with the economic crisis, two Supreme Court openings, and health care reform occupying the administration’s attention. But the delay also reflects a sense that judicial appointments are less important than other policy levers and that the president’s base does not really care about the issue.

Moreover, while the current administration has achieved an admirable degree of gender and racial diversity among its picks (nominating roughly two times more women and persons of color, by percentage, than the preceding administration had), its court of appeals nominees are generally older than their conservative counterparts. And very few of them have backgrounds either as public-interest lawyers for liberal or progressive causes or as scholars who have responded to conservative legal thought.

The administration’s inability to fill vacant seats on the federal bench has both immediate and long-term effects. Right now, more than half of Americans live in jurisdictions facing what the Administrative Office of the U.S. Courts calls “judicial emergencies”: their empty benches can’t handle heavy caseloads. Because criminal prosecutions take priority, plaintiffs with civil claims—workers, victims of government misconduct, consumers, and others—face severe delays in vindicating their rights.

In the longer term, vacancies accrued during a liberal-leaning administration that remain open for succeeding conservative administrations to fill will ensure that courts continue to skew to the right, meaning that progressive legislation and regulations will be undercut when it comes time to enforce them. The lack of a vigorous judicial response to conservative versions of originalism and “strict construction”—which often means little more than a cramped reading of broadly worded statutes and constitutional provisions—will bias popular discussion and debate.

Even the best players can be hamstrung by a hostile umpire. Unless liberals make sure the umpires are not all from deep right field, they will face serious difficulties down the line.