Palantir Software: Discrimination & Discovery? | National Review

Palantir Software: Discrimination & Discovery? | National Review


Two questions about ‘discrimination’ The arresting headline — “U.S. Department of Labor sues Palantir for racial discrimination” — could have gone with two very different stories.

The first possible story, the obvious and boring one, turns out, alas, to be the operative one: The Obama administration is going after Palantir Technologies, a “big data” concern started by Silicon Valley entrepreneur Peter Thiel, on the grounds that it discriminated against Asian-American job applicants. The case is risible, resting on an absurdly small data set (three job descriptions and 21 hiring decisions) and the assumption that applicants for highly specialized positions in one of the world’s most esoteric technology companies are interchangeable widgets.

It is less than obvious that Asian Americans have been shut out of technology jobs in Silicon Valley, but, by all means, let us consider the question.

Palantir says that the government is engaged in “flawed statistical analysis.” It seems more likely that the Obama administration is engaged in straightforward political retribution and intimidation: Peter Thiel is an increasingly vocal Republican activist who spoke on Donald Trump’s behalf at this year’s Republican National Convention. (In the interest of disclosure, I should note that he is a contributor, both editorially and financially, to National Review.) Democrats prefer being opposed, if they must be opposed at all, by southern biblioplangists who lend themselves to caricature; cerebral California technology billionaires, on the other hand, are the kind of opposition they could do without, hence the desire to make examples of those who step out of line. Given the current administration’s long, nasty, and criminal history of using agencies of the federal government to go after political enemies, that seems a perfectly reasonable explanation.

(Of course, it is just barely possible that the administration’s motives here are innocent; one of the problems with political corruption is that it casts suspicion on any action that might reasonably be interpreted as corrupt, which is one of the reasons why Lois Lerner and John Koskinen should be in a federal penitentiary.)

Thiel, who has shown a flair for litigation lately (he financed Hulk Hogan’s invasion-of-privacy lawsuit against Gawker), probably will be able to manage this conflict with the Labor Department, though the increasingly open tendency of Democrats to weaponize federal agencies and prosecutors’ offices (ask True the Vote, Kay Bailey Hutchison, Tom DeLay, Rick Perry, the Competitive Enterprise Institute, Exxon, the pastors of Houston . . . ) makes one wonder why any large and complex business concern would willingly submit to American jurisdiction when it might as easily incorporate in a country with more honest and transparent public institutions, such as Canada or Switzerland.

But what about the other possible interpretation of that headline? When I read it, I thought for a second that it might mean not that Palantir Technologies Inc. was accused of discrimination but that Palantir itself stood so accused.

Palantir is an artificial-intelligence platform. There are many versions of it operating around the world: The federal government uses it to track down financial criminals and, if the whispers are to be credited, sundry terrorists camped out in the dusty corners of Jihadistan. Hedge funds use it for their own purposes. Information Warfare Monitor used it to uncover the GhostNet in China. It was used to help organize relief efforts after Hurricane Sandy. It is, to say the least, an interesting piece of technology.

But the science-fiction stuff — artificial intelligence, machine-learning systems, neural networks, all that cool-sounding innovation — already is working its way into the much more quotidian aspects of life, particularly in areas such as actuarial analysis and credit. This presents an interesting problem, as Clive Thompson writes in the current edition of Wired: Once a sophisticated neural network is up and running, it “learns” by processing massive amounts of data, and its decision-making processes are opaque, even to the people who designed it. It is a “black box,” a very, very black one, in fact. “Ask its creator how it achieves a certain result and you’ll likely get a shrug,” he writes.

This will affect ordinary people in predictable ways. Thompson considers the case of a homeowner being denied property insurance. Today, that denial could be explained by any number of financial or geographic factors, but systems such as Palantir are useful in part because they detect relationships that are not obvious to humans, or that are counterintuitive. It is not only possible but likely that such systems will produce results that are discomfiting in some quarters. It is not difficult to imagine that they will produce substantial disparities in health-insurance prices, mortgage rates, consumer-credit offers, and the like, and that those disparities will follow demographic cleavages that are politically sensitive. Wider use of such decision-making processes — say, in screening job applicants or making admissions decisions at public universities — will produce new and knotty problems.

The European Union has passed a law entitling consumers to explanations of how financial institutions make decisions about them, but those explanations may turn out to require advanced study at MIT.

What should we think about such opaque decision-making processes? We should begin with the three most important words in public policy: “Compared to what?” Black-box systems are likely to prove superior to our current model — nerds with actuarial tables — and may be less biased. Bias in actuarial methodology is a longstanding problem in the field, and a subject of intense study by its experts. The problem with black-box systems is less likely to be their propagating bias but their revealing it.

To take one example, African-American men are shorter-lived than the average American man, and than white men. African-American men also suffer from certain health problems at much higher rates: Their rate of diabetes, for example, is 70 percent higher than that of whites. There is an interesting legal and political history to how these realities (and, in some cases, racial fictions) have been incorporated into insurance pricing. But the trend has been very strongly against that kind of discrimination, to say the least. Indeed, one of the baffling features of the so-called Affordable Care Act is its insistence that insurance companies may not “discriminate against” people with pre-existing conditions, as though it were logically possible to insure against events in the past any more than one could go to Vegas and place a bet on last year’s Super Bowl.

Credit scores and income statements are pretty blunt tools, as are many of the instruments used in calculating insurance premiums. What is likely to emerge from black-box systems is not a recapitulation of decisions based on gross racial categories but highly sophisticated and highly individualistic analyses that nonetheless produce results that are, for lack of a better word, discriminatory, though whether a machine can engage in racial discrimination properly understood is a philosophical question. No one doubts that an effective system for screening terrorism threats would spotlight more people from Kandahar than from Helsinki. What we sometimes denounce as “profiling” may be useful or not, depending — we always seem to overlook this part — on what is in the profile and how it is constructed. A badly designed profile might propagate bias; a well-designed one might reveal underlying social realities about which we would prefer not to think too much.

It is safe to predict that the Department of Labor is not thinking very much about this problem just yet.

Most of our large pieces of policy architecture date from the New Deal and the Great Society, from that enormous boom in managerial thinking that characterized mid-century America, whose faith in free markets had been shaken by a misunderstood Great Depression and whose faith in government expertise had been inflated by a misunderstood war effort. Most of our political obsessions date from that period, too.

That’s one reason why, for example, our discussions about the condition of black America mainly fail to take into account that the emergence of affluent and prosperous African and Caribbean immigrant communities has complicated what it means to be African American, that clumsy slogans like “Black Lives Matter” fail to account for the fact that the lives of Nigerian-American financiers in Menlo Park are not very much like those of people in East St. Louis. It is why our response to the problem of weak wage growth for low-skilled jobs is so hilariously crude: “Just pass a law saying McDonald’s has got to pay ’em more!” Our being mentally fixed somewhere between 1957 and 1964 prevents us from thinking intelligently about things such as the economy, trade, public pensions, entitlements, national security, and education, much less about the fact that in only a few years the question in discrimination claims will not be “Discrimination by whom?” but “Discrimination by what?”

And what have we seen from the Obama administration, which promised to be forward-looking and evidence-driven? Mainly a return to the most low-tech approach to public policy there is: the enemies list.