Instead of long-discounted proxies like skull circumference and family pedigrees, according to experts who track the far-right, today’s proponents of racial hierarchy are making their case by misinterpreting research on the human genome itself. And in debates that have largely been limited to ivory-tower forums, the scientists whose job is to mine humanity’s genetic variations for the collective good are grappling with how to respond.

“Studying human genetic diversity is easier in a society where diversity is clearly valued and celebrated — right now, that is very much on my mind,” said John Novembre, a University of Chicago evolutionary biologist who has taken to closing his visiting seminars to illustrate how one of the field’s textbook examples of natural selection has been adopted for illiberal ends.

One slide Dr. Novembre has folded into his recent talks depicts a group of white nationalists chugging milk at a 2017 gathering to draw attention to a genetic trait known to be more common in white people than others — the ability to digest lactose as adults. It also shows a social media post from an account called “Enter The Milk Zone” with a map lifted from a scientific journal article on the trait’s evolutionary history.

In most of the world, the article explains, the gene that allows for the digestion of lactose switches off after childhood. But with the arrival of the first cattle herders in Europe some 5,000 years ago, a chance mutation that left it turned on provided enough of a nutritional leg up that nearly all of those who survived eventually carried it. In the post, the link is accompanied by a snippet of hate speech urging individuals of African ancestry to leave America. “If you can’t drink milk,” it says in part, “you have to go back.”

In an inconvenient truth for white supremacists, a similar bit of evolution turns out to have occurred among cattle breeders in East Africa. Scientists need to be more aware of the racial lens through which some of their basic findings are being filtered, Dr. Novembre says, and do a better job at pointing out how they can be twisted.

But the white nationalist infatuation with dairy also heightened Dr. Novembre’s concerns about how to handle new evolutionary studies that deal with behavioral traits, such as how long people stay in school.

Anticipating misinterpretations of a recent study on how genes associated with high education attainment, identified in Europeans, varied in different populations around the world, the lead author, Fernando Racimo, created his own “frequently asked questions” document for nonscientists, which he posted on Twitter.

And in a commentary that accompanied the paper in the journal Genetics, Dr. Novembre warned that such research is “wrapped in numerous caveats” that are likely to get lost in translation.

“Great care,” his commentary concludes, “should be taken in communicating results of these studies to general audiences.”

Already, some of those audiences are flaunting DNA ancestry test results indicating exclusively European heritage as though they were racial ID cards. They are celebrating traces of Neanderthal DNA not found in people with only African ancestry. And they are trading messages with the coded term “race realism,” which takes oxygen from the claim that the liberal scientific establishment has obscured the truth about biological racial differences.

Some scientists suggest that engaging with racists would simply lend credibility to obviously specious claims. Many say that they do not study race, in any case: The racial categories used by the United States census correlate only imperfectly with the geographic ancestry groupings of interest to evolutionary geneticists. “Black,” for instance, is a socially defined term that includes many Americans who have a majority of European ancestry.

But as the pace of human population genetics research has accelerated, it has yielded results that, to many nonscientists, appear to challenge the idea of race as a wholly social construction. Genetic ancestry tests advertise “ethnicity estimates” (Senator Elizabeth Warren appealed to the perceived authority of DNA this week to demonstrate her Native American heritage, in response to mocking by President Trump), and some disease-risk genes have turned out to be more common among certain genetic ancestry groups. Doctors use patients’ self-identified race as a proxy for geographic ancestry, because individual readouts of DNA are costly, and though the correlation is imperfect, it exists.

As DNA databases tied to medical records and personal questionnaires have reached a critical mass for individuals of European descent, moreover, so-called polygenic scores that synthesize the hundreds or thousands of genes that contribute to many human traits into a single number are being developed to predict health risks, and in some cases, behavior.

Last summer, researchers developed a score that can roughly predict the level of formal education completed by white Americans by looking at their DNA. And while those scores cannot yet be compared among racial or population groups, the new techniques have prompted some scientists to feel it is the field’s responsibility to head off predictable misrepresentations.

“You have to make a judgment when you have powerful information that can be misused,” said David Reich, a Harvard geneticist who has publicly called on colleagues in a recent book and in a New York Times Op-Ed to more directly address the prospect of identifying genetic differences between populations in socially sensitive traits.

There is no evidence, scientists stress, that environmental and cultural differences will not turn out to be the primary driver of behavioral differences between population groups.

At the same time, the advances in genetic technology have put white supremacists into a kind of anticipatory lather.

“Science is on our side,” crowed Jared Taylor, the founder of the white nationalist group American Renaissance, in a recent video that cites Dr. Reich’s book.

Dr. Reich was among those to decline an invitation to lead a discussion on the topic at the San Diego meeting. “I really wanted to return to research,” he said.

The widespread uncertainty among Americans over what scientists know about genetic differences between racial groups, experts say, has left many flummoxed in the face of white supremacist claims that invoke genetics.

“I was surfing my favorite dumb picture site and I came across a post trying to prove racism with science,” a community college student in Florida wrote to Jun Z. Li, a University of Michigan geneticist whose work has been invoked to buttress racist claims of white intellectual superiority. “I read through the paper myself but I do not have the education or experience to understand and make sure I have a coherent counter argument.”

For white Americans half-inclined to blame nonwhite immigrants or African-Americans for perceived social problems, the veneer of a scientific rationale for white superiority, researchers say, can tip them toward racial resentment. It can be more effective than base appeals to tribalism, especially for the educated demographic the far-right has been targeting.

And while much of current white nationalist rhetoric is framed in terms of preserving a white cultural identity, experts say it relies on a familiar narrative of immutable biological differences. On a YouTube talk show earlier this year, for instance, Gavin McInnes, founder of the Proud Boys, whose appearance set off a brawl outside a Republican club in Manhattan last week, echoed the pet white supremacist theory that the environmental challenges of cold winters explain the supposed higher intelligence of northern Europeans.

Some geneticists have penned blog posts explaining why new genetic tools will not support white nationalist claims that average behavioral differences between groups are immutable. Others — including Dr. Li — have replied directly to individual queries.