Orin points to a New Republic editorial that credits this quote to Judge Carlos Bea and Ted Olson:

Today, the view lives on in elite organizations like the Federalist Society, with which Roberts has long been affiliated. Indeed, the much-cited coda to Roberts's opinion — that "the way to stop discrimination on the basis of race is to stop discriminating on the basis of race" — is lifted almost verbatim from a 2005 dissent by circuit court judge Carlos Bea, also a Federalist Society booster, which itself recalls a slogan favored a decade ago by former solicitor general Theodore Olson, another Federalista.

Note, though, that Judge Bea actually credited the forbears of his quote, and he didn't include Ted Olson. Here's what Judge Bea wrote:

Or, as more recently said by the late Justice Stanley Mosk of the California Supreme Court:

Racism will never disappear by employing devices of classifying people and of thus measuring their rights. Rather, wrote Professor Van Alstyne, 'one gets beyond racism by getting beyond it now: by a complete, resolute, and credible commitment [n]ever to tolerate in one's own life or in the life or practices of one's government the differential treatment of other human beings by race. Indeed, that is the great lesson for government itself to teach: in all we do in *1222 life, whatever we do in life, to treat any person less well than another or to favor any more than another for being black or white or brown or red, is wrong. Let that be our fundamental law and we shall have a Constitution universally worth expounding.'

Price v. Civil Serv. Comm., 26 Cal.3d 257, 161 Cal.Rptr. 475, 604 P.2d 1365, 1391 (1980) (Mosk, J., dissenting) (quoting William Van Alstyne, Rites of Passage: Race, the Supreme Court, and the Constitution, 46 U. Chi. L.Rev.. 775, 809-10 (1979)).

The way to end racial discrimination is to stop discriminating by race.

Judge Mosk, of course, was generally seen as a leading liberal California Supreme Court Justice. William Van Alstyne had been on the ACLU National Board of Directors until three years before he published his Chicago law review article. Many Federalist Society members do share the view (as, polls suggest, do many Democrats) — but the quote's origin seems to be pretty solidly outside the Federalist Society.