The Court has repeatedly held that some kinds of speech is categorically excluded from First Amendment protection (at least setting aside certain kinds of content-based discriminations within those exceptions, see R.A.V. v. City of St. Paul, which I won’t discuss here). The standard list is obscenity, incitement, libel (or, more precisely, false statements of fact), fighting words, and threats, but in 1982 the Court also recognized an exception for child pornography, and in 2008 for solicitation of crime. The exceptions have generally been defined quite narrowly, and the labels don’t fully capture the definitions; they don’t cover, for instance, all false statements of fact, or all speech that might be seen as “inciting” or “fighting words” by people who aren’t familiar with the First Amendment doctrine. But they do exist, and are an important part of the First Amendment doctrine.
The Court has generally described these exceptions as involving speech that is of very little constitutional value, and (usually) that is demonstrably harmful. That’s not always so: Some punishable incitement might be political speech, of a sort that’s usually seen as valuable, but might be unprotected just because of the magnitude and immediacy of the harm that it threatens; obscenity is seen as being of low value, but even when the Court recognized the exception it didn’t claim that it had been proven to be harmful. But it was easy to see the exceptions as the product of a category-by-category estimation of value and harm. And this was even before we got to the question whether the speech restriction was narrowly tailored to a compelling government interest; that question is asked as to restrictions on speech that is seen as constitutionally valuable because it’s outside an exception.