As Fabian points out, starting at the 1:04 minute mark in the video below, the comments were offensive, ignorant, stupid, and any other number of adjectives thrown at him for his remarks, which a video/audio of those remarks will be shown below the Fabian4Liberty video, but they still fall under the right of free speech.
Especially private free speech, which without his permission was made public.
If this NBA ban is enforced, then as the title of the video states below, free speech is in real trouble. Free speech protects even stupid, offensive speech as long as it isn't meant to incite violence.
Another point made is that the free market can take care of this all by itself. Fans can stop going to games, sponsors can, and some have, drop their contracts, players can get out of contracts. Listen to the rest below.