This is itself a racist statement, and it's ugly.
You're looking for racism where none exists, and in the process have judged a racial group based on your preconceived biases, and it's inexcusable.
When people behave as you are, you don't capitulate, you tell them to !@#$ off, because people who make unfounded charges of racism are !@#$s, and are the entire problem with race relations in this country right now.