From the start, Little Britain is letting you know they are out of f***s to give with their comedy. You dig it or you don't.
Is it racist? Well, not exactly. If anything it's more politically incorrect considering the climate of ultraconservativeness that plagues the demographic. They are just comedically saying what a lot of regular joes are saying in public and behind closed doors and trying satirically to bring a discussion to the forefront.
However, as seasons went on and they became more popular, society by and large took the message the WRONG WAY. They aren't a champion of xenophobia, but can be perceived that way because they are so biting at their stereotypes against women, minorities, and anyone that isn't a white British male. Even with poking fun at the LGBTQQAI community and with the disabled, it's easy for racist groups to champion them because they are pointing out a subliminal fact there are many in society that refuse to give you 100% rights or look at people as individuals if they aren't white, male, not gay, not disabled, and definitely not in an interracial relationship. And its tongue in cheek how they do it.
Honestly, this is what Python could have been when it was more politically incorrect but they didn't have the nerve.