It is the tribal impulse that everyone is wired with that automatically fires off that we don’t like the “other.” It can be any difference — race, country language, religion, ethnicity, social standing, anything.
The general rule is that the more intellect and sophistication that one acquires the easier it is to override this primitive part of the mind.