Racism is but one small part of natual human xenophobia. At least in some way people are hostile to anything opposite of themselves or their interests. Country folk find reason not to like city folk and the other way around. Think of all the religious wars, territorial wars. Even sports rivalries are usually a harmless means of exercising this competitive hostility.
Indeed, people often take up sides based on nothing more than race, religion, region or language.
The positive side of the tribal impulse is for people of like kind to work together for the common good. The negative offset, and there is most always a negative offset in life, is war, fear and hate of the “other.”