After the war, many Sangheili leaders still held a great mistrust towards humanity and wanted to finish them off so they weren't a threat and so, many splinter groups, each claiming to be "The Covenant" continued a campaign against humanity, one of the most prominent and powerful being the reformed Covenant under Jul M'dama, who's "Storm" faction you see in the trailer and fight in Halo 4.
tl;dr Splinter factions of the elites still hate humanity.
The Unggoy followed whoever would lead them because they posses no technology of their own or the knowledge to use advanced tech like space-faring ships. As for the Kig-Yar, they're pirates and mercenaries mostly. They work for whoever. Some of them established trade routes with Humanity, some just went home, some stayed with the Sangheili, and some went back to their pre-Covenant ways of piracy.