The Eirev Rav were the instigators of chet ha'egel, and brought the rest of Klal Yisroel down with them. Although we usually accept geirim (converts), we do not accept them when they want to join in a time when everything is going well for the Jews. Their motives are selfish, and they are not choosing to fulfill the mitzvos because of a genuine desire. The Eirev Rav joined Bnei Yisroel because it appeared to be materially worthwhile, but did not give up their ties to avodah zarah. This selfishness is a strong contrast to the selflessness of the Leviim, who ignored family ties and self-preservation in order to kill those who took part in the chet ha'egel, including their own relatives. The Leviim were elevated to their status of serving in the mikdash because of this selflessness.