Well, the Banished are only a faction, not the Covenant proper. Most Brutes seemed pretty zealous as new religious converts usually are. But I think the bigger issue is that the Elites operated at a co-equal relationship with the Prophets that put them at extreme risk. The Elites as a whole were starting to ask too many questions.

Harupsis over at Halo Archive made a nice observation that in light of the Banished threat, the "demotion" of the revered Elite leader 'Jar Wattinree makes more sense—they didn't stick him with a fleet out doing nothing, he might have been tasked with fighting their shadow war, away from the spotlight. Fighting the Banished may have been an easy way of trying to get rid of troublesome units within the Covenant—although, like with Atriox, it may have also sometimes *bolstered* their forces rather than reduced them.