...The Imperium does interfere: no slavery, no nukes, no psionics (well, not so much of that now compared to the past). What else does the Imperium proscribe?
The Imperium can't enforce day-to-day civil rights without governing directly, and any attempt to do so would scare the bejeezus out of most of the member worlds. It can enforce simple bans by removing any government that breaks them. And as long as the other member worlds can say, "Oh, well, that won't happen to us because we won't employ chattel slavery" or "Oh, well, psionics; those guys deserved all they got" or something like that, the Imperium can do it. ...
Well, that is a measure of interference. You're right, though, the 3I can't very well open up a civil rights office on every world and investigate every disgruntled crank who wants to claim the Boss o' Bosses is enslaving the populus. A lot of these worlds are pretty oppressive - must be a lot of people who'd love to stow away on some ship and then make complaints from a safe distance.
Kinda makes me wonder what, if any, method they do have for detecting member naughtiness.