This bit sums it up best, really. Combine that with the Royals chumminess and it's not hard to see where Morrissey's vitriol comes from. If I happened to be one of his inner circle I'd probably tell him "Let it go," to which he'd probably reply "But it's a source of great enjoyment for me." Fair enough.
Don't kid yourself. They are just easy targets, I mean good he hangs around with Russell, David W and Jonathan Ross, they are hardly shinning lights. They have also been chummy with Royals.
As well as engaging in mild sexism, bullying old men and eating meat (Ross and David)