I have had none of these problems. I have built well over a dozen systems in my XB and have had no paint loss and the only time holes don't match is if something isn't seated right. I have had all the cages in and out, I have removed mesh's and buttons to paint, I have had the feet on and off for different desks, 80, 90, 120, 140 and 200mm fans, multiple liquid loops, and drives of all kinds. My system is apart about every 6 to 8 weeks for cleaning and then whenever I make a change, which is often. The only problem I have had was I broke part of one of the hot swap sockets but it was my fault and it still works. Maybe you got an inferior one but CM is great about support. As far as getting to those connectors it takes less than 5 mins to do that and I'm including with a mounted radiator. Maybe you chose poorly for your needs.
Good luck and I hope you find a way to appreciate all the good things the XB has to offer.