I was surprised too, but it's been said many times by those who knew him. I suppose it isn't that surprising, for reasons that you said. I suppose the downside of being famous is that you never know if people like you for who you are, or because of the fame you have. And everything he said or did could be turned against him... I think it makes sense that he would prefer to isolate himself.