Why don't I see gold at the end of the remote working rainbow?

From talking to friends, I feel this is a very controversial opinion to have and I don’t really get why. Any help to make me understand would be greatly appreciated! And just to be clear, I absolutely do get that for some people (fresh parents, people living at home to take care of their parents etc.) remote work is a real blessing. I am just wondering if that is really the case for the majority or what it is that I’m missing.