This is all situational. It strictly depends on your build and your existing toughness to healing ratio. And when I mean healing o mean dynamic ( lps, lph, lpk). You can have 2 trillion toughness but still get 1 shot in high GRs, how so? Because you lack sustain aka true healing. Molten is worthless if all your toughness is from vitality.
Builds with inherently high APS, esoteric will be better because that lps and lps from paragon is plenty. But the "off cooldown" builds need some kind of sustain. Example HC LoN bomb I preferred molten. But HC Grenade DH I would go wth esoteric.
Like I said it is very build dependent.
That is a very good point - molten loses value as vitality increases. Really good point.
Most players will tell you that Esoteric is better all day, every day than Molten. However, I love that gem. I find that the life shield is helpful and the regen helps me get back into the action more quickly. There's probably a GR/damage scaling argument in there somewhere.