Ive seen the past threads about the wardrobe system losing gems, and so personally i dont use the wardrobe system at all, but I have however still managed to join the many that have lost gems.
A quick summary of events... after finishing levelling a Boon of the Hoarder to 50 I equipped it into a ring and left the previously equipped Pain Enhancer in my bags. I then restarted the game to change the Torment level (without fully relogging) and afterwards found that the Boon of the Hoarder was no longer equipped nor in my bags but the Pain Enhancer was still in the bags were it was left.
My uneducated guess is that the cause would be a syncing delay or error between client and server, the final action before relogging was to equip the Boon, and even then i wasnt particularly fast to restart so there really shouldnt have been a problem.
Now to me a 50 Boon isnt really a big deal, but if my posting helps someone else to avoid losing a high levelled gem it would make a little more acceptable at least, so pls take care not to relog after equipping or changing gems guys.