One thing that i am wondering: Imagine i open a portal to an 'uberboss', but then find out that i'm not powerful enough to beat him. Will this cost me the 3 keys, even although i didn't defeat the boss?
I'm a bit disturbed by the fact that you apparently need an entire team to beat them. I play solo most of the time, and when i do play with a few friends, it's a lot more casual as they have very little time to play and hence have terrible gear.
Yes, you will lose the infernal machine when you create the portal. You can keep trying until you leave the game. You don't need a team to beat them - however, it is much more efficient to do so since only one of you has to use your infernal machine. And you can fight them at MP1 if you like, though they won't be very likely to drop the ingredients.