For this key, one can quickly kill the two Stalfos with arrows.
Doing a jump attack and holding the sword button as Link lands will make the block switch toggle back and forth quickly. This can be used to speed up the movement to the key here.
With a precise jump, one can skip crossing the bridge to the Nightmare door.
After breaking the bottle, charge your sword and wait for the Genie to re-appear and do his attack. Make sure to wait in the middle of the room while this happens, as standing in the area where the Genie might spawn will delay when he spawns until you move. After he appears, take the hit from his fireball and release your spin attack, then quickly get two sword slashes in before he starts spinning around again. Perform the same strategy for the second time he respawns and he should die on this cycle.
A slightly faster and a little more advanced strat is to look for which of the 2 is closer to the wall, that tells you which one is the real one and lets you anticipate where Genie will appear so you can attack it before it throws the fireball, just slash as fast a possible and repeat on the second cycle. The tricky part is if you don't interrupt the fireball throw, this strat will not work. Here's a video of it