If he doesn't see you, you get what you needed and no harm no foul.
If he sees you, or if he has cameras that see you, then he gets to feel the arousal aspect of your connection without the actual relationship, and he's going to hope it happens again, and fantasize about it until it creates a monstrous longing. That might be what you want, and it might even inspire him to try to patch things up (if you're open to that), but you might also find yourself dealing with creepy or stalker-ish behavior.
