From Wiktionary, the free dictionary
you had to be there
- Used to indicate to the interlocutor that the situation being talked about could only be properly understood if that person had been present.
What he did was so hilarious, the way he was dancing... well, you had to be there.
the situation could only be understood if the person had been present