iTerm fires off X11 when using SSH (solved)

I've been trying to figure out why every time I use iTerm to ssh to a machine X11 starts. It slows down the whole login process, and is annoying because then X11 has focus, not iTerm.

I finally found it in my ~/.ssh/config file:

ForwardX11 yes

Commented that out and now it's not firing up X11. Not sure when I set that up, might have been as an experiment to see how someone else's setup worked.