@Tinkrbl44 - We have ALWAYS watched the Super Bowl! Sometimes we're home; sometimes we're at a party.
IIRC it used to be Sunday afternoon, some time around 2 or 4ish. I remember that because we had to make sure Sunday services were over on time!
One year about 20 years ago we had a party for the youth at church. We had food and brought a TV in. (Now we have 3 big screens). The kids had fun, but we didn't do it the following year. Most of them wanted to watch with their families, in their own homes, on a big screen!
I always thought it was on Sunday because that's when most NFL games are played. Another reason might be most viewers can be home on Sunday.
I also remember about 20 years ago a lot of stores had special "sales" for women. They were all day long, so if I went to one I was always home in time for the actual game. -- and the commercials of course! 