I'm very much the same way - though Empire Strikes Back is my favorite, the original Star Wars comes in a very close second, ROTJ comes in a distant third. The Ewoks were bad enough, but the disappointing way Boba Fett died was the worst traveshamockery in cinema.
I loath the prequels (midichlorians just ruined it for me - more mystique about the force). I don't even use selective memory - I've basically wiped them from the archives.
To be fair, Highlander and Highlander 2, the Quickening - is the same thing. The Planet Zeist almost ruined the first movie for me. ...it took me years to go back and watch the original Highlander after that freaking disaster or of a movie Highlander 2.
I enjoyed TFA, but it certainly wasn't up to snuff when it comes to ESB and maybe not even ROTJ. After TFA, I had my doubts any new SW movie will ever live up to the lofty standards or ESB or the original Star Wars.
I do have my hopes Rian Johnson will bring that "third party writer/director magic" that Irvin Kershner brought to Empire Strikes Back. Rian's writing and directing of Looper gives me much so hope.
I'm definitely a fan of Rogue One. Wasn't as good as the ESB or ANH - but I liked it better than ROTJ. So there's another bit of hope. ...And I love being back in the original Star Wars era story arc. Bring on a young Han Solo and Boba Fett, Kenobi, and more Vader. And as much as I loathed the prequels, story wise they were just awful, and acting wise, worse, I would love to see Ewan McGregor reprise the OWK role. He was the one redeeming element of the prequels.
Now for sure, age of viewing has a LOT to do with it. I was 8 when I first saw Star Wars, and 11 when I first saw Empire Strikes Back. I was tiny, wide-eyed, and the screen was huge and engulfing. It was so easy to get absorbed into that universe when the screen was so huge and I was so small.
I was 30 when I saw Phantom Menace, and it was much easier to analyze and pick apart the glaring inconsistencies of the movie during the first viewing.
My kids love Phantom Menace. They hate Star Wars - just isn't as big, bright and colorful as Phantom Menace. I know a lot of kids who feel this way. A couple of my neighbors are also Original Trilogy nerds like me. Their kids prefer the Phantom Menace.
So yeah. I think age of first viewing has SO much to do with it.