I want to make it perfectly clear: I do love Goblet of Fire. It's just not my favorite of the books. I've never been entirely sure why that is, but I think it has something to do with the first time I read the book. I was pretty young (about seven) when I started reading the Harry Potter books, and back then Sorcerer's Stone was considered a thick book to me. I remember feeling kind of intimidated by the size of Goblet. I was a good reader for my age, but I couldn't imagine reading a book that thick.
I don't think I've ever admitted this to anyone, but I actually never finished Goblet until after reading Order of the Phoenix. When I was about seven, I started reading Goblet, got bored sometime during the Quidditch World Cup, and gave up. I even lied to my parents and told them that I'd finished the book. (I don't think they believed me. No seven year old reads a book as long as Goblet that fast.) I was nine when Phoenix came out, and I read it with no problem at all. (Although it took me a couple of weeks to finish it.) There were several things in Phoenix that didn't make sense to me, and I knew that was because I hadn't finished Goblet. By the time I'd finished reading Phoenix, I knew I was going to have to go back and read Goblet. This time I successfully finished the book, but I still found it boring compared to the others.
I think my biggest problem with it is the Quidditch World Cup. I'm not a sports person, so the Quidditch games have never been my favorite parts of the books. That's were Goblet lost me the first time I tried to read it. My second time reading it, I forced my way through that, and I thought the rest of it was a lot better. Still, I've never been a huge fan of the Triwizard Tournament, and I think this is also because I'm not a big sports person. It's just not my type of thing.
There are certain parts of Goblet that I love. The Yule Ball and the graveyard scene for example. Yet everytime I re-read the book, I'm always a little bored during the World Cup and tournament tasks. They're just not my thing, and I'm sure they're the reason Goblet of Fire has always been my least favorite Harry Potter book.