Not only because they are better than you gave them credit for, but especially because you were listening to far worse songs at the time. A real "those in glass houses" moment, I mean.
Now, when this first came out, I loved it as much as everybody else. The almost childlike sincerity must have struck a nostalgic chord in most of the audience. I was in fifth grade, though, so it felt like music made just for us as we were becoming possessed by our first big crushes. It was on the air all the time, and that's one of the earliest memories I have of pulling the radio closer and singing along, feeling like they were right there in my room.
Scarcely two years later it was 1993, with the alterna-grunge and gangsta rap trends destroying the catchy, make-you-come-alive music of whites and blacks, respectively. Every young person was tripping over themselves to prove how extreme and antagonistic they were, even though we were all just a bunch of posers and wiggers. (Sadly that was not as bad as it would get, and it's only gone downhill since then.)
When such a massive shift is underway, the triumphalist hordes of dorky teenagers will look for the easiest victims to make an example out of, so that the others understand we're doing things differently now. The antithesis of the new zeitgeist was something soft or moderate and reaching out rather than shoving away, so naturally we went after rock ballads. You might think that the acoustic, folk-influenced sound would have spared Mr. Big during the era of MTV Unplugged, but to us it just made it sound even wussier.
These still are not my favorite kind of song, and I wouldn't put most of them near the top, but again remember what the sacrificers were listening to themselves: Nirvana and Dr. Dre. Compared to that, let alone the even less listenable white and black music of today, "To Be with You" is a real breath of fresh air -- melodic, uplifting, and charming.
I recently bought a 2-disc compilation called MONSTER BALLADS, and it's for sure one of the most replayable tunes on there.
I'm curious to hear other people's stories, particularly if they're from a different time period. Aside from hair metal ballads, I'd guess that disco deserves the greatest apology from anyone who makes an honest reckoning of their own tastes when they were bashing it. Punk wasn't so great in itself, but more as an ingredient that was later incorporated -- along with disco -- into New Wave, sometimes by the same artist (like Billy Idol). And mainstream rock of 1980-'81 wasn't too hot either, kind of figuring out where to go after the various Seventies sounds and before the explosion of heartland rock.
Again I don't think a dispassionate look would put disco up near the top of popular music, but certainly much higher than it's been regarded, and better than what most of its detractors were listening to.