Yearning songs and bittersweet songs
You don't hear either type anymore, now that everyone's so afraid to take risks, particularly in social settings. That does seem to be the major reason for the disappearance of those emotions, in real life or in pop culture.
Yearning songs are not mopey, like "wah-wah, why don't you notice me or want me?" It's meant to galvanize you into action, and it does so by tugging at your heart-strings. That way, you feel like you're no longer in control of yourself, and can just go on auto-pilot. It really is a huge risk to go after the type of person they're singing about, requiring an almost blind leap of faith to make a move. But it's a lot easier when you get pumped up and shoved out onto the dance floor. Good music can spur you on like that.
The genre got going as people became comfortable leaving their mid-century risk-averse cocoons, starting with "All I Have to Do Is Dream" by the Everly Brothers and "In Dreams" by Roy Orbison. It doesn't really get going until the mid-'60s, though, with "Needles and Pins" by the Searchers and "I'll Feel a Whole Lot Better" by the Byrds (yearning for the girlfriend you thought she'd be, and feeling emboldened to move on). This earlier stage of pure pining reached its peak in the mid-to-late 1970s with "September Gurls" by Big Star and "American Girl" by Tom Petty and the Heartbreakers, with an honorable mention to "Is She Really Going Out with Him?" by Joe Jackson.
And it wasn't just guys longing for girls. The '70s style pining song carried over into the early '80s among boy-crazy female singers -- "He's So Shy" by the Pointer Sisters, "Goodbye to You" by Patty Smyth / Scandal, and "Johnny Are You Queer" by Josie Cotton.
There don't seem to be as many pining songs in the '80s overall, though, and the ones that were made sound less inexperienced, for lack of a better word, compared to the ones from the '70s, whose uninhibited eagerness makes them sound more like the singers are anticipating courtship rather than having been through it many times already. The ones from the '80s aren't jaded or cynical, just a little more restrained or even reluctant -- not out of a fear of putting yourself out there, but simply from having experienced the eagerness-and-let-down cycle too many times before.
"Pulling Mussels (From the Shell)" by Squeeze bridges the gap between the earlier and later styles, while from the same year "I Got You" by Split Enz has that distinctly, subtly ominous undercurrent of so much '80s music. "In Between Days" by the Cure is part throwback to the exuberant '70s style, yet just gloomy enough for the experienced Eighties. By far the best example of this later style is "Tenderness" by General Public. In contrast to the uniformly eager songs of the '60s and '70s, this one heightens the tension and ultimate sense of release by swinging back and forth (effortlessly) between caution and abandon. So totally new wave.
Also in that vein, albeit in a more straightforward rock style, is "Hey Jealousy" by Gin Blossoms, one of the few, perhaps the only one worth mentioning from the early '90s, before the genre died off completely.
Where the yearning song looks forward to potential relationships, the bittersweet song also looks forward, but after the relationship has run its course. That's the down-side of taking a big risk -- most of them fall far short of what you'd hoped for -- and you just have to move on, while still acknowledging rather than denying what you shared.
And while some song forms may accomplish that, bittersweet songs do even better by creating an uplifting feeling -- beyond acknowledging the past, they treasure it, without getting so sappy that they remain stuck there. It's a sign of greater maturity of both the artist and the audience when they become a hit -- not pouting after a risk turned out less than perfect, even expressing joy at having had the chance to share the experience at all. When they vanish, it is a sure sign of our emotional stuntedness.
Because of its greater demands on maturity, I think this form takes longer to develop. From what little I've read about the early history of jazz, the bittersweet songs are from the 1920s and early '30s, not during its beginnings in rag-time. That seems to be the case with the last wave of great music: while they might be there, it's hard for me to immediately think of examples from the late '50s and '60s that are both sad and cheerful. I'm drawing a blank on examples from the '70s as well. It's not that they're not there, just not as ubiquitous as they became during the '80s.
There are two sub-types here -- where the sadness comes from the end of a relationship, and where it comes from the uncertainty that this great thing we've got going may not last forever. But hey, nothing risked, nothing gained, right?
The greatest example from the "current relationship" type has to be "Waiting for a Star to Fall" by Boy Meets Girl, nothing else comes close to its uplifting power and sincere tinges of sadness. Most of these are in the new wave / synth-pop styles, but this one is pure pop, complete with a kickass sax solo. Another stylistic outlier is a smooth jazz hit, "Your Love Is King" by Sade.
Before Madonna began slutting and butching up her image in the '90s, she released the vulnerable yet upbeat "Borderline". And if that's too adolescent for you, check out the similar but more mature song "Point of No Return" by Nu Shooz, better known for their new wave / disco hit "I Can't Wait". Also in that vein is "Secret" by OMD, which enjoys a restraint and cheerfulness that's a little lacking in their more famous "If You Leave". Roxy Music's "More Than This" is a great mellow dance tune in the genre, although if you're looking for something more bouncy, nothing beats "Temptation" by New Order (the 1987 re-recording for their Substance compilation album).
As for the "ended relationships" type, "Walk of Life" by Dire Straits and "Don't Look Back" by Fine Young Cannibals may not be like the others lyrically, but it's the overall feeling that counts. For that matter, the bittersweet loss song doesn't even have to be about a romantic partner. One of the most touching and uplifting is "Nightshift" by the Commodores, about two beloved singers who had recently died (Marvin Gaye and Jackie Wilson).
In more romantic contexts, "What Have I Done to Deserve This" by Pet Shop Boys succeeds more from its instrumentals and Dusty Springfield's singing than from the emotionally flatter main vocals. It's down enough and upbeat enough to work, if in a bipolar kind of way. More light-hearted is "Always Something There to Remind Me" by Naked Eyes. More aggressive is "Modern Love" by David Bowie. There's almost a resignation in "Everytime You Go Away" by Paul Young, whereas Whitney Houston's resolve carries her through "I Wanna Dance With Somebody (Who Loves Me)".
I haven't provided any links to YouTube since everyone knows how to use it. But just for fun, here are a couple that not everybody would recognize from those above. (Hard to find a clip of the second one that wasn't harshly loud or maxed out on bass, but that's why we should hear music on CD rather than YouTube.)