Before the serial drama became a popular genre in television, it was not possible to do a serious "Halloween episode" because the dominant genre was the sit-com. Ghosts, spirits, and the like would have clashed with the light and comedic tone of the series overall. So, on hit sit-coms like Roseanne and The Simpsons, the Halloween episode took the form of the characters playing morbid pranks on one another and telling scary stories.
Then came series that follow fantastic plotlines and themes on an ongoing basis, such as Buffy the Vampire Slayer and Supernatural. The presence of ghosts, spirits, etc., on Halloween would actually feel ordinary in the worlds that these shows are set in. Bummer.
To get the Halloween atmosphere right, the series ought to be naturalistic, so that the Halloween episode stands out as up-ending the ordinary order of things, just as it is supposed to be in real life. We've already seen that it needs to be a drama rather than sit-com, so that the tone of the Halloween episode won't clash with the usual tone. And the narrative should be serial, so that we can tell that there's a disturbance to the ordinary goings-on with the characters whose lives we already know.
The Halloween episode I remember most is the one from My So-Called Life, which features a plotline about a ghost of a former student at the high school who died on Halloween. It is played seriously, as though something supernatural has entered the mundane world of the series, which usually focuses on typical teenage drama.
In fact, for their Christmas episode they again introduced a ghost plotline that was played seriously (the ghost of a homeless teen runaway, not the Christmas Carol kind of ghost). Both ghosts appear in ordinary human form rather than transparent and wispy, which helps the viewer of a naturalistic series to suspend disbelief.
Judging from reviews of these episodes at the Onion AV Club, hardcore nerds can't stand the serious introduction of supernatural elements in a realistic drama. I don't remember these episodes having a credibility problem back then, nor when I re-watched the series a few years ago. There's nothing unrealistic about the occasional encounter with something that can't be explained naturalistically. Haven't we all had some kind of experience like that in real life? And what better time to set it in than during holidays where there's a carnivalesque atmosphere of "up is down and down is up"?
If ghosts, spirits, etc. only made appearances as recurring characters in a fantasy world, they wouldn't stand out as beyond the ordinary. The nerds who write that kind of stuff (e.g. Joss Whedon, Buffy the Vampire Slayer) seem to think of ghosts, vampires, etc. as like invisible friends who interact with the normal characters on a regular basis.
In reality, ghosts are something that we only encounter very rarely, perhaps only once or twice in a lifetime. And those encounters do seem to take place within our mundane lives before and after, not as though we feel transported to some fantasy world where ghosts dwell, or as though some portal has opened up into our world like a horror movie. They have an unsettling surreal quality, where we can't tell if this is a natural or supernatural experience.
Have there been any similar episodes in the 20 years since the ones from My So-Called Life? Wikipedia has a list of Halloween TV specials with sections for "American drama" and "teen drama". It's hard to judge whether ghosts appear at all, or whether it's just about partying on Halloween, let alone whether the ghost encounter is portrayed seriously. It sure doesn't look like it, though.
Earlier, Twin Peaks made surreal episodes -- every episode. Expecting them every day or week is not how we have ghost-like experiences in real life.
In any case, you can watch the entire sole season of My So-Called Life for free on Hulu. The Halloween and Christmas episodes are 9 and 15, although it's worth watching the other ones in order, too, to appreciate the contrast in subject matter and theme. It's one of the few pop culture phenomena of my teenage years that I'm not embarrassed to have enjoyed, a kind of Breakfast Club serial drama for the grunge/alternative era. There's certainly a lot worse you could be streaming to get in the mood for Halloween.