The Birthday Party: A little bit more anarchic than the Bad Seeds or his solo stuff. Also a bit less retro, a bit more post-punk. The Mutiny Sessions is a bit more... I don't know... palatable? It's loud and fast, but not nearly as strange as Junkyard, but both of them are really great. If you like craziness, you'll love this stuff.
I love all of The Firstborn Is Dead. I actually dig a number of the songs on Nocturama, even though most people don't like it. Also, check out Henry's Dream. I like it, at least.
The Bad Seeds also appear on the Soundtrack from Until The End Of The World. "(I'll Love You) Until The End Of The World" is their only song on the thing, and I didn't ever really like it, but then I had a girlfriend who loved it, so maybe it's sentimental, or maybe I actually started to like it for itself.
Songs For A November Night is great. Also the stuff that other people said. "The Mercy Seat" was covered by Johnny Cash, and it's just as good as the original one.