Personally, I think Fold Your Hands Child You Walk Like a Peasant is B&S's nadir. That album is a mess. After Stuart David left the band, they suffered. Dear Catastrophe Waitress redefined their sound, but it definitely wasn't "classic" Belle and Sebastian.
They suffered after Isobel left, as well.
I have this theory about bands losing it after their original bassist leaves, but the only examples I can think of off the top of my head are B&S and Weezer.