I've also heard of ecstasy "burning out" seratonin receptors in heavy users' brains, which won't kill you directly, but seratonin is a pretty rad thing to have. If your brain can't process it any more, that completely sucks for you. It is like you get depression all the time.
Of course, this is mostly anecdotal evidence, as I haven't kept up with my friends from college who used ecstasy/cannabis all the time. I graduated and they didn't and then we kind of lost track. They certainly did have good times on drugs, though, and I got to tell them what they had done the next day. Win-win!
I'll bet that if LSD/e was produced legally and subjected to the same standards other drugs are held to, they could be safer than, say, alcohol (though I wouldn't want to share the road with them either). Amsterdam pretty much has managed to survive without outlawing cannabis. Like everything else, it's usually not a problem until it becomes abuse.
And to paraphrase Bill Hicks, if you manage to kill yourself while on marijuana, we didn't need you anyway.