Some substances are harmful even if used once, like LSD that can take people on hallucinogenic terror trips from which they never fully recover. Young, non accustomed people drinking in binges are prone to ethyllic coma, which can progress to death if undetected and the person is left to "sleep it off."Just because a substance is harmful if abused doesn't mean it should be illegal, nor should it's users be treated as societal anathemas.
It's not a black and white issue, few things in life are.
In most traditional cultures, mind altering substances were used in a very controlled environment, presenting as a religious/spiritual ceremony and these have common features: The person using the substance was prepared, monitored, and the others involved had some familiarity with the adverse effects as well as some clues on how to deal with them. Yet these events were known also to have significant risk. In any case, it is definitely not something that can be qualified as "recreational." To the contrary, it is a dead serious, life changing event. The casual attitude of Westerners toward the substances indicate how much they miss the point of the whole experience: there is nothing casual about it, it is a spiritual experience lost on one who is not spiritual to start with, the substance is only part of the story, and it is a social event, happening within a group. The safety of the user lies with the group, as well as the usefulness of the experience. I recommend reading on the Bwiti tradition of Central West Africa, which uses the Iboga root.