10 Scientific and Technological Visionaries Who Experimented With Drugs. Same here.
We evolved along with, and because of, our diet. There weren't any classifications of "illegal drugs" back then, and these things flourish in all sorts of conditions in the wild - it's hard not to imagine at least occasional consumption. Considering mankind's predilection to experimenting with consciousness, along with the unique effects of these substances (including euphoria), it was probably more than occasional.
If these fungi were part of a diet over time, they'd likely have some effect on our development. How to increase serotonin in the human brain without drugs. Different stages of drugs explained with trees.