Like many religious/spiritual movements, Occultism manifested as a reflection of it's time.
The Occult movement of the late 1800's and early 1900's borrowed heavily from psychology. When they wrote of learning astrology, it was with the understanding that planetary associations were models of psychological traits and states. Indeed, everything that they studied – qaballa, alchemy, or tarot was understood through this filter.
One of the primary aspirations of the rituals was knowledge and conversation of your holy guardian angel. This was interpreted as coming into contact with one's inner genius, or higher self.
At the time of the Spiritualist trend, psychology was a new discipline, and like all things new, it was attractive to intellectuals. At the time, many of the British educated class found a lot charm in Occultism. It related to the Spiritualist trend, without the stigma of the mediums and palm-readers. The latter would have been seen as soothsayers to the superstitious, whereas occultism was seen as a self-driven act of psychic ascension.
Since the Occult sciences fit so well into the psychological models, it's easy to believe that earlier practitioners really had an understanding of the human mind, and the manipulation of it. But this is what I want to believe, so I'm inclined to be extra-critical, so as not to get emotionally swept away by facts that may or may not fit the beliefs. At the end of the day, Alistair Crowley said it best: “The truth of the myth is not so important as the affect the story has on the believer.”