Baba’s earliest years at Shirdi were passed in complete obscurity. Baba had no home; Fakirs have none. Hermits are, therefore, to reside either at a tree-foot or chavadi or temple or other public places.
When he came into Shirdi, first Baba visited Khandoba temple at the outskirts of the village, and then noting how solitary and calm a place it was, exclaimed, ‘What a nice place this, for ascetics like me to live in.’ Mahlsapathy, the man then in charge objected to this observation and said that no Mohammadan would be allowed to put his foot into the Khandoba temple.
He was evidently thinking that Baba was a Muslim and that he would break the images and defile the temple. But Baba was just the opposite.
"I draw to Me, My man from far off or even across the seven Seas to Shirdi, like a sparrow with a string fastened to its feet"-Shirdi Sai Baba, Indian Spiritual Guru-God 