Brown was born roughly some time during the 1930s. She was born at Purpurna, a waterhole that is sacred to the Pitjantjatjara. She grew up living a traditional, nomadic way of life in the bush with her family, before any contact with Euro-Australian society. In the 1950s, her family was moved out of the bush to live at Warburton, with many other Aboriginal families. Warburton was a Christian mission at the time, and Brown was taught at school here by missionaries. When she was older, Brown moved to Irrunytju and married Nyakul Dawson.