Born in Charlottesville, Fountain was a descendant of Wormley Hughes and Ursula Granger his grandparents had been between the residence slaves owned by Jefferson at Monticello. Jefferson permitted two of Sally's young children to leave Monticello devoid of formal manumission when they arrived of age 5 other slaves, such as the two remaining sons of Sally, have been freed by his will on his death.