When Peyroux was six, her family moved from Georgia to Brooklyn and later southern California. At age 13 her parents divorced, and she moved with her mother to Paris. In interviews, Peyroux describes her parents as "hippies" and "eccentric educators" who helped her pursue a career in music. As a child, she listened to her father's old records and Peyroux learned to play her mother's ukulele.
Peyroux started singing at the age of fifteen and performed with street musicians in the Latin Quarter in Paris. She joined a group called the Riverboat Shufflers. At age sixteen she joined The Lost Wandering Blues and Jazz Band and toured Europe with them.
