Oppenheimer an haife shi ne a Birnin New York a ranar 22 ga Afrilu, 1904, zuwa Julius Oppenheimer, wani mai shigo da kayan masarufi na Yahudawa wanda ya yi hijira zuwa Amurka daga Jamus a 1888, da Ella Friedman, wani mai zane. Julius ya zo Amurka ba tare da kudi ba, ba tare da karatun baccalaureate ba, kuma ba tare da sanin harshen Ingilishi ba. Ya sami aiki a kamfanin masana'anta kuma a cikin shekaru goma ya kasance mai zartarwa tare da kamfanin. Ella daga Baltimore ne. Oppenheimer sun kasance Yahudawa ne na Ashkenazi. A 1912 iyalin sun koma wani ɗakin a bene na 11 na 155 Riverside Drive, kusa da West 88th Street, Manhattan, wani yanki da aka sani da gidaje masu kyau da gidaje.
shekaru uku