Judd was born in Excelsior Springs, Missouri. He served in the Army from 1946–1947 as an engineer and in 1948 began his studies in philosophy at the College of William and Mary, later transferring to Columbia University School of General Studies. At Columbia, he earned a degree in philosophy and worked towards a master's in art history under Rudolf Wittkower and Meyer Schapiro. At this time he also attended night classes at the Art Students League of New York. He supported himself by writing art criticism for major American art magazines between 1959 and 1965. In 1968 Judd bought a five-story cast-iron building, designed by Nicholas Whyte in 1870, at 101 Spring Street for under $70,000, serving as his New York residence and studio. Over the next 25 years, Judd renovated the building floor by floor, sometimes installing works he purchased or commissioned from other artists.