Both his grandfather (Clarence J. Brown) and father (Clarence J. Brown Jr.) were United States Representatives from the 7th District of Ohio. Clarence J. Brown served from 1939-65 and Clarence J. Brown Jr. from 1965-83. Both men also ran, unsuccessfully, for governor of Ohio.