Andrew Knox was born in England in 1835, and settled in Danbury, Connecticut, sometime before 1860. In that year, he married an 18-year-old named Sarah. Knox was working as a painter when the Civil War broke out, and in April 1861, he was mustered into Company E, 1st Connecticut Volunteers, served his term of three months, and received an honorable discharge. He re-enlisted and was mustered into Company B, 1st Connecticut Volunteer Heavy Artillery on May 12, 1862, at the rank of second lieutenant, but was promoted to first lieutenant 11 days later.
Along with his regiment, Knox participated in the battles of Yorktown, Gaines’ Mill, Hanover Courthouse, Malvern Hill, Chancellorsville, and Gettysburg. He survived the war and returned to Danbury, where he continued to work as a painter. By 1880, James and Sarah had six children living at home.