James Louis Petigru, late in life