According to historian Anthony Pitch, in The Burning of Washington ... the British neared the White House, Dolley Madison directed that a Gilbert Stuart portrait of George Washington be removed.