During the administration of President James Madison, the White House was burnt by the British on August 24, 1814 - during the War of 1812. The only thing to survive was the exterior walls and the Gilbert Stuart painting of George Washington saved by Dolley Madison. Afterwards, many members of Congress wanted to move the nation's capitol elsewhere, but James Madison insisted upon the rebuilding of the White House.