First built in 1880 near Fort Charles, the Giddy House was originally a Royal Artillery House meant to store weapons and gunpowder for the adjacent Victoria and Albert Battery. After Port Royal was struck by an earthquake in 1907, the Giddy House partially sunk. Its nickname, Giddy House, comes from the feeling visitors have when trying to stand straight while inside.