Walterboro Library Society Building is a historic library building located at Walterboro, Colleton County, South Carolina. It was built in 1820, and is a small, white, Federal style frame building with a side-gabled roof. The front façade features a Palladian-style door surround capped by a fanlight. The building is occupied by the Colleton County Historical Society.