Factory Butte is the most recognizable feature of a large area of stark, barren land either side of the Fremont River known as the Upper Blue Hills, bordered by Capitol Reef to the west, the Henry Mountains to the south, San Rafael Swell to the north and the San Rafael Desert to the east.