In addition to the exhibition of “Historical Manual Fire Pumps", a Lapidary is located in the stables of the "New Palace". After basic repair and reconstruction it reopened in the fall of 1992 with the permanent exhibition of “Architectural Sculptures of the 16th Century - Baroque Graves of the 18th Century".
The 16th century is represented by a portal and several sculptural fragments of the former Royal Palace Neideck. This Renaissance castle was built between 1553 and 1560/65 on behalf of Count Günther XLI. ("The Warlike") in Arnstadt under the guidance of the Dutch architect Erhard van der Meere. Used as count’s, later royal residence until 1716, during the second half of the 18th century, the castle slowly began to decay.
In 1772, after a partial collapse of the castle, large parts of the building were demolished. In the exhibition, architectural components used for the installation of an artificial ruin in 1811, can be seen. The graves from the 18th century are almost without exception from the "Old Cemetery" in Arnstadt. Before closing this burial site in 1894 Arnstadt started with the rescue operation and placement of these valuable grave stones and sculptures in the cemetery church of the "Old Cemetery". Reconstruction and renovation work on the “Church of the Ascension” in 1980 made a rearrangement of the graves to the Castle Museum Arnstadt necessary.