Several houses formally stood on this site during the Middle Ages and in the late Middle Ages they were replaced by the Werboczy Mansion. During the Turkish occupation, or at least from the end of the 16th century, it functioned as the palace of the pashas of Buda (Seraglio). Following the re-occupation, the land complex was first obtained by the Jesuits, than by the Carmelites who completed the construction of their monastery on this site by 1734. Following the dispersal of the Order in 1784, the monastery was converted into a block of flats, then later into a casino and after that into a military headquarters. Albeit in various forms, the military headquarters managed to preserve its function right until 1945. Following the siege of World War II the building suffered heavy damage. After its reconstruction (1976-1977) the building hosted various institutions. 

The rectangle building has an enclosed courtyard and a projecting block, - the former refectory, - is situated at its nortern angle. Originally, only a wing with a single corridor used to face the street. Following the dispersal of the Carmelite Order, the originally Baroque and Baroque Classical building was rebuilt during the 19th century in neo-Classical style. This is when the street section was expanded into a two-floor one, and a new gate was established. The neo-Classical style nineteen bays long main front is divided by a string-course and a sill, and crowned with an entablature. At each angle of the building there is a three bays long projection. The flat arched gate between Doric pillars is set in the middle axis. The eastern façade facing the castle has preserved its Baroque features, door and window frames. The building's northern facade was also constructed in the Baroque style. Its large court often provides the venue for various cultural events.