On the site of the building, two parallel, rectangular plan army post buildings facing the bastion were constructed soon after the liberation from the Turkish occupation. The cope vaulted stable was built at the turn of the 18th and 19th centuries. In 1819 the two two-storey garrison buildings were jointed at their eastern end. By using up some of its parts, the outdated, penurious building complex was replaced in 1847 by vertically placed two-storey neo-Classical-style wings, one alongside the still existing former Bastion (today: Tóth Árpád) Lane, the other looking onto Kapisztrán Square. This building complex, which gave home to the Nándor Garrison, was again expanded by the construction of another wing along Bastion Lane in the 1880's. 

After World War I the building was given a new function. The Military Museum and Military History Archives were originally set up in 1918 and located in the Maria Theresa Garrison in Ülloi út in Pest. Later in 1929 it was removed to the wing of the Nándor Garrison, which faces the Vérmezo ("Blood Field"), the wing previously reconstructed for this purpose (between 1926 and 1929). The museum opened its doors to the public on May 29th, 1937 and ever since the collections of the museum and the archives have been continually growing thanks to donations and purchases both in Hungary and abroad. The Institute and Museum of Military History is a public collection performing scientific and public educational duties with a national sphere of authority. 

Both the building and its collection were extensively damaged during World War II. After a partial rebuilding the museum's first exhibition opened in 1949, however, works lasted for another two decades.