In the absence of documentary sources, the origins of the building are unknown but it seems linked to the lords of Monteil, vassals of the counts of La Marche. They will always keep the surname of their extinct family (the Aubussons) whose title will continue to designate the family castle of Monteil au Vicomte.
This imposing castle was the hub of an important family heritage made up of land, mills, ponds, forests and feudal royalties, as well as the rights of justice over the inhabitants.
Victim of the Hundred Years War, and after several restoration campaigns, it was sold as a National Good during the Revolution. Its stones are then widely used in different constructions of the town, a frequent practice.