After the previous rectory fell into disrepair, Pastor Bonifatius Populo had a new one built on the same site at the beginning of the 18th century. It comprises 10 rooms, including the so-called Berghauser and Fredeburger Stuben. In the past, older pastors from Berghausen and Fredeburg were cared for until their death. A farm building, which has since been demolished, provided space for tenants and animals. Today, the parish hall built in 1979 stands there. During the Second World War, the then pastor Rüsing set up a dressing station; wounded soldiers were also cared for here. When the American troops Wormbach conquered, they first stormed the rectory: they probably suspected a resistance nest. Due to fierce resistance from German SS troops, the Sauerland was particularly hard-fought at the end of the war, so that village areas were also the target of air raids. Over 100 windows of the rectory were broken.

Built in 1717/1718, the rectory is a stately eaves building with a high, slated hipped roof. The solid brick first floor is plastered and supports a simply structured half-timbered upper floor, which is slated to the west. While all the openings on the first floor have a segmental arch, the windows on the half-timbered floor have profiled, colourfully contrasted lugs on the lintels. The corners of the house are constructed with K-struts. The timbers above the first floor are carved and painted. The house originally had 10 rooms. New pastors often adapted the layout to their own needs and the spirit of the times. Despite many alterations, some of the original baroque doors have been preserved. The first extensive renovation took place in 1890, with the last major building work amounting to 500,000 euros in 2015. The grounds were renovated in 2020/21, including paths, beech hedges and the edging around the fountain.

In addition to the current pastors, old pastors and those in need of care lived in the Wormbach vicarage time and again. During the Second World War, wounded German soldiers were cared for in the rooms. After the war, refugee families moved in and lived here until the early 1950s. Pastor August Rüsing left his mark on the parish and the town long after his death. He was in office from 1937-1968, from 1947 as dean. At his suggestion, wooden crosses were hung on the lime trees around the church. They are intended to commemorate the soldiers from Wormbach remember them. The uniform roofed wooden crosses in the cemetery were also his idea. In this way, he wanted to make it clear that everyone is equal in death. Today, the parish office, the parish archive and the offices of the parish clerks are located in the basement of the rectory. The upper floor houses the pastor's apartment and the Bible workshop.