The first knight from the Schade family to be mentioned in writing was Antonius Scathe in 1238. The noble family from the duchy of Westfalen split early on. Schade-Salwey was one of many lines and had probably been resident in the area since the 16th century. Around 1540, Jost von Schade came to Obersalwey, where his wife probably already owned property. The original castle was built in the same year. In 1820, Wilhelm von Schade gave it its present form. However, the building burned down shortly afterwards. His family had the castle rebuilt according to this model. The estate remained in the family until 1867, after which it changed hands several times. After the Second World War, the building was used for various purposes. It has been a listed building since 1984 and was renovated the following year. Today, Salwey Castle belongs to the Count von Spee family.

The two-storey castle Obersalwey has a rectangular footprint and is brightly plastered. Its high crippled hipped roof is covered with slate shingles. Several small dormers are mirrored in the roof. The former manor house has an axially symmetrical design. A double-leg staircase leads up to the central main entrance. Above the simple entrance door is a triangular fanlight and the window of the central window axis. A small triangular gable completes this axis. To the right and left of the central axis are 3 further window axes. To the north-east is an attached round tower with a polygonal pointed roof. The weather vane probably dates from the 17th century. The castle is framed by two farm buildings. One of the two-storey half-timbered buildings in front with a continuous transverse roof and passages may have originally been an outer bailey.

At first, it was probably Jost von Schade and his wife Maria von Cobbenrade who built a castle in Salwey. The estate remained in the family until 1867, when Friederike von Schade sold it after the death of her husband. The castle changed hands several times through sale and inheritance: during this time, it belonged to the von Rump, Landsberg-Velen-Gemen and Wrede-Meschede. In 1908, Anna Wrede married Count Stephan von Spee. The von Spee family still owns the castle today. After the Second World War, the von Spees' forestry administration was housed here for a time. After the manor house was no longer needed by the family, the interior was rebuilt and renovated in the early 1960s. The private brewery Iserlohn leased the castle and used it as a hotel. It later stood empty for several years. After the renovation, a studio for advertising photography moved in in 1985. Today, the castle is used privately.