Wealthy Flemish financier Henri Joseph Stier of Antwerp fled Europe with his family during the French Revolution. In 1799, Stier's younger daughter, Rosalie Eugenie, married planter George Calvert of Prince George’s County, a descendant of the fifth Lord Baltimore.
In 1800, Stier bought 729 acres near the port of Bladensburg and commissioned the building of Riversdale. The Stiers moved into their partially built house in 1802.
In 1803, Mr. and Mrs. Stier and their older children returned to Antwerp. George and Rosalie Calvert moved into Riversdale and completed construction in 1807. The correspondence between Rosalie Calvert and her family survives and is the basis for Mistress of Riversdale: the Plantation Letters of Rosalie Stier Calvert, edited by Margaret Law Callcott (Baltimore: Johns Hopkins University Press, 1991). These letters provide a rich source for the restoration and interpretation of the house. Mrs. Calvert never returned to Europe. She died at Riversdale in 1821 at age 42 having borne nine children; five lived to maturity. George Calvert did not remarry and died in 1838.
Charles Benedict Calvert, the Calverts’ son, continued living at Riversdale. A progressive farmer, he founded the Maryland Agricultural College (now the University of Maryland, College Park) and, as a U.S. congressman, sponsored legislation establishing the U.S. Department of Agriculture. He died in 1864.
Riversdale's success was dependent on the labor of enslaved African-Americans such as Adam Francis Plummer. Born into slavery in 1819, Plummer was moved to Riversdale at the age of ten. Unlike most enslaved workers, he could read and write, and in 1841, shortly after his marriage to Emily Saunders Arnold, he began keeping a diary. This diary is one of the few first-hand accounts of slavery and emancipation in Maryland.
Following Charles Benedict Calvert’s death, Riversdale began to decline. In 1887, the estate was sold to developers who used the house as their headquarters as they built the commuter suburb of Riverdale Park. The mansion served as a boarding house and a country club before being purchased by William Pickford. The house was leased to U.S. Senator Hiram Johnson, former governor of California. In the late 1920s, U.S. Senator Thaddeus Caraway of Arkansas bought the house. Caraway died in office and his widow, Hattie, went on to become the first woman elected to the Senate. The last private owner of the house was Abraham Walter Lafferty, who lived there until the 1940s.
In 1949, Lafferty sold the property to The Maryland-National Capital Park and Planning Commission, and Riversdale was used as office space. Restoration began in 1988 and the house was opened to the public as a museum in 1993.