The Royal Watermen's and Lightermen's Almshouses were built to provide accommodation for retired freemen of the Company of Watermen and Lightermen and their widows. In 1839 two acres of land, including a well, were presented to the Company by wealthy Penge landowner John Dudin Brown, and Queen Adelaide became patron of the new institution, donating 100 guineas towards the cost. The first stone was laid in May 1840 by Lord Mayor of London, Sir Chapman Marshall, and in 1841, 34 married men, 10 unmarried men, and 32 widows became the almshouses' first residents. It remained a retirement home for the Company until 1973, when a new home was built in Hastings.

By that time, the cottages were too small, without bathrooms, damp, and in a poor state of repair. Demolition had been threatened since 1963, with proposals for modern flats and commercial development. But the Greater London Council, then responsible for housing capital-wide, caused a preservation order to be placed on the site and did major refurbishment, knocking pairs of cottages into one as privately-owned and housing association residences. Further renovation work was done in 2010/11.

The gardens, at the north end of Penge High Street, are open to the public, access being through the gates next to St John's Church and at the corner with Penge lane, or the driveway further down Penge Lane. The buildings are not open to the public.

Anerley Vestry Hall was built in 1878 at a cost of £4,341 for conduct of public business of the district, then part of the Borough of Lewisham. It became the Town Hall when Penge Urban District Council was formed in 1900 by the London Government Act of 1899. It was considerably enlarged in 1911 at a cost of £3,229 and contained offices, the council chamber, committee rooms, a public hall, and a petty sessional court opened in 1925.

It was used by Penge Council until 1965 when the area became part of the new London Borough of Bromley. It now houses Anerley Library, a playgroup, and other amenity groups, and is thus the longest-serving public building in the area.

The photograph was taken in the 1930s, but the building is little changed today. One of the rails for the trams which ran up Anerley Hill to the Crystal Palace can be seen in the foreground.