The veterans of the Fifth Maine remained in close contact upon their return to civilian life and, in 1888, constructed the Fifth Maine Regiment Memorial Hall as a memorial to their deceased comrades and as a reunion hall for themselves. This Queen Anne style cottage boasts memorial windows in the main hall crafted of hand-blown flash glass and engraved with the names of men who served in the regiment. The red cross above the doors on the oceanside of the hall memorializes the Sixth Corps whose insignia was a red cross.

The hall contained fifteen "sleeping rooms" that were rented out to the men and their families during the summer months. A common dining area and several kitchenettes were located on the lower level. Water was pumped into a kitchen sink, a two-hole privy was located in the back yard, and lighting was provided by the carbide gas plant on the island. Modern conveniences such as indoor plumbing and electricity were installed as soon as they became available.

Since 1954 the hall has been cared for by the Fifth Maine Regiment Community Association, an all-volunteer group dedicated to preserving this unique building and what it stands for. In 1978 it was listed in the National Register of Historic Places and, in 1984, was designated as a Greater Portland Landmark.