The tiny island seen in the middle distance in this photo of Boothbay Harbor was so covered with buildings when this photograph was taken around 1910, it almost appeared to be just a cluster of buildings floating atop pilings. The photographer stood on Mt. Pisgah (a tall hill in town), looking southwest out over the harbor, to achieve this view.

Known successively over the years as John's Island, McFarland's Island, and Sawyer's Island, it bore the names of owners of the point of land just to the right of it. At the time of this photograph, Melvin Sawyer (b. 1865) was using the island as a fish facility, with flakeyards, fish processing buildings, smokehouses, oil storage areas, a long wharf, and a windmill to pump salt water.

For many years, Melvin's father Billy and then Melvin's brother, also Billy, had used the island as a spot to strand boats for his wrecking and salvage business. The Sawyer family referred to the island as Little Island by 1900.

Sometime after 1933, when artist Fred Marsh transformed the island into a summer retreat, it acquired its current name, Harbor Island.