If you’re familiar with the Hamptons’ upscale vibe, you’ll soon realize that, though it’s in the Hamptons, Sag Harbor is not quite as “Hampton-y” as its other destinations. On the South Fork’s north coast, this historic hamlet has a strong maritime flavor that stems from its history as a whaling port. The first white settlers arrived in the late 1600s, learned a thing or two about whaling from the resident Native Americans, and started sending out whaleboats in the mid-1700s. By the time the industry hit its peak in the mid-1800s, Sag Harbor had become one of the world’s busiest ports.