St Ives in Cornwall will hold a referendum in May asking its 12,000 residents whether they agreed to a ban on building houses except where they are intended to be the principal home.
The vote comes after the town council revealed that a quarter of new homes were being used as second homes.
Prices have risen due to the high number of holiday homes, and locals are increasingly finding the area unaffordable, Lucy Davis, a neighbourhood plan coordinator, said.