Kalpitiya was known by the ancient Tamils as Kav Putti and was a popular hub for Arab merchants. In 1544 the Portuguese invaded the area, naming the area Kardiv Island. The King of Portugal bestowing the territory to the Jesuits, who built a chapel here and established a small garrison to defend it.
In an effort to free the port from the Portuguese the King of Kandy, King Rasjasinha II, sought assistance from the Dutch, who in 1659 conquered the area but did not return it to the King. Instead, they commenced construction of a fort in 1667, which was completed in 1676. Kalpitiya was strategically important for the Dutch East India Company (VOC), as it enabled them to control the external trade of the Kingdom of Kandy, by controlling the large Muslim trading community in the area.
The Dutch fort is almost square in shape, with walls about 4 mtrs (12 ft) high, constructed out of coral and limestone from the surrounding area. It has a single entrance, which fronts the lagoon, has a pediment, with a belfry above and looks like the entrance to a church.The yellow bricks that comprise the entrance arch were reportedly brought especially from Holland. The story is that the King of Kandy had granted permission only to build a church here and that the Dutch had built the arch to mislead the King into believing that this was simply a fortified church.
The Fort has four bastions on each corner, each with its own guard post, two smaller bastions facing the lagoon. Inside, the buildings are located around the periphery creating an empty space in the middle of the fort. The walls of a chapel, barracks, dining hall, commander's residence and prison are still evident, though the roofs have been replaced. There were two tunnels leading away from the fort — one led to the sea and the other to the Dutch Reformed Church approximately 400 mtrs (1,300 ft) outside the fort. These tunnels are blocked and inaccessible.
In 1795 the fort was surrendered to the British. The British continued with their military occupation of the Kalpitiya Fort till 1859. The Kalpitiya fort was neglected for many years until it was turned into a Sri Lankan Naval base for training.