Highgarden is a castle. White Harbour and Sunspear are small towns.

Sunspear is described as pretty substantial, isn't it? Highgarden I can't actually remember how it's described apart from large gardens, so there you are probably correct. I would personally say that White Harbour is large enough to count, but that's really up to your definition, I suppose.

That seems fair. Possibly Sunspear is the third, though I've had the impression that it's all that large.

The wiki calls it "no more than a queer, dusty town." Make of that what you will.

The fact, though, that the entire continent, despite 2 very fertile large regions, only has like 3-4 cities, and not even that many towns apart from those, remains a weird piece of world building. Not as big a crime as the history of the setting, though, but still weird.

You'd expect the third city to be around Highgarden, since that's the centre of the most fertile region. Maybe there is a city there, though the wiki doesn't seem aware of it.