From medieval castles to rejuvenated quays, Shilpa Ganatra finds a city with no shortage of cultural delights, immersive art, dining spots and Irish charm