Castles cannot be damaged by spells - they are normally, though, protected by defensive spells, prayers, runes, and their massive size! Generally speaking, spells are best targeted at troops defending the walls.