I thought that was the whole point about being an assassin.Unless of course you hide and sneak up on them
Just remove the helmets with see invisibility charges etc. and things become fun again.
And it is true, keeping int and dex high is hard. But the same problem poses itself with paladins, CoTs, bards and monks.
I think most people would be able to cope with a reduction in the duration of freedom of movement, so it becomes a combat buff in stead of a buf you cast when you wake up and which expires when you go to sleep again.
That way they're able to save themselves when they see you, but are in serious trouble when they don't.
And the casters get to use their fun spells again.