if you use a drift that's made of something thats similar in hardness to the aluminium castle nut (or softer) - like a thin piece of scrap aluminium bar for example then you won't hurt anything.

I think its abit like parting the crank cases for the first time when they're abit reluctant to split - you've just gotta be careful but confident.

That said, there's nothing like having the proper tool for the job sat in your tool chest!
_________________
After years of moaning about immigrants now i am one...