Roy is right. The rubbers fitted to the inner clutch plate are only there to dampen the plate and reduce noise.
If the torque arm fix doesn't cure it have a look at the clutch steels and make sure they are flat. If not they will produce a little drag and can make the box clunk into 1st.
_________________
If I have to take the carbs off once more...