Problem solved ( i think) the cable is stretched, ive adjusted it numerous times but always ends up with play. Ive had the cable too tight to the point its slipping when the lever is released then within a short time it can be loose enough its not disengaging when changing gears.

Still not sure why it only affcts the lower gears on downshifting only tho? Im guessing its because you can upshift without the clutch?