I rely on #0000 steel wool and 3-in-1 oil, working just the area with the 'active' rust with just enough pressure to remove it. I then wipe it dry with a paper towel, wet another one with 3-in-1, and wipe the treated area until I see no more red on the paper towel. Sometimes I have to revisit the spot with the #0000 (again in conjunction with the 3-in-1) and repeat the process.
#0000 steel wool is equivalent to somewhere between 600-grit and 1000-grit sandpaper, so with oil your safe.
You would have to try really, really
hard to remove patina.