Didn't see those parts... Yep, that's the pickup shoe. For sure I'd get a new one of those for better current pickup. New one should have spring.
Evapo-Rust is much better than Naval Jelly. Removes a lot more rust and is easier to use. Instead of a paste it is a chemical bath. You just keep soaking the parts until the rust is gone. I'll soak a part until the black residue builds up, pull it and scrape the black stuff off then soak it some more until it's "done". Maybe switching the parts to fresh fluid, it does wear out as it converts the rust.
Don't know what actual chemical is the active agent and there is another brand of the same stuff who's name I forget now... Found out about if from somebody over on the 3-rail forum. Can't live without it now!
http://www.evaporust.com/Don't forget to use gloves.