Go to your local hardware store, they carry cleaner for concrete. Most of the cleaners do have acid in them, this is the only way you will remove the rust? Muratic acid is the pretty popular one. The cleaner I used for my driveway had acid in it, but it cleaned the rust stains.
Don