My tool of choice: a DeWalt Cut-Off Grinder tool, with a well-used blade. (they are normally much wider).  Good for removing LARGE amounts of putty, and to smooth over the grip's finish.  CARE MUST BE USED WITH THIS POWER TOOL!
When using this tool, I suggest you hold the tool stationary on your workbench and moving the grip against the rotating blade.  This lets you control the pressure and the amount removed or polished.  You can use the flat portion or the edge of the blade.
Click on image to see album.