It all has to do with the way you set it up. Electrolysis is used to derust iron cannon quite successfully. It's a standard method for conservation of iron objects from salt water.
It's more or less a line of sight deal. The current will travel from the cathode to the anode. The larger the anode, the more rust you will remove. One piece of rebar used as an anode will result in rust removal on the surface of the part facing the anode. For successful rust removal, you want to surround the part with the anode. I've used a large number of pieces of rebar wired together, but chicken wire works well. The big conservation labs will use the a big tub or vat as the anode - picture a garbage dumpster full of water with the parts in it. You want the part surrounded by the anode for best results.
You also need an electrolyte in the water. I've always used washing soda.