A couple years ago I found that the plastic housing (distributor body) I had put on the distributor did not fit tightly and the gap was different depending on which way I pushed on the housing. I found a housing in my collection that fit tightly and that gave uniform gap on all contacts. (I am not sure if contacts is the correct name as the rotor is not normally contacting them.)
After that I set the gap by bending the brass part on the rotor until I had a very small gap. The specification is for 0.025 gap but I found that my car would run a little better with a smaller gap. I ended up with about 0.010 gap. That will only work if there is no slop in any of the parts: rotor, housing, or distributor shaft.
I suggest examining all of the parts, looking for a cracked rotor, as Jay Jay said, or worn bearing in the distributor, as ndnchf said, or a loose housing.