The pin has a knurled section that fills the top tab of the hinge. That's where the rust/crud/binding is taking place.
I ended up grinding the top cap off the old pin, then using a series of smaller-larger drill bits down the center of the pin to collapse the top part of the pin. Then drift it up from the bottom.