The oil pump idler pulley setup is different on the 8BA from the earlier 32-48 engines. Frank Oddo in, "Ford Flathead V-8 Builder's Handbook 1932-1953," recommends using a drift punch to drive the shaft out from inside the block. George McNicholl in, "How to Build Ford Flathead V-8 Horsepower," recommends using a 3/8 UNC bolt 1 1/2" long and a thick flat washer to pull against using a slide hammer with jaws to pull against the flat washer. Good luck with your project.
I don't think you necessarily need to remove it. If it rotates freely with no discernable wobble, and the teeth are in good condition you could leave it in place.
Just my take on it.
Mart.
Do yourself a favor and buy a slide hammer. You'd be done by now if you had one.