If you pull outward to some extent, it will want to fall but get stuck because the starter drive gear mechanism hits the flywheel ring gear. So, if you push at the side of the starter while simultaneously pulling the starter out until the gear mechanism clears, you can get the end of the starter shaft out of it's cubby hole in the bellhousing. At that point, you've got the heavy end of the starter practically sandwiched between the big frame crossmember and the engine block.
If I remember correctly, seems like if you tilt the shaft end of the starter downward while pushing the heavy end of the starter in the opposite direction (back toward the transmission), you can worm the starter out through the bottom. I never needed to turn the front wheels any certain direction and it does not matter how high the car is lifted up, you can still get it out the same way.