They’re hard to get out because, if they had a loose fit, it would be problematic to hold the grease gun fitting tightly enough over them to ensure the grease would not gush outside the rim of the fitting.
The best way to remove these original grease fittings is to hit them from side to side with the blunt end of a flat chisel with a decent size hammer behind the chisel to loosen them a bit and then angle the chisel at the tip of the grease fitting in the next blow (blows) to push them out of their mounting holes. This does take practice, but it works in most cases without removal by drilling and the chance if damaging the mounting hole.