You can take a LOT of weight out of the flywheel and still have a very good driving Model A. Actually, it will feel a little more "Alive" and still Idle just fine. If you have a countered weighted Crank, I would definately lighten it. Just be sure to have it balanced AFTER you do any machining.
I believe Les Andrews gets into this in one of his books.
Chris