Your handles are held on to a square sided shaft with a cotter pin. If you have the spring loaded escutchions, compress it and reach behind and remove pin. I use a putty knife to remove panels. I slide it in as far as it will go and gently work the retainer clips out of the panel in the inner door. These clips might be rusted and patience is required to get them out without doing damage to the door panel.
I have had this condition on one of my doors. I just reach out the door and use both to open. Mine has not gotten any better in two and half years. Good luck with the project.