It could come to that if just torquing it up doesn't make the noise go away. You can fabricate a holding tool to attach to the yoke or companion flange as they used to call it and apply the torque (140 to 180 ft/lbs).
They used to set up the preload with special tools to substitute for the removed flange. They would then remove the tools and install the rest of the flange component parts like the slinger, seal, & the deflector ring and stuff like that. They have changed the seal configuration several times since 1950 when they started using the Dana 44 in Fords & Mercurys. You have to know which you have to get the correct parts on there. Not all of them are the same and some are way different than others.