It can go anyplace downstream of the engine - usually a couple feet back from the block or head. On a flathead, your "ports" are already like headers, so I put mine back from where the rear port tube enters the main tube - about 6 - 8" back. I like to have the bungs before any header gasket or potential leak areas.
One thing that is very important is that you have NO exhaust leaks ahead of the O2 sensor (or the readings will be wrong).
If you're welding up headers for an OHV setup, I'd put mine in the collector area (at a 10 or 2 o-clock angle). Also, if you're making headers, consider putting some EGT bungs up by the block - as close as possible. In tuning, it is nice to be able to see the EGT temps and the O2 readings. If you can only find space for a single EGT bung, put it on the rear cylinder (which tends to run the hottest). If on a flathead, I put an EGT bung in the center port tube and the rear port tube.
I'd always put the bungs in both sides of the engine - in case you need to test/tune or compare one side to the other.