1954 was the first year they started using an aluminum rear extension housing at the rear of the transmission where the rear motor mount is located.
I would be skeptical of removing the front steady rest entirely on a '54/55/56, because you would be relying on that rear aluminum housing to support a lot of weight.
I know that aluminum housing is pretty thick, but with only 3 supports, could the aluminum housing crack under the added stress?