Another thing that works well is naptha based wax and grease remover such as Prep Sol. No other solvents that I am aware of work as well and many would harm the paint.
Silicone glue? You mean like gasket sealer or chaulking stuff? I'd try rubbing/peeling it off with a microfiber towel before I went chemical on it. That stuff won't invade the surface finish, it just rides on top.