I have the problem when there is more than one view using more than one scale. Any extra work I add over the view seems to keep to the scale on the drawing sheet, i.e. changed as you said with the Edit, Value route.
By extra work over the view, do you mean draft entities like lines, arcs, and circles? You should try to avoid using those for additional detailing. Is it possible to achieve what you're trying to document with a cosmetic sketch feature? Because those will be scaled correctly in the part / assembly model and you can control their display in individual views via the Show / Erase dialog box.