File down his sights

What does the phrase "he filed down his sights" or "I've got to file down my sights" mean? I know it's a metaphor, and I know it's language taken from riflery. But that's as far as I can get with it. Thank you.

I've looked in several references and found "zero."

The result of filing down the fore sight would be to aim lower. This corresponds to being less ambitious. Does this fit the context?

Not sure if it fits. But what about the idea of accuracy? Might the idea of filing sights have ever had to do with that?