You have left undone those things which you ought to have done up

"...my poor grandmother once incidentally observing, in the course of an instruction upon the use of the Prayer-book, that it was highly improbable that I should ever do much that I ought not to do, but that she felt convinced beyond a doubt that I should leave undone pretty well everything that I ought to do."

Now, Smokey Stover indicated that grandmother uses here the language of the Common Prayer Book; and indeed that's what it written there:

"We have offended against thy holy laws. We have left undone those things which we ought to have done; And we have done those things which we ought not to have done."

My question is: What is the origin of the last excerpt? It must be the Bible. But where in the Bible?

Thank you.

It is from:
THE BOOK OF COMMON PRAYER

and Administration of the Sacraments
and Other Rites
and Ceremonies of the Church
Together with The Psalter or Psalms of David

According to the use of

The Episcopal Church

"Almighty and most merciful Father,
we have erred and strayed from thy ways like lost sheep,
we have followed too much the devices and desires of our own hearts,
we have offended against thy holy laws,
we have left undone those things which we ought to have done..."

CJ said that to Reggie Perrin when Reggie left his fly open, on an episode of "The Fall and Rise of Reginald Perrin", which was a BBC show from around 1977 and starring the late Leonard Rossiter.