If in doubt, don't risk it, there's plenty of rabbit holes that aren't near setts. I'd give it a miss. It may run very deep as well so not worth the hassle, especially as you couldn't dig the ferret out if it lies up without the risk of prosecution for disturbing a sett. Find a different set of holes.
Cheers, D.