If your neigbor is realy anti your shooting I'd be very careful. Imagine the response from the police if they called 999 to report 'some nutter with a gun taking potshots in next doors garden'.
And good a shot as you may be how do you prove none of your pellets leave your property (miss, pass though, deflection loads of way it could end up somewhere other than in your quarry)? Dead easy for someone who wants to to a) stir up a shedload of trouble for you, and wind the local press up into an anti airguns frenzy.