First, the tree can’t be damaged by being pruned. In other words, you aren’t allowed to prune it if pruning the tree makes it die or causes damage which harms it.
The second thing is that you aren’t allowed to go onto your neighbor’s property to prune the tree. Being limited to your side of the fence creates a dilemma. Because, while you may be able to trim back the growth, you can’t actively prune the branch or branches causing your troubles.