First, ignore and keep walking. If he physically interferes, call the police and demand to speak to the Returning Officer responsible for the vote to report voting interference. While you wait, ask to see *his* ID. Confirm some kind of law enforcement officer. Note his name and badge number.

Make it clear that you consider this election interference, and make it as awkward as possible for them to continue.

Thanks-great!