On the plus side, you could do fair bit of creativity with a brick wall than with a fence.
render and do some creative painting work if that is your thing
do…

Doesn't this read like they will put the fence back, implying they will be building slightly off the boundary and just need to remove the fence to build it properly?
In…

if the looks is your primary concern, then it's in your best interest to let them access your property, so they can do the job nicely. otherwise they will do…

Just ask them to build slightly off the boundary and then reinstate the fence. You should be able to see the plans, that is weird.