Yeah, we had that talk too, until I had to hammer some nails into the carport rafters 2 meters from the nest......
I was happy for them to have their little 'upside-down mushroom' like nests around the place, they were harmless.
We were out dismantling some framing that was on my pergola and the fekkers had stung me and #1 son once each, my second sting was on my ear
that was the last straw.
Armed with a can of fly spray and a can of surface spray, all 4 nests got both barrels... I ran away from each nest after a 5 second blast, then came back 10 mins later to finish them off.
Its a bit shitty because Paper Wasps are considered pollinators