My barbeque grill has a wasp problem. Going to take care of that later on.

The wasps didn't attack you as you demolished their settlement because they were planning their counter-offensive as they buzz about your other plants. You have harmed their young, there will be consequences.

You may want to check all your wooden things in the garden for more. They seem too complacent...