Do you know whether they are paper wasps or yellow jackets? Paper wasps are annual species that make these nests. Although you usually see these nests hanging under eaves, some species build them on/in the ground. They are much less aggressive than yellow jackets and honey bees. If it is paper wasps in the compost, and you don’t immediately need to use that area of the compost pile, then I would leave them for this year. They are beneficial in the garden, both as minor pollinators and caterpillar killers (which they feed to their young). In the winter when they are dead simply remove the nest. If you need to use the pile this year, wait until night fall and you can pour or spray mint oil (you can buy this at the grocery store in the baking isle) on the nest. Mint oil is a nerve toxin for insects. I use mint oil to spray paper wasp nests around the doorways to our house- it will drop a wasp in mid air.
If you have yellow jackets, I would be a little more cautious – you could try the mint oil method; I have never used it on yellow jackets.
I have a serious yellow jacket problem, several in ground nests. I will try the mint oil spray Does anyone have another suggestion for getting rid of these very very aggressive insects. BOth my son and I have been stung several times.