Ace hardware sells a non-toxic spray (I believe the active ingredient is mint oil). Sorry I don’t have any other ideas. I came accross this nice article in the Washington Post this week about why Yellow Jackets become so aggressive this time of year. (click on the third week of September to find the yellow jacket story.)
Please let us know what you end up doing and how it works!