Sorry, there is no keyboard shortcut that works (at least on Monterey beta).
But the best way I have is to make them go away quickly, and automatically.
Step 1. Change alert to banner
In System Preferences > Notifications & Focus, change for every app that send irritating alerts, which needs explicit dismiss action.
Change the style to Banners, which will auto dismiss after a few seconds.
Step 2. Dismiss banner quickly
There is a default number of seconds before a banner dismiss itself.
Make it go away quicker. I give 2 seconds of my precious attention.
defaults write com.apple.notificationcenterui bannerTime 2
Run the above command on your terminal.