I made a similar thread a little while back, so there might be some suggestions that were missed over here:
http://shmups.system11.org/viewtopic.php?f=3&t=45013
Personally, I've been having a blast with the Simple Series of games so far; I'm sure you know of them, and not sure if you're into kusoge or not; but if you dig bad games, so far I have:
The Maid Uniform and Machine Guns - terrible goddamn controls, pretty much only one song in the whole game, and horrible scoring mechanics. Lots of fun for some simple beat em up + machine gun action. 8/10
The Overwhelming Hell (The Tairyu Jigoku) - very very strange survival horror game, but I'm digging it so far. I'm pretty sure this might have been intended as softcore eroge, if you're into some bug on girl action. Awful controls, but that's the norm for survival horror. Some pretty cool tunes though, and it's dumb enough to be a fun conversation piece. 7/10
Earth Defense Force 2/Chikyuu Boiegun 2 - how has this not been mentioned yet? An absolute MUST, some of the most fun I've had so far.
and for non Simple-series games, I'm also loving Poinie's Poin. This game's a total trip: completely nonsensical, made for kids, but has so much strange language and humor that it really doesn't have a target audience. It's entirely in english (with options for japanese voice and audio -- lots of attention to detail for this game, strangely enough), so no issues playing it. If you're into straight up weirdess, I can't recommend it enough.