The ideal stick basically = best arcade parts - you can ge them now in stock form but they can be a bit expensive for what they are.
Given that you don't want to mod yourself (?) an easy one to go straight for would be one of the mayflash Universal Fighting Stick - ps2/3 + pc without any adapters ... and I believe... [runs off to find out]
.... brokenhalo does a modding service for them -
http://shmups.system11.org/viewtopic.ph ... sh#p490506
The basic stick is cheap and not too bad, but with a modded one you'll pretty much have a perfect playing stick.
Personally I'd recommend sanwa stick and buttons - most likely what you'll find in a good jap arcade machine. The JLF stick is precise and buttons very sensitive.
The regular slight shmup alternative is with the seimitsu stick and buttons - stick has basically a slightly shorter throw and slightly less resistance - bit more loss feeling, and the buttons are less sensitive [helps avoid accidently pressing - usually bombing

]
Everyone has their own preference, and its pretty well debated over the forums - I would personally recommend sanwa setup, as your first stick - especially if you're likely to play some fighters - maybe get a seimitsu later to try out once you're hooked - you will be
As for vital games - why the mayflash is good - opens up the relatively cheap ps2 + 1 libraries... You'll no doubt be interested in some of the Cave games - eg. dondonpachi (often cited as father of bullet hell, in truth a debabatable claim, but there we go - its still very good

... I would personally also recommend some neo-geo games such as last resort + blazing star as something 'a bit off the beaten track'... if rather tough... use those continues
I'm sure you'll soon form your own ideas of must play games though - enjoy
ps. if you're not too up on the japan based shmup games I heartily recommend
http://www.world-of-arcades.net/ which outlines some of the big names and their games ... it was also put together by some well informed people [so not me

] that you'll no doubt see on these forums...