In short, no, that's the whole mechanic of Bullet Soul where defeating enemies gets rid of bullets.
The game's not easy by standards towards the later levels, even with the mechanic in place. especially on Version A. Version B is somewhat easier, though there's Loop, lol.
They just confirmed today that BOTH Bullet Soul AND Infinite Burst are backwards compatible on Xbox One and both are available digitally right now in the US (and possibly Europe) for $14.99 each.
The Japanese producer recently said they're considering two content updates to Bullet Soul PC to address fan requests--a high-res texture pack and a new harder mode--but they're not guaranteed and are basically contingent on the game continuing to sell. (He also said he wasn't sure if adding a hard mode would even be appropriate as Bullet Soul was intentionally designed to be simple, apparently.)