Maybe, but at least it's a chance. And if he thinks the bullets can't hurt him (which as a rule they can't), he may ignore them and get hit with something that at least leaves a scratch.
If I were Lex Luthor and had Luthor's priorities, I'd try to (subtly, as untraceably as possible which this was
not) use all kinds of random stuff on Superman just to see if it even inconvenienced him or harmed him in any way. Because if you don't know what his weakness is,
maybe the hardest known metals can penetrate his skin, or
maybe his defenses are only invulnerable from one direction at a time, or
maybe he's not immune to poison gas because the inside of his lungs isn't "super." Who knows?
Like in
this fic, though I must caution people that they won't like the ending...
Conversely if I were Superman I'd dodge basically everything I could that wasn't going to hit an innocent person. Because... who knows if
this time, my number might be up? He's supposed to be almost as fast as the Flash, so I don't see why he should get hit any more often than Flash if he doesn't want to.
Anyway, this is basically the only way Luthor could conceivably find a way to kill Superman aside from literally magically stumbling on something that can hurt him by sheer blind luck. He'd have to experiment, try things that
might work, like armor piercing bullets or incendiaries or lasers or chemical weapons. That would justify some degree of experimentation with things that haven't been specifically tried on Superman.