"I would rather have them check my ID than have a valid signature available to a theft if my wallet gets stolen"

Assuming your ID is in the same stolen wallet, as would be the case for most people, now the thief has your ID and the capability to modify the picture on the ID to their own likeness should they desire, though most wouldn't go to the trouble. I see less, not more, security that way. Worse still is the fact that an unsigned card offers a golden opportunity to a thief to sign it and have their signature match exactly. If you've written "See ID" in the signature column, please be aware that Visa rules for merchants states the following:

"Some customers write "See ID" or "Ask for ID" in the signature panel, thinking that this is a deterrent against fraud or forgery; that is, if their signature is not on the card, a fraudster will not be able to forge it. In reality, criminals don't take the time to practice signatures: they use cards as quickly as possible after a theft and prior to the accounts being blocked. They are actually counting on the you not to look at the back of the card and compare signatures--they may even have access to counterfeit identification with a signature in their own handwriting. "See ID" or "Ask for ID" is not a valid substitute for a signature. The customer must sign the card in your presence"

Leaving a card unsigned simply defies logic.