It's two. Lynn Frazier from North Dakota in 1921, and Gray Davis from California in 2003.
I had to laugh at my own sampling bias. My assumptions were pretty understandable....I've been of voting age since 1999, and in that time this has happened twice. Therefore it was reasonable to assume this happened at least occasionally. I figured about once every 10 years, which would be 23 or 24 in American history. I was pretty sure not every state had a recall option, so I halved it. 12 felt good.
This is the problem when data leaves out key points....it relies on our own assumptions to fill in the details. Engineers are normally trained to get explicit with their assumptions when estimating, as evidenced by the famous Fermi problem. However, even the most carefully thought through assumptions are still guesses.
That's why it's important to remember the quote above: what you're shown is important, but it's not half as interesting as what's hidden.