Thought I was anonymous?

I am creeped out, because I e-mailed an author of a book, critiquing his work, and I guess he was insulted, because he e-mailed me back, very angrily. Then he sent me another e-mail, with the questions "How is _________?" (the town I live in), as if to say "I know where you live" possibly. I find this very creepy, and I don't understand how he knows my town, since even if my name is listed in the e-mail, I have a fairly common name, but we are unlisted---how would he know my town?