Successful social engineering starts with pretexting. Start by giving the victim a back story, or pretext, that makes your questions seem legitimate. Why? Because if I call you and ask for your birthdate and mother's maiden name you should tell me to take a long walk off a short plank. You don't know who I am and I probably intend to do something bad with your info. But these questions are perfectly normal when you call your bank. In fact, you should be suspicious if your bank doesn't verify your personal information on the phone. If I believe that I'm on the phone with the bank (the pretext), I'm more likely to reveal personal information to a complete stranger. The social engineer hopes you forget who called who (or don't even think about it) and just give up the info. Of course this pretext doesn't do any good if the social engineer calls from the wrong "bank."
How do they pick the bank?
We post a wealth of information about ourselves online every day. In particular, sites like LinkedIn tend to have our work histories. Given a company and the time you worked there, it's relatively easy to find out who manages the 401k (or other retirement plan) there. That's who they call from. If you say that you transferred all the money to your new retirement plan with your current company, well sir, it's your lucky day. An audit of our records show the employer stiffed you on some matching funds/profit sharing/etc. you should have been entitled to. People jump at the opportunity for free money, so that's a win.
But how did they get my phone number?
In my case it's on my business card. If you don't have a business card, it's probably on pastebin from some site you registered with years ago. Do you have a resume posted online? Maybe the phone number is there. Plus there are all sorts of sites attackers can use to look up your number for pocket change. Finally, they can simply use the phone book.
Protecting yourself from this attack
Protecting yourself is pretty easy, albeit less convenient. Don't given any information to someone who calls you. You should call them back. Where should you get the number? Don't, and I can't stress this enough, call the number given to you by the caller. I've seen many a social engineering attempt resurrected when the victim says "I've been trained to only give information to people I call on verified numbers." But then they follow up by calling the attacker back on his "direct line." Go to the company website and get a valid phone number. There's usually an option to transfer to an extension.
So was this a social engineering attack? Probably not, but I don't take risks like that. The reason I suspect this was legitimate is based only on the fact that the caller initially asked to set up an appointment to talk to me rather than getting my information now. But that is a good technique for putting me at ease about the situation. Does it matter whether you get my information today or later this week? Probably not. Even in the context of penetration testing, these engagements usually last a while. He has time to wait.