I typically do one of two things if I don't feel like calling them back.
1. Give them a wrong piece of contact information, like the wrong house address on the correct street. A scammer without this information would probably accept it; a company that already has this information will point out that that it's wrong.
2. Only give them a part of the information, like the last digit of my house number, and ask them to supply the rest - this assures both of us that we're the person we're expecting to talk to.