I can't say I have practical knowledge that will be of help, but here are a few thoughts. If people know about your app and don't download it means that they perceive the value of the app to them to be less than the effort of downloading it. That could be because you have failed to explain the value of your app to them. It could be they understand what your app does and they see it as yet another "me too" social app and they are simply burned out of dealing with yet another app. Is your app something different that solves a real problem in their life? If it is, then make sure you communicate the value to them.
Yes that could actually be one reason. The marketing till today has been more to tell that it's available and less to explain what it does. Maybe we should try changing our App store desc/screenshots to reflect the why.
