Math is the manipulation of abstract symbols according to abstract rules. If you don't like symbols, you don't like math. If you are illiterate in symbols you are illiterate in math.
The word is often used imprecisely, though. Because so many real-world problems can be translated into math, there is a temptation to equate "math" with "any problem that can be expressed in math". Thus you frequently see poetic statements like "my cat is great at solving differential equations", or "music is math because it's all about harmonic series and Fourier analysis". But these things aren't literally true. You can put a bucket under a flowing faucet, and it will collect all the water, but that isn't really integration. The bucket isn't doing math.
And being ignorant of math isn't the same as being stupid. As the OP points out, you can get a lot of quantitative reasoning done without using math. A classic Fun Fact About Math is that it took thousands of years to invent the number zero. And it's true. But that doesn't mean that the ancient Egyptians used to waste hours staring into newly-emptied buckets and baskets in stunned amazement, murmuring "what on earth is that" to themselves in Coptic. People understood what "having no objects" meant long before there was a symbol for "the number of objects in an empty basket". It was the highly abstract symbol "zero", and the highly abstract operations involving zero, that had to be invented. 
It's worthwhile to recognize that interpreting the real world in terms of abstract symbols, and vice versa, is a terribly difficult skill that requires lots of practice. (In my case, I was well through grad school before many bits of physics clicked.) And it's worthwhile to recognize that you can often do without math: You can reason quantitatively without it. Birds do it! Bees do it! But don't pretend that you're doing math unless you are actually doing math. The abstractions are the math.
 Or, rather, discovered. Although we'd better stop there, because I won't be able to cope with the ensuing philosophical back-and-forth.
"0" is a formal symbol with particular formal behavior
"empty/missing/none" is a well-known physical concept
Zero is a precise, powerful mathematical object which can be represented by them both.
This is difficult to deny. Unless you want to deny the providence of most widely recognized mathematicians throughout history, you have to accept that formal language of math is relatively new. Furthermore, it's alive and growing, inconsistant and incomplete. There is a meaningful frontier, and there you observe mathematicians are really studying something else and furiously creating the formal language to describe it.
In this light, metaphor is absolutely a useful tool in the same class as formal language for explaining and reasoning about math. You're right to point out the non-equivalence of the two, but the author's Kill Math project is in no way not math. Furthermore, I'm anecdotally a supporter of the author's belief that doing math competently requires knowing a the metaphorical side since your symbolic projects may fail or be unclear.
I'd be willing to accept that metaphor will never be as powerful as formal language, but it does discredit to the way (I'd wager) most people understand math to deny the metaphorical.
At the heart of this trouble of definitions is Gödel's Incompletenesses. The practical effect of their discovery was the destruction of the dreams of formalists who had for years hope to discover the essential shape of the formal language from which all math would spring. With Incompleteness however, we are forced to admit that we can study, meaningfully, the behavior of mathematical objects for which the language of math cannot be used to reason about.
Then we extend that language, of course.
I have no time to craft a nuanced reply so let me just take a shortcut and concede: If you can get the student from "here are a bunch of physical concepts" to "here is a mathematical object, with interesting abstract properties that you can reason about" without introducing formalism you'll have succeeded in teaching math. Excellent.
Is this plan really going to work very often? It is easy to say that you can derive, say, the utility of "zero" without ever doing any arithmetic -- just as it is easy to say that you can be a full-fledged computer scientist without ever touching a computer -- but in practice?
It's true that the presence of a supercomputer in everyone's pocket will change this argument significantly. But simulations go only so far. They, too, are only metaphors, and if you don't know enough to tinker under their covers they are rather inflexible metaphors. Your classical mechanics simulator is not going to discover quantum mechanics for you.
In my experience learning about mathematical abstractions requires all of the above tools -- you tinker with the formalism, you ponder the physical analogies, you draw mental pictures of clouds and colors, you play with a simulator, you build some circuits in the lab, you go for a walk, you tinker with the formalism again, and six years later you finally get it.
I agree completely that the process of learning mathematics is probably highly multidimensional for... pretty much everyone ever. In particular, it's easy to see how formal descriptions can push mathematical generalization forward far before we have a suitable concept of the mathematical object we're describing.
I think we're all (incl. the op) in some kind of agreement here about the didacticism of math. The op didn't disregard the power and utility of mathematical languages — he came from being trained pretty heavily in engineering math, at least up to playing with higher-order differential equations — but instead was, perhaps not directly, arguing for increased metaphorical/physical descriptions in taught mathematics. He's just responding to the rather eye-opening feeling one gets when one starts to realize that math is so interpretable!
I think that's a perfectly fair argument to have. I know that in my own experience, I never understood the joy of math until the day that linear algebra took an interpretation as linear space transformation.
So we're just sort of all oscillating here in strong rebuttals of whatever interpretation of the "heart" of math the prior author champions for a while. Which is fun but unproductive.
I like the challenge of taking someone from physical concepts and metaphors directly to a mathematical object. I think it'd be possible, and maybe even useful when someone first starts to learn real math, but certainly it's not the most efficient way to become well-read. It'd be a lot like explaining the meaning of, I dunno, Día de los Muertos without immersing someone in Mexican language and culture. A single point of contact can be forced, but you lose so much context and fluency.
More directly, what I meant to say is that since there exist true theorems which cannot be proven within any particular choice of mathematical formalism, we need to operate with tools beyond simply symbolic manipulation. That was the death knell of Hilbert's Program and solidly separated the formal specification of math from "that thing which we're studying".
I'm not really sure that you mean about the "formal specification" for math vs. the "that thing we're studying". An informal (ie, not expressed in ZFC + 1rst order predicate calculus) proof of something non-trivial can go on for dozens, if not hundreds of dense pages of symbols. If I recall correctly, Whitehead's Principae Mathematica derived arithmetic from ZCF and predicate calculus, and it took the whole book.
I did a little reading to refresh myself on the subject, and this stood out as a good summary of the topic:
"In a sense, the crisis has not been resolved, but faded away: most mathematicians either do not work from axiomatic systems, or if they do, do not doubt the consistency of ZFC, generally their preferred axiomatic system. In most of mathematics as it is practiced, the various logical paradoxes never played a role anyway, and in those branches in which they do (such as logic and category theory), they may be avoided."
I mostly wanted to walk around the historical event I mentioned, the breaking of the Hilbert Program. At the time, it seemed that formal specification of math would provide a complete picture of what math was! Once the Program was finished then the job of mathematician would eke out into "computer" (of the abacus sort) or into other fields which interpreted the canon.
I'm not sure which death stroke was stronger, the incredible opaqueness and complexity of proof systems like ZFC or Gödel just saying what he was trying was outright impossible, but Hilbert's Program was killed before it even seriously took off, leaving the study of mathematics and the practical formalisms we use to study it pretty ad-hoc instead of grand and unified.
I'm unifying that with the fact that the way math seems to be practiced never comes from the formal language but instead first comes from imagining some kind of "mathematical object" and then taming its behavior with formalisms. You could consider them to be one and the same and argue that the difference is highly philosophical, and then this is where I'd invoke Gödel and inform you that there definitely exist things we could benefit from reasoning about that your formal language would fail to describe. This existence proof separates the classes of true things and provable things and makes their distinction more than philosophical.
Now, talking about what a "mathematical object" is gets you to the bleeding heart of the philosophy of science and epistemology. It's a tough question!
As a final note, ZFC is ZF + Axiom of Choice... which, yes, most practicing mathematicians just accept AoC so that they can integrate or whatever. The formal world without AoC is very sparse, but nobody has any sort of idea what the arbitrary decision means. I know that there has been some significant study of ZF-C, though it's been "impractical", I don't know if anyone is willing or capable of stating that ZF-C is in any way worse than ZFC. Impractical is a Mathematicians favorite adjective, so they're just two extant formal systems which disagree quite a lot on important things but we mostly pay attention to ZFC.
At that point, all the linear algebra I couldn't figure out for the life of me all those years finally made sense. And it was the same for most of my classmates. After that, whenever I saw xY, I thought "the vector x is being moved into a new space", and all the equations made sense to me.
You could explain what an SVM is with equations to me all day, but it's only when you say "you're trying to get the plane to separate your data by a margin as wide as possible" do I actually get it, and then all the math becomes easy.
Different people have different ways of manipulating the abstract symbols, and for me it's to equate them to something I already have experience in. Then I can get the solutions intuitively, rather than pore over pages and pages of equations.
In the end, I quit academia precisely because I couldn't manipulate symbols, and thus my way of learning wasn't compatible with everyone's way of teaching. Maybe I can come up with something better if someone explains things to me in terms I can understand.
I think anyone is capable of grokking anything, just the time taken to do so is variable. people that give up on "learning" something (academia in your case) just don't want to spend that time.
p.s. I also gave up on academia for the same reasons :)
* Math is full of symbology with implied meaning. For example, theta is often used for 'angle'. How many other symbols have implied meaning like that? Granted, it forms dense, concise, precise papers. Which brings me to my second point.
* If you don't know the symbology, it's difficult to read it. I believe that people suffer reading comprehension problems if they don't know how to verbalize a symbol like 'θ'.
* Lastly, the symbols make it very difficult to google for concepts.
That's the definition of a calculus, not the entirety of math.
>More generally, calculus (plural calculi) refers to any method or system of calculation guided by the symbolic manipulation of expressions. Some examples of other well-known calculi are propositional calculus, variational calculus, lambda calculus, pi calculus, and join calculus.
Math, fundamentally, is about abstract concepts, not symbols.
Sure we have intuition. But intuition can be wrong. Not all problems are as simple to explain as zero and empty buckets. Intuition is also worthless if you cannot communicate those ideas in an unambiguous fashion.
So if the OP really develops a method for communicating the concepts described by math in a much more efficient way - I am all for it! But I find that highly unlikely when even it is admitted in the article itself that he has no idea what this will look like.