- ikea whiteboard - good if you have intention to work in pairs. starting from '-', '+' gradually increasing numbers and complexity. Mine likes drawing and erasing
- games - tic tac toe (different strategies), checkers just starting, playing cards (counting numbers, bigger lesser numbers), Battleship paper game
- diy kits - like on ali express 'diy kids science kits'. They're pretty low quality but gives general idea of some physics
What doesn't work for us:
- apps - boring for kid in a long run / questionable impact on eyesight / he's getting used to games rather then apps
- toys - like that: https://www.amazon.com/Learning-Resources-Robot-Activity-Pie... doesn't work in long run
I'd love to hear what else is inspiring kids to learn.
I agree also with your comment on apps, with only one exception: https://www.playosmo.com/en/. The Numbers, Tangram, and Words games have been particularly effective.
Uno is a good card game for order, addition (draw 2 + draw 2 + draw 4 is how many?).
Robot Turtles has been fun from a "let's try to discover what a function does" perspective but I'm not 100% sure what has registered yet. I found 4.5 yrs to be the sweet spot to start there.
The biggest thing though: time with your kid working on these things. If you're there, and you are engaged and having fun, it's likely they'll also be engaged and having fun. Start with anything. Keep it light. Let discovery happen without correcting. It's fun to watch.
Any similar books I can buy to guide teaching math my toddler-kid-etc?
Like DragonBox's Elements, Algebra and Numbers ?
Or the ones by DuckDuckMoose, such as MooseMath ?
KhanAcademy's very first levels might be good as well .
With your oldest you might try a game like Junior Catan , that than graduates to Catan where you can talk about the probability distribution of the sum of two dice, an essential aspect of Catan.
You could perhaps try a programming language like Scratch. Have a go at some of code.org's "games"  and perhaps even MIT app inventor's with your oldest? 
That said, if it's fun then there's no harm in it and keep it up. If it ever gets to be no fun though, my advice would be to let it go until they actually need it.
As far as games go, look for something that's fun first and mathematical second. I like pencil and paper games for this. We play them together often while waiting for meals at restaurants or during other down times. Rarely do I call attention to the mathy nature of the game, it's just a fun way to "be mathematical."
The other thing I think is important (that I still have a hard time living up to myself) is to really focus on things that need open-ended exploration. Particularly for children who are in school away from home, instead of trying to reinforce what the teachers are probably already doing a good job of teaching, find ways to get your kids to investigate things on their own. Games with known optimal strategies can be a fun way to get into this because they can then go dominate at school with their new knowledge. They don't get as much opportunity for this kind of thing at school because of the necessity of teaching the mechanics of computation, but keeping their curiosity alive is a great goal to strive toward.
Some resources I like:
Martin Gardner's books
Games from http://www.papg.com/
http://www.expii.com (this is advanced stuff, but can make for fun discussions because some of the situations are neat)
The Beast Academy math curriuculum from Art of Problem Solving (starts at second grade, but when you get there I really like it). http://beastacademy.com
Can you explain this part? A quick google search didn't reveal any relevant games/apps, so the only thing I can think of is that a school of fish can grow exponentially, and that a baby is the logarithm of an adult :-)
I didn't read the whole thing unfortunately, but I do agree with much of the material at the very beginning - make math interesting and challenging, and spur creative thinking rather than trying to force kids through a curriculum.
The insanity is saddening. Happily my son who is 6 is oblivious to math but brilliant at Lego, incredible at drawing, and astoundingly proficient at laughing. When he eats ice cream he uses his whole face. Love it, love him, and I wish sometimes he never has to grow up.
In that context, your claim is that playing with a knife is just as good as playing with math, that singing nonsense songs is just as good as singing the alphabet song. The kid has fun either way, but one style of play will lead to a kid who is a confident learner, and the other will lead to a kid who gets frustrated at school and thinks he is not as smart as his peers.
and a paper describing it in detail
It's somewhat related constructivist (or constructionist?) method of teaching mathematics that's actually used in some schools.
Would be interesting to hear people's thoughts on that
He has very weird and non-standard ideas that you should not take too seriously, specially if you get into his videos on more advanced stuff, but his way of teaching basic arithmetic is certainly inspiring.