
Ask HN: What does it really mean to do ML/DL at the hobbyist/research levels? - TrashQuestion
I have taken a  few ML&#x2F;DL courses (college, Coursera, Udacity) and have implemented a bunch of models (SVM, KNN, Neural Network, etc) from scratch. I learned some tensorflow and made a CNN, RNN, and even a GAN for generating human faces (Udacity Course). I do hardware&#x2F;firmware design professionally but i dabble with software and machine learning as a hobby.<p>I liked the courses where math was a huge focus because it allowed me to understand from first principles.<p>When i took the Udacity course it was just learning the tensorflow library for the most of it. Conceptually i understood CNNs and RNNs  but i couldn&#x27;t implement it from scratch because i didn&#x27;t truly understand it. I could maybe implement the forward propogation for RNNs but didn&#x27;t understand the nuance of the backprop.<p>I ended up basically just understanding the tensorflow library, which is still useful and powerful. But that left me wondering why do people who do this for research and hobby really do?<p>I highly doubt researchers are spending their time re-inventing the wheel to implement their own basic CNN or RNN model, but they obviously are doing more than just function calls to a set library right?<p>What should i be trying to learn if i am a hobbyist wanting to really understand ML&#x2F;DL? Is implementing theses advanced DL models from scratch even useful? What steps do i take to become &quot;better&quot; at ML&#x2F;DL rather than just following the tutorials and courses that taught me how to use the tensorflow library to create basic RNNs&#x2F;GANs?<p>When people say they are &quot;better&quot; at ML&#x2F;DL than they were before does that just mean they have a better intuition for what model, how many layers, how much dropout, batch sizes, etc to use for a certain problem?<p>I feel lame just piecing together models with high level code like `keras.add_fully_connected(layer_size)` or `tf.MultiCellRNN`. It feels like cheating because i&#x27;m just throwing things together and seeing what works. The science aspect feels lost.
======
p1esk
Not sure I understand your question, or what you're trying to achieve. It's
the same as if I asked you " What does it really mean to do hardware/firmware
design at the hobbyist/research levels?" Does it feels like cheating when
you're just throwing things together and seeing what works?

First thing you need to understand is that data is king. To use DL models you
need lots of data. If you don't have lots of data, you will probably have more
success with older ML models.

Hardest part of any DL/ML project is getting/cleaning/augmenting data. In the
real world, dataset preparation will consume >75% of your time/effort. Size
and quality of your dataset will affect the resulting accuracy more than a
choice of a particular model.

Once you have data, you can just try several models which have been used for
your task, and see if size of model makes a difference. Language or frameworks
is not that important, but most people use Python and Tensorflow or Pytorch.
Again, does not matter. Of course you need to understand the basics. For
example backprop in RNN is tedious to trace, but you should be able to do it,
if necessary: it's no more complicated than linear circuit analysis, or
designing a FSM. There are no "nuances", just apply the chain rule backwards,
and think in terms of unrolled computation graph. You rarely need to think
about low level details like that, but occasionally you might need to use an
operation which is not differentiable, or does not have gradient implemented
in the framework you're using.

If you want to get "better" at ML/DL, go to Kaggle, and try to solve problems.
Read about winning solutions. Read relevant papers. Read code. Try things.
Eventually you'll get it.

------
spyckie2
two routes from what i've seen to use data effectively (i work with quite a
few data scientists):

1\. Go very technical, try to win kaggle competitions, try to create the very
best solutions possible. This probably means learning all the cutting edge
techniques, being very up-to-date with what's going on in the algo space, and
lots of practice/experience, and you probably have to have a knack/passion for
this kind of competition as well.

2\. Go very domain/business driven - force yourself to optimize thinking about
high impact business questions that you'll be answering with data. If you can
craft a strong answer that leads to high level business strategy, it doesn't
matter what techniques you used, people value that. This kind of person is not
ML centered, they are more of a stats person with some tensorflow mixed in.

Generally speaking, Google and AI/tech companies prefer #1, non tech companies
prefers #2.

Most people who want to get into data science do it for the exact same reasons
you do - it's math-y and new. Other types of programming are old, not so
math-y, boring. However, data science at a business level is completely
different from data science at a hobby level because doing data science is
very expensive. It requires much more capital and infrastructure than regular
programming. Definitely more expensive than developing hardware in my opinion,
especially if you look at data collection costs, data scientist salaries, and
how much work is needed to extract meaningful business value out of data.

A hobby level data scientist is probably similar to a hardware or robotics
hobbyist in my opinion - they both play with fun tools and spend money/time
but have difficulty translating it into real businesses/profession due to the
high capital cost needed to even access the really good tooling, let alone do
something at a competitive level with those tools.

