

Ask HN: Advice on how to tackle a summer research project - SandB0x

I'm about to start a three month project for my masters, in the field of computer vision and machine learning.<p>Of course I have an advisor, but mostly this is going to be a period of fairly open ended independent work, and at the end I need to have something to show for my time.<p>It's a bit daunting, and I'd appreciate receiving some wisdom from you on how to get good research done in such a short time. Some specific points that come to mind are:<p>&#62; Ideas for testing techniques in a simplified form, to avoid wasting weeks reaching dead ends.<p>&#62; Avoiding feature creep - I get distracted by every new paper I read and want to apply it to my problem.<p>&#62; How to keep track of my work, so that the final write-up is as painless as possible.<p>Any other thoughts most welcome. Thanks in advance
======
_delirium
Hmm, it's a kind of broad question. What kind of project roughly do you have
in mind? Is it something along the lines of: compare/benchmark existing
techniques for [some problem]? Or: improve current widely used technique by
using [some ideas]? Or: propose new technique based on some idea of yours,
your advisor's, or from the literature? The risks/etc. of the different
approaches are pretty different.

As a time-limited project, one way of reducing risk is to think up front about
your hypotheses, and stick to testing the hypotheses and investigating
explanations for your results, rather than engaging in a more open-ended quest
like "improve performance on [problem]". That is, your successful end state
is: I confirmed or refuted this hypothesis, and did a good job explaining
every step of the way.

For example, based on background knowledge/etc., you could come up with an
argument for why it's plausible that some algorithms could be improved in some
particular ways. Perhaps you're proposing an improvement that was used
successfully in a different domain, or on a related algorithm, but hasn't been
tried in this one; or you have an argument for why it should work based on
some sort of theoretical analysis. Then, implement and run tests, and try some
variations based on those results. Then report whether your improvements
worked or didn't. If your initial hypothesis is reasonable, even a negative
result can be fine, especially if you can, after the fact, explain why this
plausible-seeming technique or improvement turned out not to work (i.e. what
was wrong about the initial intuition, or what differs about your domain where
it didn't work from other cases where something similar was applied
successfully).

The biggest risk is probably proposing to come up with a solution to a problem
where no decent solutions currently exist. Then it's hard to promise that you
can get one in one summer. It's possible you might, or it's possible you'll
just find 5 ways that didn't work. If you do plan to go that way, you should
make sure your advisor would accept a "progress towards solution" writeup in
the event you don't come up with something. That kind of thing is common in
math and theoretical CS, where it's quite possible even a PhD student will
fail to prove some important theorem, but after working on it for a few years,
will write up a thesis that's more or less: here are some approaches to
Important Theorem, and what did and didn't work about them. It's a bit less
common in ML, but if you can explain why the problem is interesting, and
demonstrate that the obvious solutions don't work and lay out the parameters
of what would have to be done to overcome those problem they have, I could see
that being a good thesis.

\---

A short version: come up early on with some idea of what your hypothesis will
be, and what sorts of positive _or negative_ writeups you could write,
depending on how experiments come out. What you don't want to do is just spend
11 weeks working on random things, then one frantic week trying to figure out
what out of that you can write up.

~~~
SandB0x
Thanks for taking the time - this is all really helpful.

You seem to have understood the situation well (unless I'm reading this like a
horoscope) - there are currently no decent, general solutions to my problem.
Most recent papers use techniques that work well on images, apply them to
video, show a couple of limited results and of course the authors don't
publish any code. Conceptually, everyone can see that this class of techniques
is not suitable, and whenever a paper is re-implemented the results are
_extremely_ poor. Basically, it looks like a dead end and I think even a
tentative first step in a new direction would be counted as a success. But yes
I will make sure to clarify this.

So based on your advice, over the next few days I will nail down the
techniques to try, write a short hypothesis for why I think each one can work,
and plan my time so that I am able to run with a promising approach or
combination of approaches for the last month or so.

Thanks again. If you're interested, drop me an email and if I have something
fun I will send you a copy of my writeup later this year.

