
Google Brain Residency - aleksi
http://tinyclouds.org/residency/
======
gabrielgoh
I enjoyed the section on negative examples, and the things that don't work
out. Makes me feel a lot better about the ton of things i've tried which
didn't quite pan out the way I hoped.

------
orthoganol
Interesting to see that the Google Brain Residency isn't that much more
advanced than other graduate work or fellowship programs out there.

I disagree with him about needing a "Rails for deep learning," I think it's
quite fine now in practice, especially since DL code is typically
significantly less than a web app. A file for data pipeline, a file for your
model, and a file for training and/or inference. Not sure it really needs to
be much more complicated than that.

~~~
aub3bhat
I actually strongly agree with him, In fact the progress in Deep Learning has
stalled due to inefficiencies in sharing data and code between researchers.
Today to replicate a study you have to spend several days downloading data,
making sure all dependencies are installed etc. This is worsened by lack of
standard formats for even simple things such as representing 2D bounding box
on an image.

People forget that the reason Deep Learning came to prominence was engineering
that enabled use of GPU's (Thanks to Alex K.) and large dataset from ImageNet
competition (Thanks to Fei-Fei Li).

Importance of software engineering in moving the field forward is often under-
appreciated. This blogpost beautifully illustrates several instances where a
great implementation made it easier for researcher to speed up experimentation
and lead to breakthroughs in Computer Vision. [1]

I am building Deep Video Analytics which aims to become the
Rail/Django/MySQL/(favorite analogy) of Visual data Analytics [2,3].

[1] [http://www.computervisionblog.com/2015/01/from-feature-
descr...](http://www.computervisionblog.com/2015/01/from-feature-descriptors-
to-deep.html)

[2]
[https://github.com/AKSHAYUBHAT/DeepVideoAnalytics/](https://github.com/AKSHAYUBHAT/DeepVideoAnalytics/)

[3] [http://www.deepvideoanalytics.com/](http://www.deepvideoanalytics.com/)

~~~
orthoganol
Well, my professional experience has been completely different, maybe hardcore
research in images is different. At any rate isn't TF already the Rails/
magical framework of DL, in some sense?

~~~
ericd
I'm not an expert, but Keras seems much closer than TF.

~~~
espadrine
They're complementary. Keras typically uses TF as a back-end, letting users
model standard architectures quickly, while TF lets you express new and
complex graphs of computation.

~~~
ericd
I'm aware, I'm using Theano as my backend. I just mean that Keras seems more
analogous to Rails than TF - TF would be Ruby in that analogy.

------
popcorncolonel
I don't really see the point in prime classification, I feel like you're never
gonna get better than doing a deterministic sieve algorithm. Even if it did do
quite well in terms of accuracy, probabilistic models aren't 100% correct so
you still have to check for primality deterministically.

However the twin prime factorizer idea was interesting and could potentially
lead to some factorization speedups (based on heuristics) if done correctly.

Either way though, prime number distribution is quite tricky - we should
probably be looking in other number bases as well rather than just base 10.

~~~
hulahoof
_Either way though, prime number distribution is quite tricky - we should
probably be looking in other number bases as well rather than just base 10._

This interested me so I went looking for how primes would work in other bases,
this answer implies that primes are the same regardless of base:

 _A prime is a prime no matter which base you use to represent it. On the
surface one might think that in Hex you would have 3_ 5 = 15 as "usual," but
it really turns out that 3 _5 = F.

The example 21 doesn't work too well because it is not prime. The base ten
number 37 is better, because it is prime, but its Hex representation is 25,
which sort of looks non-prime. Hex 25 is not, however, repeat not, 5 squared.

Okay, enough for examples. The fact of being prime or composite is just a
property of the number itself, regardless of the way you write it. 15 and F
and Roman numeral XV all mean the number, which is 3 times 5, so it is
composite. That is the way it is for all numbers, in the sense that if a base
ten number N has factors, you can represent those factors in Hex and their
product will be the number N in Hex.

Relating to your question about base 13, the base ten number 13 will be
represented as "10" in that system, but "10" will still be a prime, because
you cannot find two numbers other than 1 and "10" that will multiply together
to make "10".

I hope this helps you think about primes in other bases._

[1]
[http://mathforum.org/library/drmath/view/55880.html](http://mathforum.org/library/drmath/view/55880.html)

~~~
rahimnathwani
"this answer implies that primes are the same regardless of base"

Right. If you have some quantity of apples, then can you lay them out in a
regular rectangular-shaped grid (more than 1 apple wide)? If not, the quantity
is prime. You can count the apples in any base you like.

~~~
peteretep
This is the far more intuitive answer I use when explaining to people.

------
wyldfire
> I believe my motivating demo will be achieved some day soon—you will watch
> Charlie Chaplin in 4K resolution and it will be indistinguishable from a
> modern movie.

Wow, where does it stop, then? Can we upsample modern 4k films? Or deeper
dynamic ranges? Stereo projections from mono images?

I suppose that the limiting factor is the resolution/color depth/perspective
channels of the training images?

~~~
zitterbewegung
Since this would probably be more like it is inferring the signal by
correlating movies that have similar elements you would need an dataset of 8k
signals / dynamic ranges / stereo projections and then you can attempt to
perform the up sampling. You were right in your last statement.

~~~
clay_to_n
I don't see why you'd need a dataset of 8k images in order to upsample 4k to
8k, as long as there are 4k images of that object from at least twice as close
as you're shooting from.

(Actually I think twice as close might be 16k, but you get the point...)

~~~
phreeza
4k refers to the number of lines, so twice as close is 8k.

------
uyoakaoma
All the "what happened to ryan dahl" questions have been answered.

------
frsandstone
For context - the author of this article, Ryan Dahl, is also the creator of
Node.js.

~~~
ehsankia
Thank you, that explains the "I uprooted myself from Brooklyn and moved, yet
again, to the Bay Area in pursuit of technology." with a link to node.js in
there.

------
paultopia
This is interesting:

 _The program invites two dozen people, with varying backgrounds in ML_

... "with varying backgrounds in ML" \-- does that mean that relative
beginners have a chance of being accepted?

(I have a personal stake in this! I'm an academic researcher who is getting
into ML and working on legal applications, and I'd totally apply for this if I
thought you didn't have to basically be Andrew Ng to get in...)

~~~
gdahl
I work as a researcher on the Brain team.

An experienced machine learning researcher like Andrew Ng would probably not
join the team as a Brain resident. We hire experienced machine learning
researchers and engineers all the time (see
[https://careers.google.com/jobs#t=sq&q=j&li=20&l=false&jlo=e...](https://careers.google.com/jobs#t=sq&q=j&li=20&l=false&jlo=en-
US&j=brain+research&) ) and the residency program is probably not appropriate
for people who are already experts. It is a program designed to help people
_become_ experts in machine learning.

For residents we look for some programming ability, mathematical ability, and
machine learning knowledge. If an applicant knows absolutely nothing about
machine learning, it would be strange (why apply?). We accept people who are
not machine learning experts, but we want to be sure that people know enough
about machine learning to be making an informed choice about trying to become
machine learning researchers. Applicants need to have enough exposure to the
field to have some idea of what they are getting into and have the necessary
self-knowledge to be passionate about machine learning research.

You can see profiles of a few of the first cohort of residents here:
[https://research.google.com/teams/brain/residency/](https://research.google.com/teams/brain/residency/)

See the old job posting which should hopefully explain the qualifications:
[https://careers.google.com/jobs#!t=jo&jid=/google/google-
bra...](https://careers.google.com/jobs#!t=jo&jid=/google/google-brain-
resident-2017-start-fixed-1600-amphitheatre-pkwy-mountain-view-ca-159360005&)

~~~
paultopia
Thank you so much! I may well apply in a year or two, after doing more ML work
and talking my way into sabbatical time. :-)

------
ramgorur
The colouring algorithm is very interesting, what if we give a dinosaur image
as an input after learning the network with images from all the 10,000 reptile
species?

~~~
yorwba
It will probably be colored somewhat grayish-green, since most reptiles are
small animals that benefit from being able to hide in foliage. I doubt it will
give better insight about dinosaur coloration than just having the illustrator
who made the image also do the coloring (and they at least have the option of
including fossilized pigments into their educated guess.)

------
thebouv
I admit I'm a little disappointed that "Google Brain Residency" isn't a
prototype for uploading your consciousness to "the Net".

Loved the article though. Dashed cyberpunks dreams notwithstanding.

~~~
tricolon
I was half-expecting it to be about their invention assignment agreement.

------
dekhn
yup, the very best machine learning models are still made from distilled
postdoc tears...

------
chis
This is just incredibly impressive on the surface. I wonder if it could be
used for compression of videos and pictures?

~~~
santaclaus
I'm waiting for someone to train a NN to make over compressed Spotify
streaming to sound not-crappy.

~~~
spynxic
Perhaps the solution could be approached from the receiving end, during
decompression, given some kind of information about the appropriate form of
the decompressed stream.

~~~
pbhjpbhj
Like feed in a compressed file (OGG or whatever) feed in a lossless file (FLAC
say) and have the [ML] system create an algorithm to run at the receiver that
will more closely approximate the lossless output.

This has been done before surely, like compression with varying decompression
algos the characteristics of which are sent with the compressed stream to
enable more optimal reproduction of the originally encoded data??

------
Zaheer
Great article! I especially like how its understandable and relatable to folks
not wholly within the ML space.

------
known
[https://how-old.net/](https://how-old.net/) has improved a lot

------
known
[http://how-old.net/](http://how-old.net/)

------
kgc
That XKCD feels spot on... [https://xkcd.com/1838/](https://xkcd.com/1838/)

~~~
jnwatson
That was literally at the end of the article.

