
Ask HN: How to break into AR/MR Industry - mendeza
I am in graduate school at Cornell and I am super passionate about augmented reality and mixed reality. I am in need of advice on how to break into the AR&#x2F;MR&#x2F;VR industry. I am having a hard time getting interviews for top companies (Magic Leap, Oculus,Apple), and it is incredibly frustrating since I feel I am very qualified to work at the companies in the role of a computer vision or graphics developer.<p>I have strong knowledge in computer vision and computer graphics, and I have worked on projects at MIT and Verizon building Augmented Reality apps. I keep having a hard time getting interviews for top AR&#x2F;VR companies such as Magic Leap and Oculus, and I really want to work in these companies to be at the forefront of the industry.<p>I am thinking starting to freelance for small companies and brands to build AR as several companies have reached out to me to develop AR, but I feel that working for a company that is pushing the industry forward will be more valuable.<p>Any advice will be super helpful, and I am interested in hearing about any opportunities if anyone knows of anything. Shoot me an email at hub503bub@gmail.com.
======
Sologram
Oh! After this much education, u have bias. The only MR working gadget,
Hololens is from MS. Did you try them? They are ahead of every one.

~~~
mendeza
How do I have bias? I have seen many different platforms facilitate AR and how
AR has progressed since 2011. How do you even define MR, when Google coins MR
for their VR head tracking, but magic leap talk about MR as being able to see
in front of you.

You are mistaken that hololens is the only device that does MR. Occipital does
better MR in my opinion, as they have a larger FOV and they leverage SceneKit,
an amazing platform to design 3D games and software. Have you seen Daquri and
Meta's MR headsets as well?

------
vitovito
I'm not in the current AR/MR/VR industry, but I have friends and colleagues
who are, and I worked in VR last time it was a thing.

I'd guess you're missing one or more of: (a) serious EE/ME experience, (b)
having _contributed_ to computer vision/graphics instead of just
_understanding_ the current state of the art and/or tooling, and/or (c) having
designed and built AR/MR/VR hardware and software systems from scratch instead
of building apps for existing systems or using existing tooling.

At a company that is inventing something new, the most valuable, sought-after
hires are of people who invent and advance the company's work; and people who
can interface between engineering and content.

The former means people who invent and write algorithms for OpenCV, not people
who simply use OpenCV. Or having built AR hardware and software in
combination, and therefore have a deep, practical understanding of the
tradeoffs in different generations of hardware and different methods of
projection, etc., rather than having only built apps for existing AR toolkits
or platforms.

The latter means software developers who can also work with the hardware. They
may not be _required_ to design a circuit board or engineer some optics, but
they understand the operation of the hardware, can fix it if it breaks, can
diagnose hardware problems and provide the sort of hardware bug reports every
software developer wishes an end-user could provide, etc.

(An example of this would be the technical artist in video game studios.
They're a respected, senior artist [so they can make actual art, and so their
processes to do so are reasonable and representative] who can also program,
and either creates tools for artists, or is the interface between the artists
and the tool developers.)

If you aren't one or the other, then you're a step removed from where the
company is at: inventing new things. They don't have the time or manpower to
hold your hand. You're stuck only being able to build for the things that
already exist, not helping them advance.

It's why Microsoft's #1 recommendation for non-technical designers who want to
get into HoloLens work is to become technical enough to be proficient in
Unity, if not the entire Windows 10 development toolchain. If they aren't
self-sufficient enough to be able to get all of their work onto a device on
their own, HoloLens isn't ready for them yet. It's not plug-and-play, everyone
involved needs to be able to wear all the hats.

If you can't show that you're already inventing new things yourself; or that
you have a practical, working knowledge of the entire system, the kind that
comes with having built something from scratch; then there may not be a place
for you yet.

~~~
mendeza
Thanks for the feedback vitovito! That was my reasoning for me to enroll into
grad school, so I can go beyond understanding the tools and build cool apps,
but to push the state of the field. I feel I am at point in my knowledge where
I can contribute the field of AR/VR, I just need guidance on what problem to
focus on.

The biggest thing that I would want to know is what
software/algorithms/knowledge would advance the work of an AR/VR company. From
these job postings:

1\. Apple: 3D Perception/Computer Vision Algorithm Engineer 2\. Snap Inc :
[https://boards.greenhouse.io/snapchat/jobs/586658#.WLNBhBIrK...](https://boards.greenhouse.io/snapchat/jobs/586658#.WLNBhBIrKHo)
3\. Magic Leap: [https://www.magicleap.com/#/job-
post/521175](https://www.magicleap.com/#/job-post/521175)

That most AR/VR companies are looking for people who are doing real-time
computer vision algorithms, such as 3D object recognition, SLAM, and gesture
recognition. I feel that this would be something I can focus on, but I do not
want to work on a problem that is already solved. Any advice on what are
important problems that would be of value to solve would be great!

Would implementing these algorithms and open them to the OpenCV community be
of value also? I am only in a 2 year masters program, so I am limited on time
to what I can contribute as a full-time grad student.

Any other advice on this you feel is important would be greatly appreciated!

~~~
vitovito
I have no idea, I'm not either of those types of people for the current
VR/AR/MR industry. And, even if I were, whatever I might say is not certain to
be useful to you.

Normally, I'd recommend you interview people working in the space to
understand what problems they're encountering now, and what they expect to run
up against next, but when they're _inventing as they go_ , by definition, they
don't (and can't!) know.

They could tell you their current problem(s), but neither you nor they know if
that's a general-case problem, or a problem specific to their implementation.
Plus, they'll solve it in parallel before you, because they have the
compounded knowledge of all their work leading up to it.

Only the people actually working on the problems know what those problems are,
and they don't have time to stop and bring you up to speed. You have to catch
up, or at least show you're on the way on your own.

The job listings seem to support this:

Apple wants either a PhD, which means having published papers advancing the
field; or industry experience _delivering products_ , which means having
discovered problems and then solved them for real users and getting paid for
them, end-to-end.

Snap wants a Master or PhD, or "experience implementing, verifying, and
optimizing," which is just short of being responsible for shipping a product.

Magic Leap is an internship, but they want "Demonstrated experience," aka
having built something yourself.

In this, you can't be a generalist, up for solving any problem. I'm sure
you're entirely capable of that, but the requirement is also that you can
figure out what the problems are and will be, on your own. In a new industry,
there aren't paths to follow, everyone's still cutting down brush.

They're not publishing most of their work, so you have no idea what's solved,
and even if they were, I think it's generally accepted that a published paper
(or launched product) is a few years behind what's currently being tried in a
lab.

You will probably inevitably be redoing someone else's work, but you won't
learn that for months or years.

I'm not sure there's any advice I could give except to pick a problem domain,
pick a use case, learn all the component pieces of that problem, make new and
different tradeoffs, and figure out what's unsolved for the choices you made.

