Say what you will about Zuckerberg, but this is a really cool project! Steps he took:
1. Connect his home devices (lights, thermostat, doors, music player, TV, CCTV etc) to his computer, so he can turn them on and off from it and in general automate his house from his PC.
2. Add an NLP component to his computer, so he could text instructions to it and it would apply the correct automation task. This can learn preferences too and be told about mistakes so it does better in the future.
3. Add face recognition, so it can recognise people at his door and automatically open it if they are expected.
4. Add speech recognition, by creating an app for the phone that is constantly on and listening for his voice. This is then converted to automation tasks again.
All in all, an interesting way to spend 100 hours. Calling it AI is a bit of a stretch though; this is basically linking up a bunch of in-accurate sensory parsers, coupled with some limited machine learning, connected to a bunch of if statements. Still a long way off of Jarvis, who is basically a genius by human standards!
Mark replied to a comment along these exact lines. Quote below;
Mark Zuckerberg: That's the thing about AI. It's sort of like magic. We only call things AI that we don't understand yet. Once we understand something, it's just math.
But if you'd asked someone 30 years if a computer system that you could talk to, that could see your friends and let them in, that could learn your listening habits and figure out when to make you toast, if that was intelligent, then most people would have said that would be AI. Now that we know how to do it, it's just math.
In my parents generation, and the generation that came after them (the one my siblings belong to, Gen X), I still think this differentiation between concepts of fictional AI were present. Consider for example the computer systems in Star Trek, and especially The Next Generation, and contrast how they worked with how Commander Data was depicted. The computer systems on Star Trek can be programmed using natural language, and by the 23rd century they're capable of even piloting an entire starship that a crew of hundreds would normally be required for. Yet, these computer systems never engage in the kind of inner exploration (or "character development") that Commander Data goes through every episode. This is what we're seeing here, Jarvis is more like a primitive version of the USS Enterprise's central computer, whereas "true" artificial intelligence, the kind that makes complex decisions on its own and has the capability of self awareness, still remains fiction.
Writers, at least, understood that there were different "flavors" of artificial intelligence. I have a funny feeling that at least science fiction enthusiasts also knew that AI would be separated in this way if it ever came to fruition.
Pardon my nerding out, but I think the difference between the computer system and Data was the degree of self-determination given to each system. When the holodeck was instructed to create an opponent capable of defeating Data, it created an sentient Moriarty AI. The computer system was fully capable of being a full AI, it was just never instructed to.
Moreover, they make the point in one of the episodes (3x06, "Booby Trap", IIRC) that the ship's computer's intelligence is INTENTIONALLY limited. The reason that Data is unique isn't simply due to his status as a fully sapient being. It's because of his positronic brain.
The ship's computer being limited to turning on the lights is just one of many points in Star Trek where the writers intentionally avoid using the fictional technology to its full capabilities in order to keep the plot exciting and suitable for a main stream audience. Transporters and replicators are another example: Suitably used they could be terrifying weapons.
> Transporters and replicators are another example: Suitably used they could be terrifying weapons.
One thing I was always wondering is why it took as long as ST:VOY to see transporters being used as a way to deliver armed torpedoes directly into the target.
In later seasons of Stargate SG-1 they sidestepped this problem cleverly - when humans received transporter technology from a more advanced species, they had non-overridable safeguards[0] (and an inspector on-board) preventing them from being used as a weapons delivery platform.
[0] - which, in the best fashion of this series, were overridden several times anyway. A lesson for those who think that if something is banned, it won't be used.
Star Trek, for all it's technology, was always a story about humans. Even Data's journey is about as human-centric as you can get. In that way even the recent series have mirrored an older age of science fiction ("Using science to opine on the human condition" etc).
The Culture novels managed to convince me that a story about true AI needs to not only look a little different, but fundamentally different. Not least, answering "Why still humans around?"
And yet 30 years we could do all that with a whole lot of if statements. It would just return false positives a lot more. A couple of others have replied saying the same thing as you that this is 'AI', but it's not. As soon as you try to use it for something it wasn't programmed for, it fails. It doesn't even understand what it is doing right now. This isn't me 'moving the goalposts' either, we have literally never achieved anything close to AI. It baffles me that people think this is even remotely AI to be honest.
It's a very long read, but it will give you a much better overview of AI and the different "calibres" than I could possibly try to in a HN comment.
What Mark has used here is "Artificial Narrow Intelligence (ANI)", the first of three levels/calibres of AI. Level 2 is "Artificial General Intelligence (AGI)", otherwise known as "Human Level AI". And level 3 is "Artificial Superintelligence (ASI)" which is "“an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” [Quote: Nick Bostrom].
sigh this is why I don't usually get involved in arguments about whether something is considered AI or not; it just devolves into people trying to define and redefine what the word 'AI' even means.
Final word: I follow this stuff closely and read the research articles, and what we currently have is really great, but it's still just pulling the wool over your eyes. These techniques and algorithms trick you into thinking they are intelligent when in fact they are anything but. What we have now is a parlour trick in comparison to what the brain is doing. When true AI actually comes along, no one will doubt it.
Superintelligence will be achieved by:
a) Training a system to trick the smartest people on the planet that it's smart, and then
b) Asking it to prove itself by solving the Riemann hypothesis.
I was unaware that we knew how how human intelligence works. </sarcasm>
If we can build systems that can fool us into thinking their intelligent (which I will agree that outside of limited scopes we have not), then we built intelligence.
What I see we have now is lots of bricks - often overly problem-specific (and it's important to stress this; a lot of things that get called "AI" in the news is really lots of compute thrown at super specific heuristics). To call it AI, we still need to connect them together into a system that can use those components to learn and solve problems it didn't see before, without bricking itself in the process.
> parlour trick in comparison to what the brain is doing
That's a good turn of phrase, because it speaks to the real rub between algorithmic insufficiency and hardware insufficiency. From my reading, it seems fair to say that "strong general AI" will require some product of both to be realized and that our current hardware capabilities likely fall short of the minimum requirement (even with optimal algorithms).
Were they imagining exascale computing when Quicksort was dreamed up in 1959? No. But does that make it any less important?
According to [1] there was a system in 1968 that “consistently outperformed humans when presented with the same recognition tasks”.
As someone who was into programming and electronics in the eighties, I also feel a need to object whenever people say things like “we thought it was AI until we could make a computer do it, and now it’s just math” (paraphrasing).
We (those of us interested in the subject back then) had a pretty good idea about what would be possible in the future (because of faster machines and more storage) and what would not be possible in the future (unless a breakthrough was made in understanding intelligence).
Now, I do think we underestimated how fast things have developed, and especially the resources people have access today via the internet, but none of what I have seen today would have been classified as AI by myself back in the eighties.
I also mucked around with computers in the 80s and agree with most of that. I guess a lot of whether stuff is AI comes down to how you define it. The Oxford English Dict has:
>[mass noun] The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.
So obviously by that speech and Zucks type of face recognition are AI. I note the '68 system was hybrid and required humans to measure the faces and enter data for where the features are so it was more analysing numbers.
But as a data point, you could do it 10 years ago. On a cheap desktop PC. And without Internet connection or cloud computing.
Source: around 2006 I made a Star Trek-like music control system with Microsoft Speech API and an electret microphone hand-soldered to a long cable and an audio jack. With ~30 minutes of training on my voice it worked as reliably in noisy environment as Google works for me now.
I think what's particularly great about this in a way is that it's not too remarkable. I'm not knocking it, but it's awesome to be around when creating an natural language, speech controlled, available anywhere system with face recognition to automate your own home is something that can be done in 100 hours.
I agree with the sentiment, the hours of course are the assembly time. When people ask me the role of an architect I try to impress upon them that well architected subsystems will be readily composable into a variety of forms and uses. That amplifies the millions of hours of research and development into a tool that can be wielded by a motivated individual or team to tremendous advantage.
95% of the time, good software creation is really just construction, not engineering. There are great paradigms and solutions to the majority of problems. There's room for design and UX, but you shouldn't need to custom engineer the walls out of carbon fiber or some aerospace composites unless you're launching the house to Mars. The engineering aspect really only comes into play when no components meet your needs.
The construction metaphor is full of great wisdom to be gleaned as software developers. And I'm not just saying that in hopes I didn't waste all those hours watching History Channel documentaries. :-D
For me a well designed house is actually, you know, about the design. And whilst the construction is important due to standards it is largely irrelevant.
The analogy to software is apt. Well designed, well architected software is so important. Anyone can write syntactically proper code. Far harder to design a system that has just the right amount of extensibility, performance, elegance etc to meet the requirements without going over the top.
Ask owners of beautifully designed Frank Lloyd Wright houses how happy they are with the leaky roofs that plague them. Wright didn't understand construction which lead to many practical flaws when people tried to actually build his designs.
When you're talking about civil/mechanical engineering, good engineering is 'just construction' too. Your job is to take well known, tried-and-tested techniques and combine them in a safe and reliable way to produce a design which meets all of your requirements and can be built for your target price.
Engineering isn't really a place for more than very limited creativity. That place is R&D.
I think the remarkable bit is that he did this while being the CEO of Facebook. There's not a huge number of corporate CEOs who still code at all, and have time to put in a big project like this. And as someone who's worked with writing some of the same sort of code, it's really enjoyable to see someone drastically more talented making many of the same observations and encountering similar challenges.
Oh come on, it was clearly both. Try to tell us this excerpt is not an advertisement for the facebook SDK:
> I'd learn a lot about the state of AI this year, but I didn't realize I would also learn so much about what it's like to be an engineer at Facebook. And it's impressive.
> My experience of ramping up in the Facebook codebase is probably pretty similar to what most new engineers here go through. I was consistently impressed by how well organized our code is, and how easy it was to find what you're looking for -- whether it's related to face recognition, speech recognition, the Messenger Bot Framework [messenger.com/platform] or iOS development...
The rest of that paragraph goes through all the other developer tools you can sign up for in the facebook ecosystem, including convenient links. Then it continues:
> One of our values is "move fast". That means you should be able to come here and build an app faster than you can anywhere else, including on your own. You should be able to come here and use our infra and AI tools to build things it would take you a long time to build on your own.
It was a neat project, not denying that, but there is no free lunch. The CEO of a Fortune 500 does not write up a 5-10 min read without plugging his interests.
You can enjoy what he's doing, but also recognize what it is.
I think it's worth pointing out that he started his mobile efforts by talking about how easy it was to make a Facebook Messenger bot, something they're heavily pushing... and then said shortly thereafter, to do what he wanted, he had to switch to making his own app, because Messenger couldn't do what he wanted.
If a Google exec wrote this post, it would be about how amazing a Google product was as a solution to the problem, and that would be the end of it. But here's Mark saying "you know what, Messenger is great but I needed more". And maybe that could turn into some pretty big advice for his Messenger team on what to add next, but he seemed to be pointing out where the industry (including his own company) could do better.
I don't think he intended it to be marketing, but kudos to him for being able to spin it as a way to promote Facebook too.
I see it more as R&D than marketing, personally. His personal experiments may likely hint at new directions in which he wants to take his company. Depending on how you view Facebook, this could be terrifying or exciting.
One way I see it is: Facebook knows a lot about your (online) social interactions. Home-based AI would need to know a lot about your private behaviors and preferences. If they marry those two worlds, that's a heck of a lot of behavioral information to have. Google has a big lead in this area already, so if Facebook goes this direction too, it would be an interesting to see how it plays out.
> it's awesome to be around when creating an natural language, speech controlled, available anywhere system with face recognition to automate your own home is something that can be done in 100 hours
It's not creating. It's just copy+pasting and glueing.
The actual creation of these systems took decades of research.
I don't think calling it AI is a stretch. It's a system with sensors, actuators, a knowledge base, and a ruleset; plus it can learn and adapt its ruleset based on success/failure. That's the textbook definition of AI.
You added one thing in the list that makes me agree - "it can learn and adapt".
Jarvis in my house uses sensors, actuators, lights, switches, etc - but mine is not AI. It only does what I tell it how to do. I'm pretty happy with the results, but it can't learn a thing.
The latest thing I "taught" my Jarvis to do is stream the security camera to the TV with a voice command. Now if I only had time to look into how to tell it to stop...
That being said, I don't think the ability to learn and adapt. This is entirely a statement on semantics, but if I made a tic-tac-toe playing program that had every possible game state pre-programmed in, I would not hesitate to call it a game playing AI even though it lacks the ability to learn.
In the general programming industry over the last several decades, "artificial intelligence" has also applied to simple decision-making algorithms, even as simple as playing checkers. Arthur Samuel [1] was renowned for his work on a checkers AI circa 1959.
As a follow-on to the above comment, in AI there is something called expert systems, which is just a set of IF-THEN rules which emulate the reasoning of a human expert: https://en.wikipedia.org/wiki/Expert_system
Learning/inferencing/adaptation not included or required.
This is something I like about AI in the context of video games; the technology has improved, the complexity has increased, but the meaning of AI in this context has stayed constant.
The same can't really be said of AI in most other contexts, where the goalposts continually seem to move to whatever is currently just beyond our reach.
> Add face recognition, so it can recognise people at his door and automatically open it if they are expected
I wonder if you could open his door with an A4 printout of his face. That's the sort of thing I'd worry about creating this system.
Edit: Forgive me HN, it looks like a lot of people have wondered this. I only commented so high up because I commented before reading all the comments! Go upvote the more creative/indepth versions of this question rather than this one :)
There's a decent way to handle this method: use one of the Intel close RGBD cameras. I think they go by RealSense.
Now, do facial recognition with the depth component. "Oh look, paper. How quaint... NOPE!"
You could also log all people coming by with a known-unknown axis. You could train the system so that all they need do is look at the door, and they have the appropriate access they need. And it could also, instead, default in a "pass through to app for user to decide".
Don't think of this as a huge monolithic system... But instead of dozens of component parts that can work together. Then, the problem is certainly tractable. And then adding things like Google-Now, Alexa, Siri, and similar is no longer damn near impossible.
I particularly like Node-Red for this very project. It is javasript, however the big positive side effect is easy implementation with all sorts of nodejs APIs.
Ooh you could do all that and instead of a flat out denial based system fall back to a fingerprint scanning doorbell. Probably a weird conversation to have in the pub "Alright lads I'm gonna need you all to scan your fingerprints for me"
I already wrote a system that can take new facial inputs, classify you as "new user", and put your data in a database. All you need do is go up to the door. I can easily view the timecode and enter your details (name, phone#, email) at a later date.
Indeed. Security isn't a binary. There's plenty of layers. And we went from
"Print picture" to
"Compute Sfm 3d object from thousands of photos, and have access to a 3d printer that large that can print human-sized heads, and paint it appropriately in the correct color-space as to trigger the right facial recognition"
It's then trivial to add a feature of 'Blink' and 'Smile'. We have public, free haar and lbp cascades for those. It would be a few lines to add in a check that requests those 2 actions.
"Then you have to print an articulated skull and musculature that activate the appropriate smile/blink responses, with the correct coloring across the face."
And then hope that the video feed of the disjoined head trying to pass as human isn't just sent to the butler or a family member for "quality control"....
A somewhat surprising feature of suburbia in the US (for those of us that come from less safe countries) is that someone like Mark could ever have unbarred windows at street level directly facing a public street. In many places, this kind of recognition system would be at the outer gate of the complex, surrounded by tick concrete walls ending in an electrified fence or similar[1]. At that point, the face recognition does become the weaker link.
Also, even in US suburbia, smashing the window is likely going to trigger a burglar alarm and get the police called to the location, whereas bypassing the face recognition system will not.
[1] For a non billionaire house: thinner concrete walls ending in broken glass or decorative-yet-functional metal barbs, outer wall shared by a 10-20 homes neighborhood.
> Calling it AI is a bit of a stretch though; this is basically linking up a bunch of in-accurate sensory parsers, coupled with some limited machine learning, connected to a bunch of if statements.
The bar for "AI" (as distinct from "Jarvis-level AI") isn't high. Heck, video games have had "AIs" since the 1950s: https://en.wikipedia.org/wiki/Artificial_intelligence_(video... Not sure if everyone agrees with that definition, but's it the one in common use.
If so, then Machine Learning is just predicting the next possible result using the statistical probability of previously studied data, and could not qualify as AI either.
AI is being broken down in to so many parts, some that are invisible to us, that: either we abuse it to the point of over simplification, reclassify it as common sense, and redraw the finish line of where AI starts; OR we start calling everything we do a version of AI from computers turning off the lights to balancing the controls in the first Apollo missions.
If you're in a position to. It sounds like he plugged into Facebook's face recognition, for example, and I don't think that's available as an API request ('is this one of my friends')?
He's still a hacker. CEO of one of the biggest internet companies of our time and he still hacks.
I explain to people all the time, true hackers are like true writers. I've heard writers talk about the same thing. It's like an urge of creativity inside you, you have to release it into something.
You can also see in the comments section, he's replying to people and providing feedback.
Gates' did something similar (as far as high-tech'ing your abode) in the 90s that I thought was also very hacker-ish.[0]
“First thing, as you come in, you’ll be presented with an electronic pin to clip to your clothes. This pin will connect you to the electronic services of the house,” Gates wrote in The Road Ahead. “The electronic pin you wear will tell the house who and where you are, and the house will use this information to try to meet and even anticipate your needs — all as unobtrusively as possible.” At the time, Gates speculated cameras might one day supplant the pins, as they’d be able to do facial recognition. He explained that the house would learn your preferences and adjust to your needs. Your affinity to dimmer lights, R&B, and Monet paintings would mean the lamps, music system, and screens would all adjust when you enter the room.
Funny thing: Based on the screenshot in the post header, he's using async/await and therefore the Hack language (instead of vanilla PHP): https://docs.hhvm.com/hack/async/introduction
I'm not sure why that's funny or remarkable - Facebook invented and is a heavy user of Hack, it's only natural that he's familiar with it and chose it.
More than anything else, what stood out to me was his discipline for knowing how to manage his time in a way that enabled him to build something like this while still juggling the n number of other things on his plate.
I dunno--like any CEO or even VP or SVP-level executive, I'm sure he can afford executive assistants at work and nannies/assistants/drivers/servants at home to take care of all the normal mundane aspects of life that suck up so much of normal people's time, getting in the way of side projects.
I'd guess he has more free time than the random "3rd engineer from the left" on one of his company's projects.
No, it doesn't work that way. CEOs/VPs never had house chores as major time sink. When you are at that level, you are responsible for 1000+ employees, 100s of teams and dozens of projects. Each project and team requires at least quarterly review, funding, strategy, motivation, vision, market analysis, HR, budgeting, PR, partners and so on. A typical CEO/VP works well over 12 hours, often at least 14 hours and many work one weekend days. Their calendars are filled with meetings down the hierarchy and up the management chain and across external people - usually starting at 8 AM and going down all the way to 8 PM. They have to also keep meeting customers, major shareholders, partners and so on in addition to internal people. Its extremely hard to find time for them because everything is important and not doing something can eventually lead to big disaster and loss of millions of dollars. Most developers think these people are lazy and play golf all day long while their underlings do all the work. If you look at their schedules, you had be quite wrong. Zuck is able to make time for all these is extremely amazing and should get respect from anyone for his discipline. I can't recall any other CEO leading same size of company being able to do this level of hands on hacking in past or present.
>If you look at their schedules, you had be quite wrong. //
Do you have any evidence to show your affirmation that CEOs work really hard and "often at least 14 hours and many work one weekend days" is true and other's affirmations that they "play golf all day" is absolutely false?
What makes you think that working and playing golf all day are even different things?
Larry Ellison was renowned for doing massive sales deals of Oracle on golf courses. If you get in 18 holes and leave with $10 million, were you not working?
At executive levels a part of the job deals with relationships, and those are work. Much less so the "nose to the grindstone in front of the computer" sort of work.
Clearly I've been thinking about it all wrong. When the CEO eats a lavish meal with a CEO from another company, they're not having lunch they're working hard; probably just as hard as the peons doing the manufacturing, cleaning the plant, driving the forklifts or whatever. Just because the sales teams have already negotiated the deal and the CEO only has to eat a meal and sign their name doesn't mean they're not working hard, at all.
Indeed if you get invited away to a person's private island and then they later buy from the company you "work" at then that holiday was far more work than someone spending 12 hour shifts, standing, poring over circuit boards with a soldering iron or hefting rocks with a pickaxe. That's why the management deserve 200 times the salary of the workers. /s
Truly the relationships aren't important unless the CEOs make it that way. They could just do business, decide they need X amount of pressed steel, sign the contract and not play golf and have a luxurious meal. But instead they decide that only companies that kowtow to this ideal of greasing the wheels of industry by fattening the C-grades up should win the contracts.
>If you get in 18 holes and leave with $10 million, were you not working? //
No, you're not working, you're exploiting the work of others. What function did the 18 holes of golf serve except to reinforce a despotic nepotism.
The idea that this is work is interestingly both anti-capitalist (it's an inefficient aberration) and anti-socialist (it's setting people over others but providing no useful function).
What are you willing to consider as evidence? The internet is littered with hundreds of articles, interviews, anecdotes of CEOs working long hours to build up the company, bring in new business, keep existing business, etc (and if you're feeling a bit grumpy today, yes, also to enrich themselves).
An old but concrete example: Andy Grove, in High Output Management, describes a fairly typical 8 to 5 day in which he works until he's tired, with level of fatigue vs level of "doneness" being the deciding factor of when he ends his day.
No kidding. I find it impressive both that he put aside (not found, this had to be a very deliberate decision given the demands on his time) 100 hours to do this, AND that he got all of it done in 100 hours.
Is there anything written about his personal schedule/habits? When does he go to sleep at night? When does he wake up? What kind of hours does he work?
I mean really, he has a kid, is CEO of a huge company, exercises, and still has time to do this project. My hands are more than full trying to exercise, learn Chinese, and run a small company, I have no idea where he finds time to raise a kid in there. Incredible.
When you are a billionaire, you can certainly outsource all that mundane stuff (like raising a kid) to extremely capable helpers. He only has to do exactly what he wants to do. Pretty sure Zuck isn't changing any diapers or walking the dog unless he decides he feels like it right at that moment.
Yeah, I see what you mean... I guess I meant that compared to running a multi-billion dollar company, changing diapers or reading Cat in the Hat for the millionth time seems pretty mundane.
You're missing what raising a child entails if you think that's all it entails. If you raise a child without the deep emotional investment that comes from a parent, you're not going to raise a child well at all.
I know, I have a two-year-old boy. I'm just saying he can get more of a break than a regular person that doesn't have nannies and whatnot.
I haven't had two hours to myself since the day my kid was born, so the idea of having 100 hours to do a fun project like automating my house is completely out of the realm of possibility for me.
But he's also showing videos how he interacts with his kid. He seems to put some emphasize on his family. Perhaps he just managed to delegate much more than most other CEOs.
Don't forget that as CEO, likely it is true that if he had a question about direction or implementation, a hundred engineers would almost immediately respond to him. This 100 hours' investment isn't like you think of some lone hacker poking around SO; he's gluing something together with instant handholding available.
Apart from the whole business luminary/billionaire thing, I think Zuck is a pretty cool and insightful guy. I like both the idea of having a "challenge", as well as the specific ones he's done: running, meeting a new person each day, learning Mandarin, doing a cool side project with his house. (I get to hear about a lot of this as one of my best friends works as fb)
At the risk of sounding a little bit self-aggrandizing, I often think about how similar the two of us are, not because I'm particularly special, but just that he's a torch-carrier for guys like me. I'm a month apart from Zuck in age, I'm about to get married to a woman with a serious career, and Zuck and I both grew up in the eastern US (I'm from Chicago). I had a Chinese girlfriend growing up. My family wasn't rich but we also weren't poor. I worked at Microsoft out of college, something Zuck said he might've done if he didn't found facebook.
He's pretty talented though. I definitely didn't create wirehog. I also didn't go to Exeter or have the quality of education he did, either at Exeter, or later at Harvard. I have to wonder how much that stuff influenced his later life, and whether I should try to get my kids into those places.
Controversies aside, I think he's doing an admirable job of casting the tech industry in a positive light, with his positions on taking time off from work, and trying to hire inclusively. I don't use facebook as I have genuine concerns about privacy, but I think he's a real idealist who thinks making the world more "open and connected" is a genuinely good thing. I'd love to meet him some day.
Anyone setting out to do something like this will quickly see how much of a hassle it is to exfiltrate data from relevant services and your own devices. APIs and SDKs cover a lot of use-cases, and that is all great and commendable, but we don't truly own our own devices, data, or the algorithms that make them useful. I think this will continue to hamper our ability to have a truly useful 3rd party (your home in this case) orchestrate your digital life.
A worry about reading about Zuckerberg digging into this area is that anything Facebook does will likely be within their walled garden. There is certainly a glimmer of hope they'll make something on the "open" side, but I wouldn't get my hopes up.
I wonder to what extent having great advances in this area by proprietary cloud services will hamper the advancement and adoption of libre alternatives.
> My experience of ramping up in the Facebook codebase is probably pretty similar to what most new engineers here go through. I was consistently impressed by how well organized our code is, and how easy it was to find what you're looking for -- whether it's related to face recognition, speech recognition, the Messenger Bot Framework [messenger.com/platform] or iOS development. The open source Nuclide [github.com/facebook/nuclide] packages we've built to work with GitHub's Atom make development much easier. The Buck [buckbuild.com] build system we've developed to build large projects quickly also saved me a lot of time. Our open source FastText [github.com/facebookresearch/fastTex...] AI text classification tool is also a good one to check out, and if you're interested in AI development, the whole Facebook Research [github.com/facebookresearch] GitHub repo is worth taking a look at.
Face recognition and speech recognition are the most obvious pieces that look to be proprietary, but there is lots of other useful open stuff that he explicitly links. FastText looks like it would be really helpful for at least the natural language piece of Jarvis-like applications.
I haven't really dug into open source speech recognition, so I can't speak much to that side of things, except observing that advancements favor deep learning and cloud-based services over efficient on-device detection (aside from very specific "hey siri"/"okay google"/"hey alexa" recognizers). When it's not a library that can be included, but a service that must be maintained, it'll be that much hard for free and open solutions to get adoption.
For face detection & recognition, some of the greatest advancements are getting acqui-hired and locked away (e.g. Facebook acquiring Face.com) there is still progress being made with tools like OpenCV & Dlib. I just worry progress in the open-source space will lag behind and continue to be difficult to work with, as the delta grows between proprietary capabilities and open-source options.
Hey, I also work my ass off to the point of getting sick and I have accomplished nothing. By that I mean I have not made any money. I've listed some of my projects here and not even a single up vote. I don't like it but it is what it is. I'm simply trying to say that I'm no less ambitious, and that is probably true of a lot of us here. The gods of luck have simply not been on my side, yet.
It's very true that how hard you work doesn't correlate directly with success. That said, learning how to work hard is one of the best things you can do to become successful. Once you master that, you then have to learn from each experience. If some of the things you're doing haven't worked out, try to examine them and figure out where you went wrong. Feel free to post some here for others to chime in.
Also, don't necessarily think not climbing to the top of HN means you're not successful. It can be a popularity contest (Zuck will always make it to the top), and it's also somewhat random. Often the content which makes it to the frontpage was upvoted by friends, or reposted many times, I wouldn't take it too seriously.
I just want to point out that the upvoted by friends part is against Hacker News policy and not encouraged and I personally consider them as very unethical:
> Can I ask people to upvote my submission?
> No. Users should vote for a story because it's intellectually interesting, not because someone is promoting it.
> When the software detects a voting ring, it penalizes the post. Accounts that vote like this eventually get their votes ignored.
When I build side projects, I just care about learning a new language, framework, library etc...I couldn't care less if it got me upvotes on here or anywhere for that matter.
Well, I want to be successful the same way Mark is successful, I want to make money. Having money I can then work on even more ambitious projects. Isn't that what we all want to do?
Getting thousands of people to look at and/or comment on your project is one proxy and one first step. You have to get people interested on what you are working on or else you will never make money off of them.
Unfortunately this is my throw away account. I speak my mind so I come off as a troll. I'll post the latest project I'm working under my real user id in a few months.
I'm kicking myself since this appears to be a good opportunity for some self promotion. Fuck Me.
Edit: This is making me realize that I should stop being an anonymous coward and put myself out there to try to make my projects and I known to the world. I'll make this my new year's resolution.
Anonymous Cowards are something from Slashdot, right? I don't think that applies here.
I also have a few different accounts, while keeping my main account more 'professional'. I do it for a few reasons: the HN hive mentality hates dissenting opinions in general (doesn't matter how well written and based on facts), everything has to be super politically correct these days and the 'cultural fit' mantra could mean I would have a disadvantage getting a job (since many employers/startups will check your comment history to decide if you're a "fit", while simultaneously posting on their blog about diversity efforts).
I'd say keep using your throwaway accounts. It's not the end of the world you didn't get to promote your side project in this thread.
Is anyone else seeing the possible security risk shown in the 'Vision and Face Recognition' section?
The main concern I would have is someone spoofing the visual recognition system with something as crude as photographs, or something more advanced like a mask.
He states in that section, "Once it identifies the person, it checks a list to confirm I'm expecting that person, and if I am then it will let them in and tell me they're here." My first thought is someone could see if there's a friend/acquaintance who routinely visits, and then spoof their face with a mask. Let alone if the system automatically lets Zuckerberg in, then that's all that would be needed.
this is a guy who has multiple armed security guards outside hit home 24/7. Plus the fact he almost certainly has staff and assistants who are coming and going from the house. There is no way you make it to the front door if you are not expected, this is just to save his staff from having to use the key or for someone to open the door for them. This would never be secure enough for a normal persons house
This is a reference to a scene from the Michael Moore movie Bowling for Columbine, where he is walking in a Canadian neighborhood and just opens the front door of someone's house. I couldn't find a better clip than this one: https://m.youtube.com/watch?v=pjJbtb4Bs0Q
Huh, never lock the door when I'm in the house, except at night ... is that unusual? Live in a smaller UK city. Perhaps if we had anything worth stealing it'd be different.
Is it perhaps the US Americans are the peculiar ones and other countries are more like Canada?
For anyone wondering, the name is not a result of a mistake, apparently 'beta' was commonly transliterated as 'b' in the past (as opposed to 'v', the sound that it stands for).[1]
Rather what the hell happened to the guy that had tape on his laptop camera and microphone. Suddenly he's got an always on listening device. How does one go from one to the other?
Securing a local network is orders of magnitude easier than a laptop which accesses the internet constantly.
Stick the cameras on a vlan only accessable to the (local) servers doing the face recognition, stick the servers on a vlan that has no direct connection to the internet.
Compare this to a laptop which is connected to the internet with multiple attack vectors (browser, email client, etc...)
Does he have some custom build network router that he trust? Putting tape over your camera means you know enough to not trust the laptop hardware neither the OS or browsers like you said.
But he says: "We use .. Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max". So all of his appliances do get outside.
I did a double-take on Nest cam. So he's streaming videos from his house on the internet with a closed source hw and software.
I didn't read the part about Nest to be fair but with some basic network design you can easily segregate networks and reduce your attack surface massively, even if you're using internet connected devices (seperate vlans, use http proxies with ACLs, no inter device communication where not needed).
The difference between a switch and a laptop is that your switch isn't running browsers with 0days found regularly, no malicious JS payloads, no phishing emails.
To exploit a switch you generally need access to the management interface, something anyone who has any experience with networking does not put on the same network (virtual or physical) as laptops, iPads, televisions, or internet connected cameras.
I agree with what you are saying, however tape over the camera means you don't trust the OS or the hardware manufacturer with a console command like modprobe to actually do it's thing and disable the web camera.
Sounds like he knows where the data being recorded is always going and what's being done with it. Pretty different from the chance that a camera/mic on a laptop might be randomly sending data to some third party.
I think there's some law or truism that every geek who becomes a billionaire spends more time automating their house than it would take to just walk over and turn on the light switch.
This is incredibly cool. I can't overstate the effect of having a CEO who is an active hacker on the tooling and infrastructure in the company.
On top of that, this would give Zuckerberg so much insight into so many things - the pain faced by developers, the pain faced by the users, how users would want to use AI products, why developers are not able to build the products users want. And this plays well into his strengths as a CEO. There aren't many (any?) big company CEOs who can do this.
It's not voice centric like 'jarvis' would be, but it bridges most of the existing smart devices that exist right now and let's you do a plethora of many more things than 'jarvis' does. You can for example add gps tracking to your phone and have it control your thermostat for when you're close to home, or turn on the the lights at sundown and you're home, or you can program it to shut off the light when something is playing on the chromecast, etc. It supports alexa skills as well, so you can have that be a trigger for whatever you need done.
Hass's oobe is basically just a dashboard for your smart devices, but with some tinkering you can set up rules and customize it for your personal needs.
For me, the primary takeaway was: "This suggests that future AI products cannot be solely focused on voice and will need a private messaging interface as well."
> This preference for text communication over voice communication fits a pattern we're seeing with Messenger and WhatsApp overall, where the volume of text messaging around the world is growing much faster than the volume of voice communication. This suggests that future AI products cannot be solely focused on voice and will need a private messaging interface as well. Once you're enabling private messaging, it's much better to use a platform like Messenger than to build a new app from scratch. I have always been optimistic about AI bots, but my experience with Jarvis has made me even more optimistic that we'll all communicate with bots like Jarvis in the future.
I've noticed that myself with Siri. I avoided using it for years but finally gave in and enabled full Siri access so I can ask simple questions when the phone is locked. Our nightly lows in Florida have been all over the place lately and my birds prefer to stay in the porch when it's 65F or higher but if it's lower, I have to bring them indoors. So my most common question is: "Hey Siri, what's the low temperature for tonight?" Unlocking the phone, swiping right, clicking on the weather icon, waiting, and then parsing the screen to locate the low temperature is a lot more involved.
However, now that I have Siri, I try to use it for more things via voice but it doesn't always work because (1) Siri is not that good, (2) it is awkward to try to find out what exact words to use for Siri, in front of other people, and (3) there is no autosuggest for search when speaking, which helps a lot when manually typing.
Personally, I don't think voice + text is enough. Facial/body gesture recognition will definitely have a part to play in this long-term.
> One thing that surprised me about my communication with Jarvis is that when I have the choice of either speaking or texting, I text much more than I would have expected. This is for a number of reasons, but mostly it feels less disturbing to people around me. If I'm doing something that relates to them, like playing music for all of us, then speaking feels fine, but most of the time text feels more appropriate.
I've found myself doing the same thing. For queries and commands which involve only myself I prefer to type unless I'm alone, but for questions which involve the people around me asking verbally seems more useful.
I think a display in my bathroom (smart mirror style) which would alert me if the night's low would be low enough to bring the birds in would work best for me. Plus maybe a push alert to my phone if the temp got low and I hadn't ack'd the alert on the smart mirror. But then I tend to brush my teeth before getting undressed :-)
Anybody remember that NeXTMail included voice annotations? Seeing that feature almost never used made me realize that text for messaging will always have its place.
Quick startup, a good amount of CLI-specific APIs, available on many servers already, and no compiler or huge amount of tools needed.
It's a bit verbose for little scripts that should be in something like bash, but it's a great alternative to something like python if you are more familiar with it.
In addition to this, if a lot of the libraries you're using are written in PHP, it makes a ton of sense that command line scripts using those libraries would also be written in PHP.
A decade ago, I used it like this for years at work. PHP is/was an awesome CLI script language. I always felt this was one of PHPs stronger use-cases, but lesser known and not pushed anywhere. Node has taken this spot, for me, but I'll never look down on PHP... It was very good to me.
Simple text parsing in PHP7 is also faster than in Python, Ruby, Perl and NodeJS. I recently wrote some 1-liners to parse a 19Mb log file with a regex on my 2010 quad-core Mac. Python/Ruby: 2.2 secs. Perl: 1.4 secs. NodeJS: 0.9 secs. PHP7: 0.4 secs.
Ruby
IO.foreach('logs1.txt') {|x| puts x if /\b\w{15}\b/.match? x }
Python3
import re
with open('logs1.txt', 'rt') as fh:
for line in fh:
if re.search(r'\b\w{15}\b', line): print(line, end=' ')
Perl
open my $fh, '<', 'logs1.txt';
while (<$fh>) { chomp; say if $_ =~ /\b\w{15}\b/; }
close $fh;
NodeJS
var fs = require('fs'), byline = require('byline');
var stream = byline(fs.createReadStream('logs1.txt', {encoding: 'utf8'}));
stream.on('data', function(line) { if (line.match(/\b\w{15}\b/)) console.log(line); });
PHP
$fh = fopen('logs1.txt', 'r');
while (($line = fgets($fh)) !== false) {
if (preg_match('/\b\w{15}\b/', $line)) echo $line;
}
fclose($fh);
BTW, I'm definitely not saying this to knock Zuck since this was clearly more of a fun challenge / learning exercise for him (it's awesome he did this), but for anyone interested in setting up a similar thing, you can accomplish much of the voice/music/automation parts without writing much (if any) code using Alexa, SmartThings, IFTTT, etc.
Alexa can be a little frustrating sometimes but overall once you get her dialed in it feels like the future.
I even set up a pet feeder like Zuck mentioned (http://www.super-feeder.com/ you can use a normal home automation 110V relay but I wired up a 12V relay to a C.H.I.P. for more fine-grained control)
Yeah I was thinking the same thing. I use Alexa for controlling my A/V, lights and music. It works relatively well without too much configuration. Although I think it could go a step further but building some macros and rules.
I honestly screenshotted the python comment and framed it on my wall. I love python, my entire startup's backend is coded in python, and getting zuck to give a direct shoutout to any language other than a PHP family framework made me elated.
Amazed by how much work he has done in 100 hours while running one of the world's largest internet company. If anyone interested in trying out some of the "Jarvis" without hacking, checking out https://saymosaic.com. We have put together a tool to enable you interact with your smart devices/services with either voice on Echo or text on FB messenger. Currently, it is in beta with more integrations and aggregated services to come.
This is one of the most insightful, down to earth articles I have read about AI.
I hope Google/Amazon and all of the people in ML can take some time to read it.
Main Important Points:
0. Build a simple, easy prototype first. Then add better functionality later when you have finished the simple things.
1. Users will be using text over speech more than we think.
2. Users always have their phones with them. This suggests that assistant control should be done via the phone, rather than through fixed hubs around the house.
3. General AI is more akin to a limit point, rather than some attainable state.
4. Users need to be able to interface with training processes of their assistant to create more personalized interactions. A new user-machine command protocol may be invented soon to create an interface for this.
5. New data protocols may be necessary to to jump start AI and IOT together.
6. Aggressive AI will be needed to make AI more hands-off in the future; more autonomous.
7. More advanced mathematics and internal representation will be needed to allow AI systems to obtain and apply domain knowledge in isomorphic tasks.
I built something to similar this, using a Raspberry Pi, reverse SSH tunnels, Phillips Hue, Nest, and a range of sensors.
Part of the project was an admin panel, to oversee the status/health of all the systems, as well as a manual control panel. My biggest concern was security. Granted, it was an internal network in my house, but if anyone got access to it, they had camera feeds, and practically full control of my house.
I wonder how someone like Zuck protects this system, given all the nefarious people who I'm sure would love access to it.
FB as a company has probably invested in network security for him / other high level people. He already has physical armed security so it wouldn't be surprising that they would protect him / his family digitally to prevent hacking which could grow roots into his privileged access to FB's resources.
Chances that Facebook will strike a deal with Disney to adopt the Jarvis name and Paul Bettany voice and roll out their own A.I. helper personality in the future to compete with Siri/Alexa/Cortana/Google Now eventually?
Surely it's just a matter of time before someone gets a blank cheque and hires Paul Bettany for whatever is required to voice one of these systems. I'm through and through Google to a fault, but that would seriously sway me. I'd definitely give the Bettany system a go!
I was frankly surprised at how hands on Zuckerberg is. Brilliant.
Also, the most important takeaway, and one that I experience on a day-to-day basis, was the idea that most ML systems currently are heavily over-fitted and scale is going to be a huge challenge.
I think many developers have not yet realized that you dont need a PHD in mathamatics to creatively and effectively use modern ML frameworks. When (most) devs need to store data they download there favorite DB, not roll there own search algorithm.
My issue with this is calling it 'Jarvis'. Jarvis increased Stark's ability to build new, cool things. That's interesting. Something along these lines--also inspired by Jarvis--is something that interests me greatly. Regardless of implementation, the end goal of this project is boring in comparison to Jarvis' actual purpose.
Personally, I find the idea of artificial intelligence both silly in some respects, and anti-human in others. There is no benefit to humans in replacing them. Any benefits are more about specific humans and taking over the world, as in human's advantage relative to other humans. It is the same as nuclear weapons. It doesn't help humanity in any way, it just gives certain humans power over other humans. Also, most of the time, artificial intelligence is no match for human intelligence. One of his goals was to get the AI to "learn his tastes", but to me, this just means he will be given poorly selected music, as what you want to listen to is much more complex then "learning your tastes".
I'm not a fan of facebook or Zuckerberg, but I do appreciate it that he still keep challenge himself.Maybe some of his projects is not really "AI" stuff. But it's not easy to keep coding when you are CEO of a big company.
Mark is at a place that he can really envision and execute a lot of things relatively easily that is unimaginable for people who are struggling for basic acceptance and success i.e most of us.
Success usually begets success and more determination.
Trust me I do. I was just trying to highlight just how successful people find time and resolve to do things which a lot of others cannot even imagine doing - like learning Mandarin or building Jarvis.
> If it gets the mood wrong, I can just tell it, for example, "that's not light, play something light", and it can both learn the classification for that song and adjust immediately.
That sounds awesome.
Although - and I know this isn't a Facebook project - Facebook is very low in the order of companies I trust with home automation/AI, if he decides to make Jarvis commercial (probably necessitating a name change!) I think I'm not sure I'd trust it even if ostensibly "not a Facebook company".
I'm viewing the article without logging into Facebook. In the comment / response section where "X, Y, Z, and a billion others liked this", the first person is 'Priscilla Chan', which I'm guessing is the same person that is Mark's wife.
I'm curious, is that priority computed based on Mark's circle of friends? Or, are these values hard-coded / special-cased for the CEO, and potentially other individuals aswell?
Now i don't know how much of this was his "Personal Project" but to me it seems more or less as facebook's or better yet mark zuckerberg's marketing strategy to get into the Home Assistance field. I am sure Facebook's Personal Assistant is not that far.
I've wanted to test something similar - holding up a giant "Speed Limit: 75MPH" sign when a google car drives by on these little 30mph Mountain View roads and see what happens.
Probably nothing but I can't shake the comical thought of one of those little bugs rocketing to turbo mode while the dude inside is frantically slapping the big red "stop" button.
UK motorways have varying speed limits communicated via road-side signage. When repairs are being made there are sandwich board style signs that say "dead slow loose chippings" [chippings being small stones] or sometimes they have a 15mph label on a similar sign.
You'd have to guarantee perfect information from road repair crews, at least, if you were going to ignore situational signage.
Hopefully that's a sufficient well-known trick to be aware of, given its appearance in 1997's "Fifth Element": https://www.youtube.com/watch?v=nah_3vO0uhM -- Matthieu Kassovitz as the low-tech robber trying to trick the hi-tech security system.
"For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other."
It is important to note that Zuckerberg did not claim any original contributions, nor does he name any of the non-Facebook libraries used, denying the acknowledgement of others' work which he relied upon. For all we know, he used some off-the-shelf libraries for face recognition, language processing, and speech recognition, while having little to no understanding of how they actually work. This is what a lamer does, not a hacker.
There is a huge gap in public perception of the competence of the elite in the tech industry. People will get the wrong impression that Zuckerberg is an expert on any of these topics. It reminds me of the antagonist of Ex Machina who is portrayed as CEO of a giant software company, AI expert, and robotics engineer, all in one person.
> ... reliably delivering push notifications, managing identity and permissions for different people, and more. You can learn about the bot framework at ...
For a moment I thought I was reading an infomercial.
I did this for my home at a much smaller scale. Using a rasberry pi connected with a IR emitter I can control light switches in my house. Got the cheap $20 for 5 IR sockets.
With computers getting smaller and faster exponentially, a lot of science fiction is quite close.
The hard part is how we can handle this new power. Facebook face recognition can identify me anywhere. Cars can see people/kids in 3d space. Everything I say can be recorded to speech and be searched .
It doesn't take long for AI to realize humans cause a lot of pain to lesser species and eat them as food. They also pollute the planet like crazy and a way to save the planet might be to exterminate some of them.
I wonder what he used for the speech recognition component- quite certain it's an API, but is it internal or 3rd party? Maybe something by Wit.ai which is still unreleased?
That's an interface to it, but all of the voice processing, image processing, logging etc. happens locally and personally sensitive data (like sound recordings etc.) never leaves the house. At least that's how I read it.
Compare with things like Amazon Echo (and I believe Siri and similar services) which send audio clips to a cloud service and do all of the processing there.
There is a world of difference between a home automation system that requires an internet connection (processing on their servers) and one that has a "cloud" function (processed locally with remote access).
The latter is extremely easy to find because that's what's been in use for the last 30 years.
If it's literally Zuckerberg building all this, I bet it's full of security holes (not that he's a bad dev, I wouldn't know, just that one dev can't reasonably be expected to develop a secure app at a pace even remotely quickly).
Dunno what that means, just that it would be mildly funny if someone used this fact to try to get a job at Facebook or something.
Actually, there are some ways to make this very secure that don't take much effort that in are common use around facebook. It is quite easy to build secure thrift services, and to design a thrift api it is just creating a text file with the necessary idl. You can then access the service from most any language by generating the right client. I'm sure same thing goes for gRPC and Stubby on the google side (more familiar with the fbthrift, folly, wangle, proxygen tech stack myself, but I'm sure its just as easy.)
I'm not saying its easy, I'm saying a developer using the tools and processes available at a large internet corporation building something small need not be insecure. Basically until we look at the code and the security processes in place, or do pentesting, we have no clue one way or the other. I'm mostly just saying we shouldn't discount something like this automatically as insecure. It might be terrible, it might be very well thought out and have security be one of the most important considerations that was top of mind when he coded it.
I didn't mean to sound like I'm "discounting" this, though I guess I "discount" everything that hasn't had a cycle of security focused QA, let alone zero QA at all.
I bet, with no information other than what's supplied, that this system is insecure. It's not really a knock on Zuckerberg, just a statement based on my experience. I am totally open to being wrong, I just think we know more than nothing about hobby projects and their propensity to consider security secondary, tertiary, or not at all.
Admittedly, this is Mark Zuckerberg, so who knows, it might have been meticulously coded to be perfect. The guy's a bit of an obsessive, based on what I've read, so I would also not be surprised to find it's tightly locked down and very high quality code. I don't the guy, just the general situation.
I've seen people go down this road before, even specifically toward Zuckerberg. It's never actually a good idea, it just makes you look like a psycho. Big difference between revealing an exploit on Facebook and demonstrating a proof of concept to their bug bounty and exploiting Mark's personal home.
Not saying you would do that, just saying for other people reading that it's not really a joking matter when it really happens =P
I don't recommend doing it, I was just commenting on the concept.
I recall reading someone trying to PM Zuckerberg their resumé by using a bug in Facebook to force him to be friends with her.
Though, I do disagree. I think it's mildly funny when someone does something like this. Not in an encouraging way, I guess, but more as a news item I get to read safely in my pseudo-anonymous home.
I remember when he said he'd be embarking on this challenge, and thinking he'd probably just end up shelving it.
Honestly, even though this stuff has been done before, I'm actually pretty impressed with how much he's been able to learn and achieve in that time. Seems like not all of his hacker skills have wasted away, even if some of it is in PHP :)
I think finding the time to do all this is actually far harder for someone in his position, making it even more impressive. Not particularly ground breaking, sure, but not a bad effort.
At first I read "Building Jenkins" and there I was thinking "Well, finally someone gets how hard that is" (trying it on Linux and not being a Java main)
Neat but this is underwhelming considering who it is with his resources. I've seen dozens of setups like this from random nobody developers over the past 10 years or so.
One-off home brew projects like this are neat but until someone creates and releases a good, extensible foundation to build on top of, meh... And even then it would probably be terrible because they would try to spin it into some business effort (which hackers will avoid b/c they don't want to be glued to someone else's product) instead of an earnest open source effort.
This was never intended to be a Facebook project, it was a personal project for him. And arguably, the amount he accomplished is pretty incredible for the timeframe considering everything else he was doing.
Sure, he has the money to pay people to do it for him, but it was as much a learning experience as anything else.
I've been working on a system for my home as well, and my goal is to implement as much code myself as possible, rather than relying on external libraries, mostly so that I can learn more about how those things work.
>> In some ways, this challenge was easier than I expected. In fact, my running challenge (I also set out to run 365 miles in 2016) took more total time <<
At a casual pace of 10 minutes per mile, that's around 60 hours. Does needing 60 hours to build all that sound plausible to anyone familiar with the task?
Edit: at the end he notes that "I spent about 100 hours building Jarvis", so Mark is a really slow runner :)
The grumpy snarky HN user in me says that a lot of the "I deployed a server to do X" and "I wrote code to do Y" in this blog post should actually say "A Facebook engineer provided me with direct API hooks to do X and Y", but it's anyone's guess.
API hooks to what though? It doesn't seem like he's integrating with much if anything from Facebook. TBH, he seems happy enough getting his hands dirty that I would imagine he purposely did it unaided just for the satisfaction of it.
He did reuse things that exist inhouse, like the face recognition piece. To rebuild a service with Google/Facebook/Microsoft/Amazon level accuracy in that task, it is definitely more effort than 100 hours by itself.
To use an existing private service, not nearly as much effort.
That being said: Of course he could have learned how to train a model himself, but even in that case he is using training infrastructure and data that is available to him that is not public. I highly doubt he had the time to build those sorts of things himself. But his goal was more talking to his home and using tools to build an interactive home assistant. Sounds like he was very successful at doing that. He wasn't out to build the individual AI pieces, doesn't make it not cool or interesting.
In his world "finding the time" to do the running counts as part of the effort. I swim farther than that each year an see no acomplishment in such a distance.
1. Connect his home devices (lights, thermostat, doors, music player, TV, CCTV etc) to his computer, so he can turn them on and off from it and in general automate his house from his PC.
2. Add an NLP component to his computer, so he could text instructions to it and it would apply the correct automation task. This can learn preferences too and be told about mistakes so it does better in the future.
3. Add face recognition, so it can recognise people at his door and automatically open it if they are expected.
4. Add speech recognition, by creating an app for the phone that is constantly on and listening for his voice. This is then converted to automation tasks again.
All in all, an interesting way to spend 100 hours. Calling it AI is a bit of a stretch though; this is basically linking up a bunch of in-accurate sensory parsers, coupled with some limited machine learning, connected to a bunch of if statements. Still a long way off of Jarvis, who is basically a genius by human standards!