> “Frankly, Kai just made decisions, and it just happened
> without a lot of committee meetings,” Ng said. “The
> ability of individuals in the company to make decisions
> like that and move infrastructure quickly is something
> I really appreciate about this company.”
That was, initially, one of the more interesting things about Google. You wanted to start a project and could get resources to run it in the various data centers relatively easily. If a guy like Ng was frustrated inside of Google by 'committee meetings' keeping him from getting things done quickly, then that says something about the current state of affairs there.
Note that Ng did not say he was comparing to Google. A third-party made that claim:
> “He ordered 1,000 GPUs [graphics processing units] and got
> them within 24 hours,” Adam Gibson, co-founder of
> deep-learning startup Skymind, told VentureBeat.
> “At Google, it would have taken him weeks or months to get that.”
According to Wikipedia [1], Ng built a 16,000-CPU-core system at Google. (It's unclear what the CPU<->GPU conversion factor is)
Google was once heavily invested in just saying no to GPUs across the board. They even had a convenient FAQ from their platform team filled with all sorts of out of date FUD demonstrating why GPUs at Google were simply never going to happen.
The hiring of Geoffrey Hinton changed all that with Deep Neural Networks which are easily 50x more efficient on GPUs. And in the meantime, Baidu demonstrated what one could achieve by integrating them top to bottom in their search engine while Google happily stuck its technological fingers in its ears singing la-la-la-la.
Given that Andrew Ng's lab's other achievement was to demonstrate that 3 quad GTX 680 workstations were equivalent to that 16,000 CPU core system at Google, Baidu seems like a really great place for him to go.
I thought that might be the reason, too, so I went to check the number of employees. According to Wikipedia, Baidu has "34,600 (As of March 31, 2014)", and Google "52,069 (Q2 2014)"
So Baidu is 2/3 of Google - it's not just the size.
My opinion is that "getting big" can either be an increase in the number of workers or the degree of maturity.
As a company becomes older, the need to run all kinds of crazy experiments suddenly doesn't matter anymore. They care more about stability and profit. Yes, there is Google X and I know Google still run all sorts of crazy data analysis. But the point is, they want to become more structural because of the number of projects needed to approve. If twenty researchers each request to run a project each takes 1M out of Google, those research better have some formal proposal or so so they can evaluate which gets priority. Also seniority? Two top talents can compete for the same resource but if one team makes more "promising" result than the other team does, the winner is obvious.
Now coming to BD. BD isn't exactly unlike Google. It's exactly China's Google. It has reached its own maturity, but when they want to expand westward and when they want to steal talents from Google, they will have to strike better deals, be it compensation, work freedom and resource support.
I will not be surprised a few years down the road Ng will find similar hurtles within BD once he's all "used up" (sorry for the lack of better word). But of course, what matters is what he can do now.
Ya, but one can hope they'd think in terms of quotas [e.g. You have XXXX of machine time and/or a $$$ budget of XXXXX to test ideas] rather than require committee meetings.
They require the committee meetings to set the quotas.
Quotas suffer from the standard budgeting problem: everybody inflates their needs and hoards resources so that if they need extra in the future, they don't have to go through the budgeting process again. As a result, when a new project comes along that's not just an off-shoot of an existing high-priority project, they find that they're starved out of quota. They don't have an existing quota because the project previously didn't exist, and then when they go to get some, they find that all machines are currently allocated because everybody else overbid to secure room for expansion.
Google was rolling out a fix for this process when I left them, but I'm not sure if I can share the details on it.
I know some bigger companies like HP make sure every department bills every other department in order to make sure that you don't get problems like you would in any centrally planned economy without a price mechanism.
This sounds fascinating. Off the top of my head, the only fix I can imagine is to punish people for overestimates, but that sounds harmful in other ways.
I very much enjoyed Andrew Ng's machine learning class, so much so that I was sad when the class ended. I wish him great success at Baidu.
A little off topic, but: Corporations no longer are loyal to employees (unless they are at the top of their field). Some countries now seem much more interested in the interests of corporations than people. Therefore, I think that it is very smart for individuals to more consider themselves as 'citizens of the world' and be very open to working for whoever pays them the most, provides the best infrastructure, etc.
I don't find this to be universally true. I was at Google for five years. When my first daughter was born, they gave me 2 weeks of paid paternity leave. When they upped the policy to 7 weeks months later, they made it retroactive and gave me another 5 weeks to spend with my new family. When I was dealing with health problems, my managers were extremely understanding and put a number of different options on the table for how they could help me cope. When my wife and I decided we needed to move somewhere without a real Google presence, they bent over backwards to find a way to make it work without my leaving the company.
It's difficult to separate "company X is being nice to employees because it is in their best interests right now" from "company X is showing loyalty to its employees," and in general morally upstanding behavior coincides closely with long-term self-interest. But to my mind the way I was treated constituted loyalty. The clincher to me is that Google has a newer extraordinary suite of benefits for the families of Google employees that die. That is a good, loyal, and _expensive_ thing to do, and I have a hard time seeing how it makes sense as part of some cynical calculation. I don't think a lot of employees are going to be attracted to or retained by Google because of its stellar death benefits.
To be sure, I know this doesn't mean Google would never ever lay devs like me off in a downturn, nor does it mean that I would never quit Google. (In point of fact, I did.) But I definitely don't advocate, as you suggest, that developers simply go to whatever employer is paying the most right now. Go to whatever employer has built a reputation for treating its employees the best.
It has been that way since at least the mid 90's. I am amazed at the number of people who still think they need to show where they work any kind of loyalty.
Corporations are legal entities and don't have qualities like "loyalty", so it doesn't even make sense to anthropomorphize them. The people that work at corporations, including managers and executives, certainly can have those sorts of qualities. Whether or not they do is another story, and that's going to vary widely from one organization to the next. Loyalty is something that's given to and accepted from people, not legal entities. I have absolutely felt degrees of loyalty to some employers and a lack of it to others.
Loyalty from a corporate perspective means things like pensions, health care, not laying off 20k people every fall, etc. Like the days that you could work for GE your entire life and actually retire. There was a mutual loyalty. The company took care of you and you therefore took care of the company. Now someone could offer me a couple thousand dollars extra in pay and I'd be gone in a heart beat. Don't forget that legal entity is made of people who decide how that legal entity will function.
Also if corporations can't show loyalty, a human quality, then how can they be people?
It does not make , but it happens. By middle managers or by endomarketing as away to make people work more for less, classic examples are in company that are going bad
I once had a friend take a low paying dev job just because he needed it. He wouldn't qualify for insurance for 3 months and had recently adopted kids. Then another offer came by that paid more and started insurance day one. The employer had the nerve to tell him that they were upset with his lack of loyalty to the company. If that company had any loyalty to the employee he wouldn't have had to work for 3 months without health insurance.
His success in Baidu and Baidu's success in general means more victims of their advertising business which is simply online fraud (of medicine and medical treatments). And most of the victims are probably less educated, poor people who are new to the internet.
For that I despise him and whoever actively takes part in Baidu's business. And the investors. This is the worst online fraud in the world because it is literally killing people.
[Ng said:] “The ability of individuals in the company to make decisions like that and move infrastructure quickly is something I really appreciate about this company.”
That might sound like a kind deference to Ng’s new employer, but he was alluding to a clear advantage Baidu has over Google.
“He ordered 1,000 GPUs [graphics processing units] and got them within 24 hours,” Adam Gibson, co-founder of deep-learning startup Skymind, told VentureBeat. “At Google, it would have taken him weeks or months to get that.”
Weeks or months? I wouldn't expect Google to get them in 24 hours, but has it really gotten that bureaucratic there?
That doesn't sound like a fundamental difference between Baidu and Google so much as it sounds like he has a lot more authority within Baidu. A lack of red tape is not inherently a good thing- if we read some other story about some other Baidu employee who asked for and received 1000 GPUs within 24 hours for a project that ended up being completely pointless, that story would be about wastefulness and lack of oversight at Baidu, not about how refreshingly free of bureaucracy it is.
I'm not necessarily disagreeing with any of the points here- there are just many different ways of looking at these data points.
Good point. Though, my impression of Andrew Ng's tenure at Google was that he was running a high visibility deep learning project[1] that was attracting some big names, and one would think he'd have had some authority to get hardware provisioned sooner rather than later. Certainly his level of authority is much greater within Baidu now than it was at Google.
Google has a LOT of highly famous AI/ML/DL researchers like Hinton, Kurzweil and Norvig to name some off the top of my head. Ng is certainly very highly regarded but he was one of many there. It seems like at Baidu he's top dog.
Non-standard hardware kind of implies dedicated servers. And multiple dedicated machine pools instead of a single shared one add a lot of ops complexity and are very bad for getting good utilization. If the GPUs aren't standard hardware, I'd be surprised if you could get them deployed in a Google data center in just months.
Imagine 16 different groups all want 1000 GPU's, each decided they want a different type.
Ignoring even the GPU's for a second, somebody has to support programming them (now you get a whole bunch of incompatible GPU toolchains), etc.
Especially for something expected to be long term, it would be stupid to let this happen en masse without at least some different folks sitting down and talking about it, which is red tape.
It depends, 1000 GPUs is a lot of money. Even if each GPU was $500 each, that's still $500,000
Granted, in Google terms that's pocket change, but still, I'd imagine there is a rigorous requirement for a solid business/project plan before management sign the dotted line.
Depends on the size of the company for them $500,000 is not a huge amount these days.
Back in the day I looked at using a neural network for an experiment to measure the efficiency of toilets that would have been £250,000 in 1982 just for the hardware - that was a bit to much.
Though back then for a single piece of HP test gear we spent around a 100 grand.
Do you guys know that reportedly 30+% or even a half of Baidu's revenue comes from false advertisements by private hospitals and medical institutes mostly owned by companies from PuTian, a city in south China? It is literally bloody dirty money. If you can read Chinese, here is some news report: http://finance.china.com.cn/industry/medicine/yygc/20140721/...
Baidu as a search provider did lots of immoral things, that's why these days it's so difficult to use Google because of GFW/censoship, and why Baidu is so dominant in China.
Can we just state the obvious? Chinese AI peeps left because they want to go back to China. This isn't an unusual trend, as an Chinese American, all of my cousins who came to study here abroad in US have plans to go back to China, unlike my parents' generation who came here for a PhD and stayed.
And even amongst my parents' generation, my uncles who have gotten PhD and tenure at US universities and left for universities in China. The reason being that they can get paid for just as much as in US, for lower living cost, a more compatible culture and in Beijing or Shanghai for better standards of living (yes before the critics jump in with air pollution, human rights issue etc., a good US wage in China could afford you a full-time personal assistant and/or nanny to do your laundry, take care of your kids, clean and cook good Chinese food and you can live in a nicer part of the city with better infrastructure than most of US lol; and you could still come out with lower cost of living).
Isn't Andrew Ng working in Silicon Valley? It is difficult to recruit in Beijing these days given the air quality, cost of living isn't low but not that bad either.
I assume you're referring to Andrew Ng himself here, but you don't seem to know him well enough to infer that he wants to live in China -- even though this position does not entail doing that anyway!
He was born in the UK, educated in Singapore, then at CMU, and then at MIT and Berkeley. There's more, but the point is: you're projecting.
AFAIK, most people who choose to work in China are finance professionals, not programmers. But I wouldn't be surprised even if they are programmers. Many companies in China are expanding westward (.e.g Biadu). Bilingual is a big bonus because you can communicate with your co-workers from both East and West. Some people like me can read, write, listen and speak perfect Chinese (Mandarin and Cantonese) and is perfect candidate to work in China.
If we put health issue aside, China's living standard isn't exactly that low, housing price is crazy... and you are a foreginer, you won't go to public school. You will go to international schools and they aren't cheap. But if they earn the same wage as they do back in US, like at least 100k a year? sure, maybe.
Otherwise, I think one of the tedious part of living in China is applying licenses and IDs. According to a research [1], a Chinese citizen may need to apply between 74 to 103 different kinds of ID, license and certificate in one life time.
