Hacker News new | comments | show | ask | jobs | submit login
Why's that company so big? I could do that in a weekend (danluu.com)
712 points by pyb on Oct 3, 2016 | hide | past | web | favorite | 423 comments

I often hear this logic from inexperienced engineers. My answer to them is: Then why don't you do it? <grin>

Recently a junior engineer said this to me about Uber, and I helped him to understand - His logic was "it's just a taxi app".

There were loads of Taxi apps before Uber and loads of them after Uber. Yet none of them were uber?

Sure that a medium complexity CRUD app can do the pickup and scheduling, and spit out some json to the app.

But what if you have 100,000 journeys taking place simultaneously, and you must track their positions so that you can render the journey map on the invoice?

Can you do live traffic planning and re-route around the busy traffic on the fly?

Does your weekend include the full billing system? This is a mandatory requirement for a business.

Can you spot busy periods and increase the fares according to the laws of supply and demand? All online in realtime? Can you continuously A/B test this to optimise profits?

Can you deal with the fact that maps and roads are always changing?

All of these and more are the things that make Uber, Uber, otherwise you're just one of those shitty taxi apps that failed.

Still think you can build it in a weekend?

I agree with everything you say, but that response, like the junior engineer, focusses on the technology. So give people six months not a weekend -- there's a good chance they still won't be Uber.

The success of many companies, and probably all of the unicorns, has nothing to do with technology. The tech is necessary, of course, but so are desks and an accounting department.

Internalizing that has been difficult for me as an engineer.

At a previous job, we had serious development bandwidth issues. The company was trying to solve that by hiring more developers, and all it was doing was making things worse.

The root cause of the development bandwidth problem was that we wasted vast amounts of time and energy on building misfeatures. And we built so many misfeatures because we had way too much development capacity in relation to our product planning capacity. So the BAs didn't have enough time to fully develop ideas before sending them to engineering, and were instead stuck throwing things over the wall when they were still half baked. All these half-baked ideas took time to build, and then they took time to fix after they bombed on the market, and they slowed everything else down since every new feature is something you have to ensure still works when you make a change in that part of the software. Day in, day out, we were taking on technical debt to cover our design debt bills.

So then we tried Agile, and man oh man did that make things worse. 'Cuz now instead of just lobbing things over the wall, the BAs had to be actively involved in every sprint. There goes any opportunity to sit down and concentrate, because now you've got a nattering horde of 20 developers all trying to get clarification on last week's decisions, which leaves less time to make this week's decisions, which means next sprint things will be even more sketchy and ill-defined.

The correct solution would have been to hire more business analysts, and they key thing we needed from those business analysts wasn't instructions on what to build. It was instructions on what _not_ to build.

I know I've seen this pattern in person, but something about the way you wrote it makes me want to go back and think about some of my other projects before I could identify that was the problem.

I know one project in particular we did a release about every 7 months. Every release cycle they estimated 2 weeks for requirements, every cycle took a month, and yet the next cycle you'd see a 2 week estimate on the roadmap.

As for switching to Agile, I know some people would tell you that Kanban would identify this problem for you by around week 8. Did you guys ever try having development liaisons help them draft requirements that were precise from an engineering perspective? Sometimes that initial meeting works better when there are only three people in the room, instead of a bunch of exasperated people being condescending to you again while you're trying to do a good job. adjusts collar nervously I'm uh... asking for a friend.

"...precise from an engineering perspective?", this (in my experience[0]) doesn't work. Because requirements are messy and always change. As they say, " No plan survives contact with the enemy." and "No requirement survives contact with implementation." The reality is that you need to come up with a timeline that accounts for this problem (say 8 weeks) and only scope the release to fill 50% (4 weeks). An 8 week release cycle may even be on the long end of a release cycle.

Why 50% of a release's entire timeline? Because you're going to have overruns, errors, re-dos, and any number of random unplanned elements creep into the scope of your release. And the reality (I may get blasted for this blasphemy) scope creep is fine. As long as it pertains to the originally scoped requirements and is clearly adding value. "A delayed game is eventually good, but a rushed game is forever bad.", Shigeru Miyamoto. The important part of that quote is "delayed", you still need to make sure you release otherwise you're just spinning and wasting company time and money.

And whatever you do, don't try and "fill in weeks". If the release is ahead of schedule, let it be. Let it finish weeks early if it has to. Then, rinse and repeat. Admittedly this type of release process can be hard to sell because there's usually a negative emotional response to the idea of only scoping 50% of the total allowed release cycle and that somehow the other 50% is waste. It isn't, it's there to guarantee that the items scoped initially actually get done on time. That way QA can get to them in a reasonable amount of time, give feedback, and (during the second half of the release) engineering can fix the issues raised by QA. Business development can also rest comfortably knowing that the features scoped into the first half of the release are actually going to come out on time.

I know it sounds like "You're just adding slack." But it's intended to be more than that. If you see teams attempting to squeeze more items into the 50% release you know there's a prioritization problem within those teams. Which can be remedied by either: 1) reducing the overall size of the release cycle further (eg 8 to 4 weeks with 2 weeks as the 50% mark) to provide more flexibility into what goes into the product in a shorter time frame. 2) Helping those teams clarify what the high-level product direction and priorities should be.

Hopefully this 15 minute rambling comment is valuable to you. If not, I apologize.

[0]The mistake I made and the advice I can personally give, don't focus on up front high-accuracy estimates and designs. Come up with a reasonable design, your best guess estimate, and then implement as furiously as possible (coding best practices should still be enforced). You'll know much earlier in the release cycle what the real timeline for your feature will be than if you'd tried to spend all that time up front.

> The company was trying to solve that by hiring more developers, and all it was doing was making things worse.

You don't mess with Brook's law!

I second this. Additionally, get a grip on your queues!

I have yet to find a better text on how to properly manage software projects than, "The Principles of Product Development Flow: Second Generation Lean Product Development"[0].

[0] https://www.amazon.com/Principles-Product-Development-Flow-G...

> The success of many companies, and probably all of the unicorns, has nothing to do with technology.

This is 100% true and unfortunately gets distorted from the financial backers who pump them up for unicorn status. Quite a dichotomy:

"Software is eating the world!"

"The hard thing isn’t setting up an organizational chart. The hard thing is getting people to communicate within the organization that you just designed."

Of course software is eating the world. Wheels ate the world. Fire ate the world. Written language ate the world. Plumbing ate the world. Electricity ate the world. Radio communications ate the world.

Not every project or human endeavor that involved wheels, fire, written language, plumbing, electricity, or radio ate the world, but every project that did not involve them eventually got eaten, for the simple reason that you needed to keep up with technology as a minimum. But beyond that, projects succeed or fail for the same reason they always have: understanding the actual problem you're trying to solve.

Uber's success is not due to software. Uber's success is due to its understanding of the transportation market and how to play the regulatory game to its advantage while never getting permanently in trouble. Uber's ability to deliver on the huge market opportunity they saw—i.e., its lack of failure—is due to software. We quite reasonably call this "success" when talking about technology, but if we're talking about companies succeeding or failing, that's not a discussion about technology.

> The success of many companies, and probably all of the unicorns, has nothing to do with technology.

So why not look at failure then?

Its harder to learn from failures than to learn from successes.

Success has the false-positive issue. Failures have the false-negative issue.

Cargo-culting false-positives is going to be less harmful than cargo-culting false-negatives. (X Company didn't use stack-ranking and then failed. Clearly stack-ranking is bad for companies)

Maybe. Just to muddy things even more, some successes are failures.

Have you ever met a billionaire who said, "I was lucky" and left it at that?

Yeah. At least with a success you have a data point that it could be successful - with a failure, you don't even know that.

It's not the technology success/failure axis in question, but how much weight to give it relative to other factors.

AKA in predicting overall success would a team with low tech expertise, but high sales expertise be have a better chance?

I know - but I was talking to a junior engineer, and I was concentrating on the technical aspects to inspire him.

I expected him to go off and actually attempt building it, that way he would get some more experience.

And he did! He went off that weekend and wrote a proof of concept route planner, it was great to see him learn something new and complex

many a junior engineer who persist on the idea its easy, can be done quicker, cheaper, and such, also rarely have the needed dedication to see it through especially when it takes longer than expected, missing requirements are identified, and such.

I have seen multi million dollar projects hit the same snags, horribly under scoped all because Mr Easy didn't think about asking all of was needed. They saw the tree and missed the forest

Proper project scoping (or to rephrase: identifying the true core business needs) is truly one of the hardest problems in tech. Over-scoping makes you end up with a ponderous system that doesn't serve core business needs very well, while underscoping can lock you into poor initial decisions that cripple your ability to expand into other related functionality.

What's the expression? "Plan your system for 100x growth, but build your system for 10x growth". Something like that except for functionality rather than scalability.

That and supporting the system. Junior engineers tend to think about building the tech, which is honestly the easy part. Keeping it working until the last of your customers are finally ready to upgrade 10 years later is harder than junior engineers tend to think.

Ha!even this sounds like a mistake to me. How about "Make the simplest thing work for one person.". Make sure they get some value, and then iterate and repeat. Once you've clearly solved the problem at hand then start to scale, optimise etc That is of course all after you've presold the solution.

I know it's not always easy to do this, but that really is part of the challenge. There's still too many deathmarch projects lead by people trying to do things 'the right way'. If you're working on a new project for the love of science, please make sure you're releasing something valuable every single week if not every one or two days!

It has a lot to do with the technology. It's not the only or even the most important factor, but it's still one of the most important.

I think a big part of the disconnect being discussed originates in the myth that CRUD-based apps are fundamentally easy. This simply isn't true. The challenges of building a modern, complex, scalable CRUD app with good UX and a maintainable code base are not 'less than' challenges rooted in algorithms or bit bashing. Doing it well enough and fast enough to be competitive in the market is HARD.

Sometimes it is the tech... just not in obvious ways. You have to look beyond the basic spec.

Uber and Lyft are not just taxi apps. They are massively polished. When I use them I am astounded at how good they've gotten the UX and then I shudder at how hard that must have been across two mobile platforms and in the presence of very unreliable networks. I've ordered an Uber with no cell Internet using barely usable coffee shop wifi from a block away.

Then you have scaling, which brings up or creates issues you just do not see at smaller scales. Big O type considerations and weird edge cases that only come up once every billion transactions start mattering.

Then there is all the big data stuff. These services do tons of route optimization and analytics.

So yeah marketing, sales, fund raising, management, etc. also matter but tech often matters as much or more. It's just that junior engineers usually only see the tops of icebergs. This is why despite decades of pure biz people saying "tech doesn't matter" you still rarely see founding teams succeed without at least some tech founders but you do occasionally see even lone tech founders make it. (But those tech founders do have to know or be willing to learn the non tech stuff!)

> The tech is necessary, of course, but so are desks and an accounting department.

> Internalizing that has been difficult for me as an engineer.

After I got over that particular hurdle, I would still think:

Ok, so "build it and they will come" is not a thing, got it. But still, engineering is most important by far. Without sales, support, marketing, or legal, you still have a product, just one that is adopted slower. Without engineering you don't even have a product to sell in the first place.

And that's true. But in the end it's just splitting hairs. It doesn't matter. Over the longer term, no successful, scalable business is ever going to exist solely as an engineering team. Every job function has a purpose[0] and gets you where you want to go faster; in some cases, lack of certain non-engineering job functions will make it impossible to even do certain things.

So sure, you can bootstrap a product with just a few engineers and nothing else, but unless you're ok with it being a niche-market product that you spend all your waking hours working on, an engineering-only org is never going to cut it.

[0] With the obvious exception of those people hired to satisfy the latest management fad du jour, or to give the CEO's deadbeat nephew something to do with his time.

> The success of many companies, and probably all of the unicorns, has nothing to do with technology.

Execs love to think this but it's just not true. Top heavy companies waste money. It's true that tech of not the only ingredient but for a tech company it's critical. Good tech is not a commodity. Top-tier engineers are worth as much as a good CEO. That is if you want to remain lean.

> Internalizing that has been difficult for me as an engineer.

I suspect this is because everyone wants to not only feel as part of something but they want to make an important contribution. I venture to guess accountants at Google do not feel they make a difference to search. The culmination of parts is what makes the company whole and all those parts are required.

Its hard to accept you are not as important as you wish you were. Its a great step in the maturation process of one's professional career though.

> The success of many companies, and probably all of the unicorns, has nothing to do with technology

Have you considered broadening your personal definition of technology past computing and engineering?

From Wikipedia:

> Technology can be the knowledge of techniques, processes, etc. or it can be embedded in machines, computers, devices and factories, which can be operated by individuals without detailed knowledge of the workings of such things.

Technology is the creation of systems. Those systems run as much on the users of that technology, and the members of the organisation supporting it, as they do inside of microprocessors.

> Internalizing that has been difficult for me as an engineer.

Technology exists largely to accelerate the pace of development and of course, it's never a sole driver of any business. There is no reason why you can't create a small-scale uber with call centres and excel spreadsheets but I doubt it would reach the scale and impact of what Uber is today. Granted that, it's hundred times easier to figure out to technology than business, I won't trivialise role of engineering because it's something allows a business to near its global maximum.

I thought like those junior devs for a while, too, and I agree to your sentiment.

The feeling that the CEO I was working with was doing mundane tasks was astonishingly - he even said that he could to his whole job using his smartphone.

But after trying to do the business stuff myself I realized that no one in all of this process is irreplaceable - finding a good software developer may not be easy, but it's also not easy to find a good CEO. This is important to realize that software developers shouldn't see themselves as some type of elite.

The main problem is that junior devs don't realize that 90% of business is based on networking and being friends with people who are in middle management and want to increase their reputation in their big corp. You don't get big clients because you're a genius hacker, most of the time those B2B applications are simple CRUD apps. You won't get that $500k because you simply didn't play golf with the key partners. I'm not even exaggerating, I've seen that the best deals are made in ones "spare time".

Maybe startups are another breed than the software companies I know, but don't drink the cool aid and don't assume that technology is your main differentiator. It's working with people all along.

I think it isn't even about scaling high-performance systems, it even starts with the mindset of those who think that software developers are entitled in some way, usually junior devs and people who never did any real business work.

>The success of many companies, and probably all of the unicorns, has nothing to do with technology.

Most of the unicorns are founded with engineering talent coming straight from Stanford, MIT, Harvard, UCB, Waterloo etc. you name it.

Making a company successful is as much about economics of scale than raw engineering and of course, both are closely related in this day and age. Engineering shapes what and how your products feels like in real-time (= understand very short code-to-production lifecycle).

It might not be politically correct to say but I am not here to make you feel better. Anyone who has actually been in a team at a company that has exponentially grown will tell you the same: your chances to ever survive that "first wave" of users is directly proportional to your ability to hire the best, which of course is a lot easier when they are your current or former classmates.

That's not to say there aren't companies founded by people not going to the schools mentioned above or that engineering talent is only available at these places. It is obviously not the case.

However, the density of talent at top-schools is absurdly high. And this is a game changer for most early stage companies and even more so for soon-to-be-unicorns.

Probably why a lot of professors say that University is often about networking opportunities just as much as it is about learning.

But aside from knowing talented people, it takes visionaries to generate fantastic outcomes. Walt Disney for example stopped drawing cartoons himself early on because he had better artists around him. However, he had the vision that ultimately brought the best out of each artist.

I'm not sure you agree with the parent if the parent is saying "no, it's not possible, because the tech is so hard", and you're saying, "it's not an issue of technology, but the business side".

As an inexperienced engineer I did the same thing. My punishment for my hubris was to have bizdevs who felt the same way about us.

The weird thing is though, I still actively encourage new devs to make this mistake in certain circumstances. For one thing, telling them no isn't going to help, we have no shared context, and the math of their productivity versus mine works against me spending too much time convincing them.

If the problem has a logistical or bureaucratic component, once in a while the stars align and they manage to pull off something the department has been trying to do for years but always runs into a wall. Some people in an organization will assume something is true if they hear it from enough independent parties. I know if three people ask me the same question about a block of our code, I start looking at that code suspiciously. Clearly something is wrong with it. Sending the 5th guy to ask some IT guy or product manager he same question, especially in the way a new person tends to formulate questions, sometimes knocks the cobwebs off and the person decides to change their behavior or volunteer to do something they have resisted doing before.

A week of junior dev time for a 5% chance to clear out a bag of tech debt is a good investment IMO. And if it doesn't work they learn something about why things are the way they are.

I had a senior coworker who used to laugh and say "You're new here aren't you?" and I started giving him grief for doing it. Sometimes the new guy is the only one who can get things changed.

Great insight there.

Additionally, the new guy will be much happier when he's treated the way you suggest. I know I would have appreciated that instead of the bs I got at my first proper job in the beginning.

I always try to point out to the junior folks when a thing is janky and broken for "historical" reasons rather than technical reasons, in the hope that their fresh eyes on the problem will find a better solution. Sometimes we're still just coasting on the best solution we could come up with 2 years ago, and in those cases it's likely that we could do better if we re-think the problem from scratch.

And even that is only a small part of what makes Uber, Uber. If you want to actually compete with Uber, then once you've gotten all those minor details worked out, you're going to have to tackle the real problem which is dealing with 100's of different regulatory frameworks for offering taxi services, being a taxi driver (or a not-taxi driver as the case may be), hiring of private contractors and two dozen or so other entirely non-technical things neither you nor I have considered.

>> dealing with 100's of different regulatory frameworks

Isn't ignoring regulations the most often quoted reason for Uber's success?

Having an effective strategy to skirt regulations probably requires just as much knowledge of the regs as a company who complies with them.

I don't think that's really true in this case. In the few cities I know any details for, the Uber answer was, "we'll just do it until the city sues us". That doesn't really take any special knowledge, just preparedness with the lawyers.

> Having an effective strategy to skirt regulations probably requires just as much knowledge of the regs as a company who complies with them.

Not really. The "effective strategy to skirt regulations" seems to be "when someone makes an issue out of it, we'll burn investor money on PR and lobbying against the regulations, and on paying any penalties that get assessed for violations".

For that, you don't need much advance study of the regulations, just a deep supply of investor funds.

They don't "ignore" regulations. They violate regulations. They do so strategically or they would have been shut down long ago.

For example, if public opinion weren't on their side, perhaps due to, say, Uber rides being as dangerous as they're sometimes claimed to be rather than as safe as statistics show them to be, some of those "little" city lawsuits could have blown up much worse.

Unfairly, IMHO. It certainly helped, but they did so much, and better, than the other apps and they made legit, value-improving changes to the ride-hailing process: actually acting on feedback, dynamically setting prices, coordinating a movement away from tipping, applying modern IT.

Remember, their core product was not violating regulations: it was filling empty spots for legit black limo drivers, and was more expensive than taxis but still competitive because of the quality of service.

Ignoring regulations is easy.

Getting regulations to ignore you is the hard part.

That and the economic recession where lots of people who had vehicles were now trying to make ends meet. Also, they paid their drivers better initially until they build up their driver network. Of course leveraging the media helped and a symphony of other things all pushed them to where they are now.

That's what the problem is. You can invent something in a weekend; the red tape takes longer. I think that still counts as "built in a weekend."

Your comment makes no sense. Invent is a pretty meaningless word in this case. Invent doesn't mean 'make a product capable in working in the real world where it has to follow laws, physical limitations, and deal with stupid people'.

If you invent an app in a few hours, then get fined out of existence your app sucks.

If you invent an app in a few hours, then it crashes under high load, your app sucks.

If you invent an app in a few hours and your code sucks and you can't adapt it later, your app sucks.

> If you invent an app in a few hours, then it crashes under high load, your app sucks.


> If you invent an app in a few hours, then get fined out of existence your app sucks.


>> If you invent an app in a few hours, then get fined out of existence your app sucks.


Sorry, but back here in reality apps must comply to regulations. It's your job as a voter to make sure your regulations comply with reality.

For one, not everyone cares about the rules. For two, rules rarely apply to all people all the time. For three, this is external to the apps and you can hardly judge the app for what arbitrary regulations their location has at that time.

Caring or not caring about the rules is irrelevant if those rules make it impossible for you to operate.

Uber has famously skirted rules and has gotten away with it largely because they've been able to throw VC money at PR, lobbying, and paying fines. Unless you happen to be similarly lucky funding-wise, your first round or two of fines for breaking rules can easily shut you down permanently.

Maybe it's a bit hyperbolic to say that this would mean "your app sucks", but the end result is that you built something that no one can use and you can't make money of of. That sucks.

Apps that stop working because the company folds definitely suck.

Something I find really interesting is that Uber drivers in the US are ordinary people. In the UK, they are straight up taxi drivers. It's the same brand for two totally different implementations of the idea.

You're going to talk about Uber? Watch this video: [1]. He talks about every regret regarding their infrastructure. At first you kind of nod your head along with "yeah, it's not that easy", but as he keeps talking you can't help but wonder why the company has absolutely no internal technical direction whatsoever.

They employ over 2000 engineers (just devs, excluding support staff, etc.). They use micro-services designed to the point where teams can't reuse pre-existing services. They are rewriting the same functionality over and over, and over and over again, as separate teams don't even have a clue what has already been written. Pick a piece of functionality you need. There are straight up 20 different APIs that do the exact same thing, but good luck figuring out which one you should be using - oh wait, why bother when you can just write the 21st version of the same thing. They are at the point where they have no clue what their repositories contain. In the video, he can't even provide an accurate number of deployed micro-services. THEY CANNOT EVEN ASSESS WHAT THEY ARE RUNNING IN PRODUCTION. They're literally drowning, and cannot recover.

The point of the article is wrong. It's not about what can be done in one weekend, it's what can be done in 12-24 months by a competent team. Yes, Uber operates internationally which comes with a very crazy set of challenges. 2000 engineers worth of challenge? Not even remotely close. You should need fewer than 100 people who really know what they're doing, and another 300-400 people who can follow the lead of the first 100. If you need more than that for a single product (ie: Google does not apply as they have 100+ full products), you are doing things wrong.

If you're going to provide the anti-thesis, don't use Uber as your example. They don't even understand how their business is managing to remain afloat from a technical perspective.

[1] https://www.youtube.com/watch?v=kb-m2fasdDY

I'm pretty sure Uber uses Google Maps in for a lot of it's mapping and navigation [1]. I'm not saying Uber adds no value - payment processing isn't easy, but it's not solving any new mapping problems that Google hasn't already solved. Its main innovation has been to ignore existing taxi laws by classifying itself as "ride-sharing".

[1] http://www.zdnet.com/article/uber-to-invest-500m-in-mapping-...

I doubt Uber would have gotten started without leaning on Google for both mapping and directions. This benefited them both as Google incorporated Uber back into it's Maps product.

Uber initially (and still does) rely Google Maps for a lot, but they have an internal team working on their own mapping, and have spent a lot of money acquiring map-related tech.

Now. But they initially bootstrapped it with Google Maps, which was sensible to do for their initial MVP, which is all an "I could do that in a weekend" project is.

They also use a home grown solution for routing: https://eng.uber.com/engineering-an-efficient-route/

Can you provide a source for the claim that Uber violates taxi laws?

As far as I'm aware, Uber doesn't violate taxi laws. Uber started out providing "black car" services which are not classified or regulated as taxis. Never have been, still aren't. Black car services are ones where you schedule a trip in advance, and there's no "hailing on the street". Being able to hail taxis is what makes them taxis and subject to regulation.

A lot of the drivers who first began working for Uber were existing black car drivers, and who operated their own black car business on the side. Many still do. Uber didn't come along and make this existing class of ride service illegal.

The only vehicles subject to taxi regulation are ones that offer rides to people on the street without prior arrangement, as far as I'm aware.

Non-hailed cabs are regulated in many cities as well. E.g. in NYC the TLC regulates black cars as well as yellow cabs.


I was offered to interview at a company, they wanted me to do a coding "pre-screening" which consisted of creating a full-stack, and fully operational registration website from scratch.

Maybe I'm a super slow architect and developer, but I'd estimate that to be a greater than 8 hour time commitment.

I assume that the org's leadership acted in good faith, that someone honestly think's a non-slapdash solution would be a couple hours of work. It boggles my mind.

And yes, I declined their invitation to pre-screen and thus did not complete the interview process.

Depends which framework you pick for the solution. Using Django what you propose is doable within an hour.

With something like cookiecutter-django a registration site takes about 5 minutes.


a nodejs solution was the requirement, but maybe if the org's leadership was comming from a Python/php background they might assume using a scafolding was a given. I don't know anything like django for nodejs though!

Express + Passport can easily be done in less than an hour. It's super simple to setup. I've got my default template for every site I whip up. Now writing that from scratch is a completely different story. I doubt any of their current engineers could just put that together in a day.

isn't express.js kind of like a django for node?

No, Express is much closer to Flask than Django. One of my biggest gripes with Node is that there's really no equivalent to Django/Rails in terms of developer productivity and best practices.

That's what gives Express its appeal. Rather than a walled in environment, I can pick and choose the pieces that I need. Much more flexible if you want to move over to a different library within Express.

I understand that's why people think Express is great, but in practice you end up rebuilding a lot of the same functionality. It can work okay in a microservices architecture, but in the end you have to reinvent a lot of the wheel.

There's also a common misconception that Django is a "walled in" environment. It provides sensible default utilities for everything, but you actually don't need to use them at all. For example, you can bring anything from your own template engine to your own ORM if you want to.

Not sure what wheels I'm reinventing, I just choose to include the libraries that are correct for the application.

If I don't use half the stuff Django is offering me, only to bring in my own. Aren't we back to the same place?

The difference is that the Django ecosystem really focuses on developer productivity and has strong best practices. Look at 5 different Express apps and you'll find 5 different ways of structuring models, controllers, and routes.

Plus the fact that Django has some suggested defaults allows you to be very productive with them. For example, I really haven't found any equivalent to DRF for Express.

I want to make it clear I'm not some Node hater. Probably half my projects are still in Node, but I do know I'm much high productivity in Django.

So really all we're seeking here is a standard for application layout. I really really dislike being forced into some heavy handed framework, not that Django is that, but it's on the topic of discussion.

Ember? Sails?

Ember is not a server-side framework.

Sails is leagues behind Django in terms of everything from tooling to quality.

I thought Ember had isomorphism now...

Server-side rendering ("isomorphism") does not make a framework into a server-side framework. Ember is not designed to be a backend for your application.

Sorry, I'm not sure how that's relevant to my comment? I personally love the diversity of the JS ecosystem, but that doesn't have any bearing on the fact that Ember is simply not a backend framework.


Meteor is not at all equivalent to Django. I've used both, and they're leagues apart. Meteor still can't even use SQL.

If you choose Meteor, it's going to dictate your entire stack from database to frontend framework. That's fine if what you really want is an all-in-one solution but it's definitely not a Django equivalent.

Just to confirm, you meant just register -> store to database + forgot password and validation right? (Greater than 8 hour time commitment do sound about right, but I'm just curious to make sure I read it correctly).

The instructions did not give an upper bound in terms of functional requirements, though the lower bound of "realistic use" was given!

So yes, I would assume that includes things like full email validation round-trip too.

Even just slopping together an in-memory, no-email-sent deployed solution is at least 3 or 4 hours in my book, which seems way excessive for a pre-screening question!

And you had to do that from scratch? No libraries allowed? That does seem rough. With libraries allowed you could probably spin something up with rails + scaffolding + devise though.

maybe libraries are allowed, but I don't know any that can really turn this into a 30 minute project. And even researching libraries is hours of work :) If you have any suggestions for this on nodejs, I'll be happy to add it to my interviewing toolbelt :D

Sorry for the late reply, Just noticed your comment. I don't know a way to do it in node. If you have setup devise a few times recently you could possibly get it done with ROR + the devise gem + scaffolding. This is a pretty terrible interview question though, as testing if someone has practiced a canned solution ( which is really all you could do in that time) doesn't seem that great, and actually taking the time to make your own solution would take much longer and an 8 hour unpaid interview question also seems bad </rambling>.

Cookie cutter Django. 30 mins seems on the high side.

Even if that were allowed as a solution, it is still a terrible prescreen interview task.

Depending upon whether or not you can use pre-existing frameworks/libraries they are either asking you to: 1) do something which takes so much work that it is completely unreasonable as a prescreen (which will filter out good programmers, who won't deal with that shit), or 2) they are asking you to basically edit some templates on top of some off-the-shelf solution (which will not filter out bad programmers).

I don't disagree at all, was just poining out it could be done :)

Coding to even talk to someone is a terrible idea.

What about user editable profiles with the associated valudations on email change ;)

I think you're making a related mistake, which is to overestimate the importance of the tech to the business.

When Uber first released, my guess is they probably couldn't do most of what you list above. But it didn't matter, because they had one compelling feature: they routed around regulation. This allowed them to pay drivers more and charge riders less than traditional taxis, while taking a cut for themselves. This is a clearly profitable position to be in, which means that they have the money to bring in capable business analysts and programmers, and the rest of what you say above will happen naturally.

The taxi apps before Uber didn't recognize that the killer feature was routing around regulation, and the taxi apps after that didn't do it first.

Sure, I can't write Uber in its current state in a weekend, but I could absolutely write Uber's minimum viable product in a few days. The problem is having the right combination of a good idea, the correct understanding of why it's a good idea, and the expertise to execute it (not just technical expertise).

You couldn't build the Uber of today without significant resources, however you could indeed build the version of Uber that made them popular and got them access to those resources in a weekend. Uber started out as a black car app. They got lucky, went viral, and the founders were well connected enough to get funding to build out much of the advanced functionality you're referring to after the app already became popular.

It isn't the advanced functionality and scalability you're talking about here (that was built in reaction to its popularity) that made it popular. It was their MVP.

To me the question isn't "Think you can build it in a weekend?"

You mentioned 5 product "concerns" that Uber must contend with to make a point. No doubt if we brainstormed more we could come up with perhaps 10 or 20 more such "concerns".

Now let's be very generous and assign 10 engineers to each concern.

I still come up with a maximum of 200 engineers for Uber.

But Uber has 2000 engineers.

I don't see any creative ideating that would justify that many engineers _as necessary for the top or bottom line operation of Uber_.

They are there for other reasons but not because Uber needs that many engineers to operate.

> You mentioned 5 product "concerns" that Uber must contend with to make a point. No doubt if we brainstormed more we could come up with perhaps 10 or 20 more such "concerns".

> Now let's be very generous and assign 10 engineers to each concern.

> I still come up with a maximum of 200 engineers for Uber. But Uber has 2000 engineers

There's two ways to interpret this: 1) Uber is overstaffed by an order of magnitude. 2) You are underestimating the complexity of Uber by an order of magnitude.

Uber might be overstaffed, but I suspect the answer is mostly that you are drastically underestimating and don't see the complexities of running a large-scale service in multiple countries, dealing with regulations, worldwide scalability, time and cost estimates, surge pricing, tracking and mapping, payroll, billing, payments, apps, internal IT, etc., etc., etc. There aren't 10-20 "concerns". There are hundreds.

I bet they have at a hundred engineers just working on billing and they're all overworked.

> I don't see any creative ideating that would justify that many engineers

Because most of Uber's engineers aren't there to build "creative" solutions. They're there to enable the business.

I agree with your two interpretations. But to clarify that is my position: I believe Uber is overstaffed by an order of magnitude.

It seems after reading through this thread that I am definitely in the minority on that opinion, but I'm not sure if I understand the position there.

How many engineers would Uber need to have before one would say "okay, that's too many". 3000? 5000?

Uber can afford 2000 engineers. That doesn't mean they are anywhere near good at utilizing them, which seems to be the implicit assumption in many comments on this thread.

Google has in the tens of thousands of engineers, which they can afford to have because of Search. I highly doubt they are anywhere near reasonably utilizing those engineers to top line or bottom line product value.

> How many engineers would Uber need to have before one would say "okay, that's too many". 3000? 5000?

However many it takes before the cost of an engineer is higher than the value added. I have no idea where that line is for Uber. Maybe it's 10K. Maybe it was back when they hit 50.

> Uber can afford 2000 engineers. That doesn't mean they are anywhere near good at utilizing them, which seems to be the implicit assumption in many comments on this thread.

This might be true but it doesn't support your point. If Uber is bad at utilizing engineers, that doesn't mean they should employ fewer. In all likelihood, they'd be about as bad at utilizing their workforce if they cut it in half.

Netflix is another example. A couple of thousand programmers and tens of thousands of VMs to build a top drawer e-commerce site. I built a drawer-below-top e-commerce site with twelve people in a year, and a third of them were muppets.

I interviewed at another well-capitalised streaming video place, and they also seemed overstaffed. They had a permission service that worked out what people could watch, based on purchases, subscriptions, and bundles. Something that's basically one SQL query. They had something like six people working on it full time.

Is it not a problem to people that this many people would be required just for billing? I don't want to grossly oversimplify but at some point people need to start getting judgemental and saying, "no, we have created something too complex. We have to simplify."

You clearly have never worked anywhere close to payments :)

Kidding aside, the regulations around anything that handles money are quite complex, and the liability issues mean that you'll spend a lot of time on getting it right, too.

In theory, payment is super-easy. In practice, any payments solution that is not just a toy eats engineers for breakfast. Stripe is probably one of the leanest ones - and they have 400+ engineers, lean on existing infrastructure, and cover only 25 countries.

>> I still come up with a maximum of 200 engineers for Uber. But Uber has 2000 engineers

> There's two ways to interpret this: 1) Uber is overstaffed by an order of magnitude. 2) You are underestimating the complexity of Uber by an order of magnitude.

The third interpretation is that we have no idea what else Uber is working on besides the operations that are visible to us as consumers. Uber can afford to pay 100s of engineers to build out a few technical concepts/prototypes that will never see the light of day. They basically need to do this, because their unicorn valuation isn't based on chasing the vanishing margins of the taxi industry, its based on owning an entire market that won't exist until they create it.

I suspect its a mix of all three.

Exactly, Uber needs to scale along axes in a way most traditional web companies do not.

The axes have been pointed out are heterogeneous city/state/nation regulations, labor law, naviagation/connectivity issues, etc. And onto each of these diverse cases, there needs to be a way to analyze results consistently and deploy business objectives from a centralized corporate strategy.

As you point out: to justify its unicorn status, Uber needs to prove that it can succeed not just in competing with taxis in the major markets where it already exists, but that it's technology platform can be deployed to the edge cases, and the areas where it hasn't.

So what the extra engineers are doing finding a way to acheive user reliability under the reality of accelerating changes to requirements and sharding of policies for separate operational units that will result from forced-growth into non-optimal markets.

To repeat myself from an earlier comment. How does one even arrive at the number 200 based on the complexity of the problem? Secondly not all 2000 will be working on their core app. There will be solutions required at each level. For example in some countries cab services are provided by larger contractors employing drivers and not individual drivers themselves. So Uber needs to have a billing system which collates data from their individual drivers and then calculates the payout to the company. This requires a separate set of engineers, not working on the core app. And then there are HR systems, Finance Systems, every bit they are not contracting to an Indian Service company has to be handled by an "engineer".

Okay. Then how many engineers would Uber need to have for you to say "that's too many"?

I don't really know. What I do know is people seem to misunderstand engineering team size as a simple sum of people working on the Uber app. It is not only the android/ios developer or the full stack developer who is an engineer in Uber. An IT administrator who helps people, technical or non-technical, with their work laptops also qualifies as an engineering team staff to the outside world.

What makes it your or my place to decide that? They're a private company, and they have plenty of profit motive for efficiency. If they didn't think they were getting value from all of those engineers, then they would fire them and save themselves some money. Or are we going to argue that corporate executives aren't greedy enough?

And my answer to your question is: your reply is pretty far off base. Quite simply: when Uber first launch, Uber was not the Uber you are describing. Telling an engineer they need all these things from the start is completely wrong.

Yes, competing with Uber now, with "this can be done in a weekend", is naive. But that's not the point.

The point is, that there are plenty of problems you can make a lot of money on, with a timeline anywhere from a week to a year. Some will be sustainable, and some will be simple cash grabs. The hardest part is identifying not just what, but when. If you have enough connections in the industry, it can be much easier to predict this than if you are on the outside of things.

I have built plenty of things people would say "I could do it in a weekend". And yes, a weekend is a little short, but many of them took just a week, at least initially. The answer is not "Uber does all these things, you need to too." The answer is, "you need to predict the next Uber, so you don't have to do all the things Uber currently does."

Fully agreed.

However, remember l that Uber (probably) started without any of those complexities. Probably just on a weekend.

So I think the absolutely incorrect "I can build this in a weekend" could be rephrased as "An absolute MVP (Or a proof of concept) like this product could be built in a weekend".

This would make sense, right?

"An absolute MVP (Or a proof of concept) like this product could be built in a weekend".

Maybe, theoretically, in some cases. But even so that's very different from the original "I could build that..." bloviation.

I'm pretty sure uber doesn't do any of that locations or traffic routing stuff you mentioned. When I take uber the driver just has their favorite app if Waze, Google or Apple Maps for the actual routing or directions.

Or the passenger in the case of Uber Pool. I couldn't believe what Uber expected of a driver when a new Pool request came up. They had to hit a button, look at the screen, rethink how they were going to go somewhere. Oh and then worry if the requester will cancel.

That happened on my last trip. I felt bad for the driver so I helped him out finding the best way to pick up a passenger after the first two cancelled.

I think Pool is a good thing but not in rush hour traffic where going slightly off the route adds 10 minutes to a trip minimum.

It has to for the time estimates and finding the 'closest' driver to you.

For me the time estimates are always super incorrect and seem only based on the distance.

I use to live next to a highway with the closets exit being a few miles away and I would constantly get drivers on the highway as my closest driver. I always had to cancel since they were actually ~15 minutes away, not 3-5.

I have found that Uber's estimates get better with time. When Uber was newer in my city, it used to give ridiculous underestimates. They've become far more realistic now.

I think the critic has the benefit of hindsight which may translate to 1000s or hours saved in DEV. E.g. Copying Uber verbatim versus all the experimentation to get there.

Hence it's not uncommon to see clones of things (e.g. Dropbox, slack, etc) developed at fractional cost.

Your junior may be onto something , get shares in his. Ew startup quick !

I think you can ask a simpler question: how do you get traction, drivers and passengers included? You application will be invisible, nobody will see it.

"If you guys were the inventors of Facebook, you'd have invented Facebook." The Social Network.

Even more than that: code is only a tiny part of a tech business, even one that has code as its most critical foundation.

A classic example is banking or money transfers (like paypal). So many people think that financial services are nothing more than moving bits around, which seems like an easy thing to code. Aside from the fact that it's not as easy to code as one might imagine, that part is maybe only 1-5% of the business. Paypal got where they are not because of the quality of their code or their operations but because of their relationships. They established relationships with banks, with regulatory agencies, with governments, and they spent years and years doing these things. They signed contracts, they established trust, they created systems for interfacing with many different banks and governments (not just tech systems, but policies and personnel). The result is that paypal is now one of the easiest ways to move money around the world with minimal friction. Cloning paypal's tech wouldn't enable you to clone their success, for those reasons.

Same thing with Amazon. Amazon's user facing software is only a fraction of their operations, their main competitive advantages are selection, price, fast fulfillment, and customer service.

>Yet none of them were uber?

You underestimate the amount of luck involved in becoming Uber. It's not all luck, but luck is a major factor.

Consider this then: If you could build an app as polished as Uber's, there are hundreds of minicab companies in the UK alone, most of which have apps massively inferior to Uber yet has to compete with them, and that are thus a market for better apps.

There probably is a lot of luck involved in becoming Uber, but if you can make an app that good in a short amount of time, then the opportunity cost in doing that and sending a demo round to a few dozen minicab companies is very, very low.

Yet these companies are still stuck with horrible apps, because in reality it takes a lot of time to develop a good one, even if a prototype is fairly trivial.

After several startup failures, the number one lesson I have learned is that you better have the right team who can execute. Uber has the right team http://www.businessinsider.com/the-power-players-of-uber-201...

I think a related thing people say (that makes me cringe) is that xyz is "just a CRUD app", quite dismissively. You could argue that the vast majority of applications are "just CRUD apps"! Once you get up to a certain scale, even "basic CRUD apps" are complicated.

A lot of the work is convincing people:

1. Convince customers to download the app, give their credit card, and then trust their rides and selves to a driver

2. Convince drivers to clean up their cars, download the driver app, and turn it on, and wait for customers to ask them for rides

The hardest parts of starting and running a business generally have nothing to do with tech. That is even true for most software businesses. This is what most developers who have not started or ran a business don't truly understand.

> Can you spot busy periods and increase the fares according to the laws of supply and demand?

And beyond the technical issues: Can you effectively deal with the negative PR that results when you increase fares during a natural disaster or terrorist attack?

Uber isn't just an app, it's also a company and a brand and a service. It's something people understand, it's something that's popular, it's something that people rely on.

So sure, let's say a really good dev could crank out a software engine that does everything Uber does in a month. Now what?

You seem to be implying that really good devs are just lying around not being used whereas great PR guys and execs are the real value. I don't buy it.

I'm not really sure how you got that from the words I wrote. There's tons of work in addition to the tech which leads to a success story like Uber. That doesn't imply that the tech doesn't matter, just that other stuff does also.

The evidence of PR being effective is right there if you think about it.

The evidence of good tech being important is at your finger tips.

Well, with 2000 engineers I could do it in a weekend :)

Anyway, here's a related link: https://www.quora.com/Why-do-AirBnb-and-Uber-need-so-many-en...

And here another: https://news.ycombinator.com/item?id=12597232

You couldn't.

No way you can delegate work to 2k engineers and tell them what to do for next 2 days. Physically impossible.

Not my problem - I'm their manager. If I have 2000 engineers I just make one of them Lead Dev and then shout at him when he hasn't been able to deliver after I gave him 1999 software engineers. What an idiot.

So Dilbert got promoted then demoted...

Yes, Amdahl's law.

I'd probably just use the 10 most productive engineers, and tell the others to prepare for testing :)

I don't want to sound bad, but again you couldn't do it.

Say you have 2k engineers for next two days. Where do you begin?

You are correct, but are taking these guys too seriously. They know about the mythical man month and are just riffing, for better or worse.

Describe the app generally, and leave it to them to form their own teams and solutions. Hackthon style, and hackathons at least usually have functional prototypes by their end.

Nope. Nine women can't make a baby in one month.

Just use one "10x woman".

Unless you pipeline them.

While I agree 100%, I think the answer is more satisfying when viewed a different perspective.

It's not that Uber needs all those engineers. It's just a question of marginal cost vs marginal return. Uber made 1.5 billion in revenue in 2015. An engineer costs roughly $150k / yr. An additional 100 engineers only have to make enough small improvements to increase revenue by 1% to pay for themselves.

> An engineer costs roughly $150k / yr

This number is far too low. Uber is headquartered in San Francisco.

ALso, they are literally trying to solve a problem similar to the travelling salesman problem when you consider Uber Pool scheduling, Uber Eats, etc.

True, but they don't have to find the optimal solution, or even anything close to it. They just have to find a solution that's slightly better than the next-best solution, which in most cases is very far from optimal.

And then you have to get a million people to say, "I'm just going to Uber there." Except replace Uber with the name of your new app.

Also, it's often the case that it's better to buy a tool and integrate it into the company than build from scratch. It's especially true with a small team. It's not the most romantic idea and sometimes engineers feel that they could build the solution themselves in a weekend.

Also, all the edge cases which become visible only after real users start pounding the service exposing all "this should never happen" conditions.

Data federation is the key to complex data. In a month or two, the first cut anyway, I doubt it could be done in a week-end.

And on top of all of that, can you get 100K people to use your app?!

Oh yeah, infrastructure & marketing ;)

Thanks for the nice specifications!

Uber didn't start with all of that either.

The MVP can be built in a weekend though.

Nope, that is the point.

I think you chose a bad example. Imho Uber has waaay to many engineers for what they are doing: to the point where they need to invent work for themselves with no business value.

I recently had a discussion about how hard it would be to have a system inside a bus to announce the next stop. Sounds like a weekend project right?

Think a bit further:

- Hardware needs to be resistant to harsh diesel engine vibrations

- 3G/4G connectivity

- Need a mobile data contract with local ISP

- Software needs to be able to handle network disconnections

- GPS needs to be able to pin-point at which stop the bus is currently stopped, even with bad GPS coverage in larger cities

- If the bus skips a stop because of a detour, the software should be able to detect it and announce the next stop

- The announcement should be bi-lingual for Airport busses

- The announcement should work for people with hearing aids

- Server needs to know all bus-stops

- Server needs to know different stop types, such as bus terminals, intersection stops, regular stops, hand-over stops, virtual stops

- Server needs to be able both work of a yearly bus schedule and real-time update

- Server needs to output in an understandable JSON/XML because the government subsidies demand an open-data format

- Server needs to publish data to an open-data server because of the subsidies

- Because the bus-company is sponsored by European Union money the control interface should be translated to German/French/English/Spanish

And now this weekend project takes a team of 10 engineers working for a year.

All London buses do this. I don't think you need a server, 3G/4G connectivity or even GPS really. You might be over thinking it.

On the other hand, the overthinking in question (which is probably true for bare-bones next stop indication) provides the base for my personal bus bug: know if/when a bus is reaching a stop from outside the bus in question, especially with geofencing, in order to know whether the bus I want to take is running early/late/not.

This is available for public buses in NYC. The city added GPS to their buses and then the ability to query distance from current stop via SMS.

Then they opened their APIs, and now I have a half dozen apps to choose from that will show the approximate time until the next bus from a particular line will reach any given stop.

(Not always 100% accurate, but useful enough to know when to wait indoors for 5 min if it's raining, or pick another line/choose to walk because of delays.)

They actually do this as well (and many have live updating signs), so the buses do have GPS linked to a server of some kind. It's normally absolutely fine, but on the couple of occasions for me where it broke it seems that the server doesn't conduct the bus in any way and kind of hopes the bus is on the right route.

That's all speculation though.

The TFL system will remove a bus if it doesn't get an update often enough that includes actual progress on the route. This is in fact the opposite problem of what you mention, in that they don't do a very good job of predicting based on past performance. E.g. I used to take a specific bus to pick my son up from nursery that always would "disappear" for a certain number of minutes at a certain part of the journey because a very short stretch of the route was always extremely congested at that time of day.

It's a tricky one, because they have big concerns about over-promising (e.g. the bus could be stuck behind an accident instead, in which case guessing that it's 5 minutes away because of past behaviour will really annoy people) so in some situations they err on the side of caution.

But they are unable to get rid of all of the corner cases. E.g. a bus that moves rapidly for a short stretch, then gets "stuck", will often appear, disappear, appear, disappear, with much less progress in time estimates than elapsed time should indicate.

The thing is, the team working on it has a huge amount of gathered info on how to give these predictions in a way that tries to minimise surprise and avoid making promises that cause problems for people. E.g. often it's better to take a bus off the list so people who are in a hurry will grab a taxi, rather than "promise" the bus will be there in just another few minutes. They still get it wrong too often, but I really don't envy them trying to get the right balance between promising too much and too little.

> They actually do this as well (and many have live updating signs)

I don't know about London, I've seen some systems where the stops had "live-updating signs" but only provided the time of the next official passage. And while I wasn't clear, my bug is not just stop signage but phone access as well (that's most useful when e.g. it's raining cats and dogs and you want to avoid waiting in the rain at uncovered bus stops)

There's a standard by Google for transit timings, and Translink, the public transport authority in my state, outputs these times. They're live.

Apps like 'Transit' also (accurately) show the current GPS location of the bus as it approaches, and it updated every minute (I believe)

In London you can go to the TFL website and see live updating times for when the bus will get to your stop: https://tfl.gov.uk/travel-information/stations-stops-and-pie...

Oslo, Norway has realtime data available for all stops as well :)


>Returns a List of StopVisits (departures) from a Stop. If no time parameter is supplied, departures in realtime will be returned.

OK, that's really cool. Definitely means they have "a server, 3G/4G connectivity [and] GPS" though.

Not necessarily. I think some of the systems use radio links to a central location and transponders at fixed points on the route to track the buses.

There is an API for London: https://api.tfl.gov.uk/

Halifax, NS, was doing that in the mid-'80s without the benefit of GPS or handy mobile apps. (Every bus stop/route had a phone extension, and major stops/terminals had talk boxes.) Granted, it wasn't a huge system, but it was hitting rocks with sticks and hollering over tin cans and string compared to what's available as a starting point today, and it worked. The idea that it should be a hard problem today, given GPS, vastly improved wireless communications and eleventy-seven different ways of figuring out where a user is (if entering a stop code seems like it might be too much work) is just plain silly. Or we've significantly redefined what "hard" means.

Or expectations have increased, both in terms of accuracy and in keeping costs low.

Consider that e.g. in London the old route indicators on the buses used odometers. Which worked for telling you were along the route a bus was. Most of the time. But didn't tell you how long it takes until the bus reaches the next stop other than on average, as it would tell you nothing about traffic etc.

Today London buses don't use just GPS to replace it either, because that's nowhere near good enough. See the "Tracking" section here [1].


When I said "worked", I meant worked, not approximated working or came lose enough not to have caused too many upsets. If expectations have grown to be higher than plus-ten-second accuracy (early wasn't tolerated), then we collectively need some sort of counselling/therapy.

You don't get that kind of accuracy in any system that has to deal with congested roads, or any dense major city. E.g. there's plenty of bus-stops near me where stops on the same route involves going around corners and traffic machines and stopping close enough that GPS would be insufficient to be guaranteed to tell the stops apart if you don't get a good enough fix on additional satellites quickly enough.

Even with all the extra data, and a lot number crunching, in London it's a challenge to get it to withing a few minutes during rush hour.

So, sure, if you're dealing with simple, small systems with little congestion, you can simplify.

>know if/when a bus is reaching a stop from outside the bus in question, especially with geofencing, in order to know whether the bus I want to take is running early/late/not.

We had that, through GPS + GPRS (or SMS?), back in 2001-2 in a small city I was living in Europe. Complete with full (flash based) website map of where each bus is.

I used it to get out of the tech park/institute I worked at and into the bus stop outside at the very last minute (since bus timetables where completely unreliable).

GPS is insufficient to give accurate enough estimates these days. It may or may not work depend on your local routes and stops, but it's too imprecise for a lot of locations and traffic conditions to give decent precision. It's good as a "first approximation" and if you don't have anything better. Most systems today end up integrating data from multiple different sensors to improve on gps.

This is the only thing I want. Displaying a bus timetable can be done just fine with laminated paper, and I'm not convinced that the ability to update (with cancellations, route changes, etc) justifies a tech solution.

Displaying the actual bus status, though, is huge progress.

Many providers have an API for this. I used one in my city to build a very simple display that shows you bus times (and I put it in a toy bus):


Sydney even uses the electronic ticketing system to provide an estimate on the capacity of the bus.

Thats clever. Most public bus systems I know in the US still see frequent cash payments onboard (the elderly, mostly). Is this not the case in Sydney, or is the system cashless?

Over the past few years they've rolled out a new electronic ticketing system which is used exclusively.

On some bus routes (and/or outside of peek times) you can pay a cash fare on the buss, but even that is still a single-use paper RFID card that should be tapped on and off. These sales are in the extreme minority though - the elderly require an electronic card to get their discount IIRC.

Really?! Where can I see this? On an app? I admittedly only use Google Maps when I'm in Sydney.

TripView has it.

I work part time at the company that manages the new style signs for bus stations (EDIT: in London, to be clear), so we've gotten to see a bit of the size of the operation TFL has to deal with gathering and processing the bus information. He might have some details wrong, but he's certainly not gotten the complexity level wrong.

what about the bus stop broadcasting a signal with the id, name etc? the system is activated by the proximity..so you can have smthing like approaching station A, leaving station A.

it is scalable because the bus-stop transmitters are added , moved, removed as needed.

updating the data can also be done remotely

My local bus network use M2M Sim's running on the Vodafone network but its basically a virtual private mobile network running on top of Vodafone's network with no connection to the internet.

These connections are in both the busses and the stops. The reason is so the stops can provide real time updates to people at the stops too. If a bus breaks down its no longer tracked on the real time updates so you are not effing and blinding because a the stop said there is a bus due in 5 mins and no bus comes. And if a bus is removed from service unexpected then the information on why can be feed to people waiting for the bus. People are less likely to be pissed that a bus didn't come when expected if they are given a valid reason.

Personally I was hoping they used some form of radio link I could of sniffed with a SDR so I could put a live updating sign in my house. I just scrape the data from their live feed they use to update the tracker on their mobile app/website instead.

Broadcasting out a signal from the stops could need a transmitter that may need to be licensed as the signal would be to be picked up quite a distance from the stop. Also not all the stops in my area are these "real time updating" stops as they are normally put into covered stops (EDIT: Bus Shelters... The correct term wouldn't come to mind while I was typing this.) and some of our stops are just poles in the ground with a sign on them. The busses tracking their own position wouldn't need these stops upgrading to give this information to people on board the bus as the covered stops can be quite a distance between each other and removes any issues if a stop get damaged and stops broadcasting data (say a car crashes into it).

The busses tracking themselves also have benefits that don't visibly benefit passengers. The services back office can monitor busses and spot problems on the network as they happen. Unexpected heavy traffic on a route may trigger the company to skip or stagger a bus (one of my local routes in advertised as being "every 5-10 mins". The real time traffic condition information could also be sold on to other companies (not sure on the value of this data esp for busses that run once an hour or even longer).

There are systems that solve Bus shelter problem by piggybacking on FM station RDS


I was surprised my local network didn't do something similar as it seemed like a logical choice. But after some research I found nothing about them using RDS to do this.

I would love to be able to talk to a tech at the travel network about this. My own research came to the suspicion that because where I am in the country they would of had to strike up deals with multiple transmission sites to get the coverage needed.

That's more expensive because the cost (installation and servicing) scales with the number of bus stops.

Also, there are things you want to do that this doesn't cover. For example, announcing the next stop when you pull away from the last stop.

The announcing the next stop could be achieved with a dual radio setup, one high powered and low powered. The high powered could be used to detect an upcoming stop and the low powered could be used to detect proximity to the stop.

(Lets ignore the security issues that have been shown with amplifying the low powered radio in cars for a second) It's a method used in keyless entry in cars. A higher powered radio is used for button press on the cars key so you can unlock the car/open the boot/sound the horn to locate the car at a distance but keyless unlock should only work at a much closer distance.

Not saying its the correct way to do it (my local bus network doesn't do this) but stop proximity can be worked into their idea.

German bus drivers have it as well. The routes are lists of stop names and their change is triggered by opening the doors. Can be adjusted by the driver if stops are passed, etc.

Many cities also have an app that displays how many minutes the bus is expected to run late, based on when it passed the previous stop.

Runs on some windows if I remember correctly and routes are probably hardcoded along with the data for the segment display outside above the windscreen.

They use GPS receivers on the bus, which calculates the position then transmits back using GPRS for central reporting https://en.wikipedia.org/wiki/IBus_(London)

Same in Munich. I think those work offline, also the driver can pick the next stop.

I guess it is easier in the long run to have all routes mapped and instead of GPS you use something like doors open / doors close as a trigger to jump to the next entry.

you need it. what if you want to connect to something and make some special announces.

What special announces? Either the bus driver does it or it's one that's used frequently (like 'Please move down inside the bus'), in which case there is a button for it.

The GPS and mobile data are overthinking it. The routes are known so simply trigger on door open/close (with some logic for denounce/etc) and cycle through.

For maximal accuracy simply have an electronic beacon of any sort broadcasting an ID that corresponds to the stop and you've got it solved.

And the entire data set for the route should be able to live offline onboard the bus. Changing bus/numbers routes should include the step for updating route data.

Except that quickly becomes inaccurate if the bus driver decides to re-open the doors for an incoming passenger, or due to traffic conditions, opens the bus doors further from the stop than the beacon reaches. Or skips a stop due to road conditions.

1. Set a minimum time until next stop can be announced 2. Add a button to skip a stop

And then when the driver accidentally hits the button and needs to go back?

Notice how we're now adding more complexity now anyway though...

> Set a minimum time until next stop can be announced

This feels like a dangerous hack that will come back to bite you in any number of unexpected scenarios that probably occur frequently during an average bus driver's day.

Consider a bus stuck in traffic. It is not unusual, at least in London, for a driver to open/close the door multiple times between stops when traffic moves slowly enough, often with longer intervals in between than there usually would be between stops.

RFID like chips on bus stops too, maybe?

While active RFID is possible, other radio tech designed to work as beacons is more likely

Bluetooth beacon (estimote?) at stops would help.

This has all kind of usability issues (now you're expecting the driver to pay attention to and correct mistakes), while providing less accurate data (buss pulls away from stop, gets stuck in traffic; estimates at next stop are based on the bus having pulled away and average times; customers gets angry) and less ability to do traffic management (e.g. spacing out buses by slowing down later buses slightly) without having drivers guessing and reporting data back.

> For maximal accuracy simply have an electronic beacon of any sort broadcasting an ID that corresponds to the stop and you've got it solved.

That's not remotely accurate for the reason above.

> And the entire data set for the route should be able to live offline onboard the bus. Changing bus/numbers routes should include the step for updating route data.

It can. E.g. the old London system used odometers. But that was sufficient only for showing "next stop ..." and for reporting very rough estimates. Today it uses odometers, gps, rate gyros and turn rate sensors to be able to more accurately position the buses along a route, and they still regularly "give up" and take a bus off the schedule when traffic conditions means the uncertainty is too high.

For an robust system I would use multiple data streams including bust status data/beacons/GPS. I've seen busses with offset stations.

3G is necessary for live ETA, if you want to get fancy. All these complications, in my view, are worth it if you want a system that works 24/7 and serves thousands of people daily. The HW and SW dev costs are acceptable, but it does need a decent team, depending on what the starting point is and the ability of the team members.

> For maximal accuracy simply have an electronic beacon of any sort broadcasting an ID that corresponds to the stop and you've got it solved.

Yeah, and now we need to install and manage thousands of beacons in bus stops in pretty hospitable conditions.

Of course, GPS isnt perfect either. It tends to fail in CBD areas.

changing bus/numbers routes manually is not feasible. in complex networks this happens daily a hundred times.

I've worked in transit. This is not only feasible, it is how it is done.

cannot see why you would want to do it manually. it is done automatically in munchen.

I'm confused, are you suggesting there are places where bus routes change hundreds of times a day?

I can't imagine many reasons a route would have to change on short notice (accident blocking a road?), and in those situations the announcements not quite being right would not be the end of the world.

Buses are often assigned to a route and then leave that route unexpectedly in the middle of the day to service another route (accident, other bus broke down, traffic, subway shutdown etc) , often not starting at the beginning of that route.

Then the bus changes its route number - certainly in the UK I see the drivers punching in the route details when they take over a bus.

Oh right, that makes more sense. I guess you could just keep a record of all the different routes in every bus though? And skip to the right spot manually as needed.

One of my "fights" at work is prototypes. A prototype can be build very quickly (cheaply) and can exhibit many problems that may be overseen by the "heavy design process". A prototype is a marvelous way to get feedback from users very early in the projects.

The only downside to prototypes is that then the managers don't understand why the actual project takes so much longer and end up asking to just go with the prototype for production.

Get better managers.

I've never had this problem.

>- Hardware needs to be resistant to harsh diesel engine vibrations

That's a non-issue.

>- 3G/4G connectivity, need a mobile data contract with local ISP

This is a duh!

>- The announcement should be bi-lingual for Airport busses

It can always not be. How would that be worse than NOT having the announcement at all? Besides what would the other language be? French, German? That would still be useless to 90% of foreign visitors...

> Besides what would the other language be? French, German?

I guess the names of the stops are untranslatable anyway :)

Canada. Here it would need to be bilingual by law.

umm I travel in GO and DRT everyday. Neither have bilingual announcement. Same with Mi-way.

Sorry, meant Montréal.

This exists in offline versions.

I think it just counts the number of stops and has some input from the driver

But, adding the wireless connections and the servers and the technical staff to code and maintain, would save the driver the labor of pressing the button. That's what technology is for.

KISS applies here. By keeping the 'driver presses a button' interaction, the complexity is reduced by at least a factor of 5, and the TCO ends up being much less.

Sometimes, a doorbell just needs to be a doorbell.

I am so glad you said this. With the boom of the tech industry, it feels like a lot of stuff is getting over engineered these days. Instead of products that focus on something that does it really well, it does its job mostly alright with half-broken bells and whistles.

Sure but if I add all the above complications then I get automated bus tracking system where everyone on their phone can see where the bus is and when it's arriving, I can track and automatically detect issues with traffic, with the bus, with the driver.

Sure you can do the simple announce but the even easier one is to just have the driver talk over a mic, he needs one anyway. There I one upp'd you on KISS. But there is a reason not to KISS.

You can publish wireless / Internet data on where the bus is, but keep the driver pressing buttons as the source of the data.

How often do you want the driver to press buttons? And how do you mark the intermediate positions in between bus stops? Because on any remotely complicated high congestion route, you will not get reasonable estimates if you only update at bus stops.

That's how it works in our (German) city. It works great.

I think you missed the implied /s

I think the GP was being sarcastic ;)

Except asking a bus driver to keep track of button pushing is distracting and could endanger the lives of the passengers or people outside. The stop needs to be announced 30-60 seconds before they arrive at the stop so then is driver is still actively driving. This is something the drivers union would be all over, it's not as straightforward as you assume.

The stop needs to be announced 30-60 seconds before they arrive at the stop

Why? You can announce the next stop a few seconds after departing from the current stop, and then again at the opening of the doors.

If there's a long gap between the stops, people may forget what's the next stop. Announcing at the opening may be too late or the stop has to be long enough. Even for slow passengers in a crowded bus. Announcing a minute before the stop is a good heads-up that bus is approaching the stop and people should prepare.

Then make the delay for the announcement of the next stop adjustable, configured according to the timetable.

but the driver forgets to press the button and then people are affected. ah, was that my stop, why wasn't it announced?

That's why bus drivers get training. And if they don't perform as expected, it's not that hard to hire a better one.

A skilled engineer weighs the costs and benefits of a system before architecting it.

Is it more economical to pay for constant cellular connectivity across 1,000 buses, support IT staff for when the system fails or needs upgrades, etc. or to have the guy you're already paying $15/hr press a button after each stop?

At least on most UK buses, the driver already needs to tell the computer where on the route the bus is so that it can issue the correct tickets to customers. This has been the case for decades.

It could just use the open the doors button with some hysteresis for opening/closing doors few times in a row in some short time.

Check out APRS[0]. Solves most of the above already by centralizing data-collection. Would just need a beacon on every vehicle. No need for internet connectivity. Receiving stations can be scaled up as required. Logging of course not an issue as can be logged on-board and/or externally.

The on-board display can overlay a map and call the next stop based on proximity. Not a weekend project, I'll admit, but many existing moving parts available off the shelf.

Hams play with this stuff... a lot.

[0] https://en.wikipedia.org/wiki/Automatic_Packet_Reporting_Sys...

I don't think anyone would have assumed that you can build such a system in a weekend.

Half of your points assume that current transportation companies don't have any digital data about bus stops and reroutes.

Anyway I haven't been in a bus that doesn't have such a system for at least for a few years (in western Europe and China).

> Half of your points assume that current transportation companies don't have any digital data about bus stops and reroutes.

The question is what is more work: Using the existing data that is stored in some obscure format for which you first have to write an importer/converter or just recreating the data. Add the fact that the existing data might need some cleanup/additions (e.g. times on timetable are only stored with minute precision and you want second precision for more accuracy).

Alternatively, you could have the driver press a button.

Far more useful is the system which tells you, at the bus stop, what the ETA of the next bus(es) is. And that really does need the whole GPS+data system, along with physically robust signs. But on the other hand, that can still be done by a small company - I know there's one here in Edinburgh but I can neither remember their name nor find them in a search.

You can do quite a lot with a small local company. It's global reach that's expensive, as soon as you start needing any kind of high-touch sales.

You can do this with a RPi and a RFID reader. Every station would have a tag, when bus stops tag is read and your script just announces the station.

You can certainly do it with an RPi - although you might have to keep rebooting it every hour or so. :)

GPS and 3G/4G are trivial add-ons. There are stacks for mobile data.

RFID doesn't have the reliable range, so that won't work as a solution.

It's not really a hard problem for a competent small engineering team with a mix of embedded hardware and software skills. It's not quite a trivial problem, because there are some interesting edge cases, mostly when a bus is diverted. But there is absolutely zero rocket science required.

You could hack something unreliable together in a weekend, but it would take up to six months to get all the bugs out, do the production engineering for a bullet-proof solution, standardise the installation process (if retrofitted), and so on.

Double that if management is poor. :)

Those last parts often get forgotten in software. Hacking something together that kind of works if you don't look at it with a visible frown is hobby coding, not engineering. Engineering is the boring work that happens after that, to build something with a quantifiable many-9s uptime that meets or exceeds a standard spec with comprehensive test cases.

Usually the announcement is not "we have stopped at Foo and Main". The announcement is "Next stop Foo and Main" while you're on the way to it, so you have time to sling your bag back over your shoulder, get your coat back on, and make your way towards the door. Depending on the density of stops along the route this can come anywhere from 1-7 blocks ahead of the stop.

Source: I don't drive,and ride the bus a lot.

You generally want to make the announcement before the bus stops so that people can be ready to get off when the bus stops. Also buses don't always stop at every station on it's route so have to deal with what happens when stop is skipped.

That sounds expensive.

Complicated, but not that bad... just need to optimize for the worst case and the best case will purr along happily.

Some sort of near-field beacon on the stops might be more durable than relying on GPS, but it won't help with missed stops.

This seems like a problem that has been solved many times in the Western world.

Or... microphone and speakers for the driver to announce the stop. If you'd like visual indicator too (accessibility FTW!) plenty of low tech and/or cheap solutions that would meet the requirements.

> Or... microphone and speakers for the driver to announce the stop.

Breaks down quickly if you need bi- or tri-lingual announcements, which is the norm in many popular tourist destinations.

Really? Either it's a location 'Piccadilly Circus' or its a place type 'Airport' - any of those can easily be learnt by the driver. Or writen on a printed sheet that they can easily see. Don't jump to making it more complex than it needs to be.

I see you have never tried it in practice. Train conductors in Austria/Czechia/Germany – which tend to be more trained and less busy than bus drivers – tend to stumble around for half a minute for each unintelligibly badly pronounced sentence. Even the pre-recorded messages in buses are of so bad quality that tourists prefer asking other passengers rather than trying to figure them out.

Laminated card in the cab with phonetic spellings for the other languages.

Aside: I'm impressed with the amount of bike-shedding for this one. Though I'm part of the problem, too :-)

Ha, indeed. Just thought of another - even lower tech - a printed list of all stops, visible to passengers on the bus, then clear signage for each stop. If the bus stops every stop by default, then this is simpler, if not then the passenger needs to be able to see this on the approach, with another time to signal that they want to get off. Can this get even lower tech?

You've just described a cellphone running apps that already exist.

But nobody wants that. They want conductors back. This is not a problem with a technological solution.

> They want conductors back

That's just faster horses though.

Conductors aren't a solution to a problem in themselves, they're a means to solve a problem. So what's the actual problem? WHY do they want conductors back?

Move to China, every bus in Shenzhen has a lady conductor.

> And now this weekend project takes a team of 10 engineers working for a year.

More like 2 guys working for a month.

I'd say ~ 5 Engineers working for ~ 3.5 months

(I calculated the geometric mean of the numbers both of you have given, in case you're wondering ;) )

Generallly my approach if engineers disagree significantly on a timeline is to multiply the worst case estimate by a large factor, on the assumption that odds are good it means neither one has a clue what the actual complexity is if they don't manage to get to rough agreement...

Absolutely impossible. There is hardware involved.

I'd say something like 3 people for 6 months to 1 year (not all 3 being needed 100% of the time).

You are massively overthinking the problem. The very first system of this type I saw didnt even have a microcontroller. Just an eprom for driving LED array, bunch of TTL logic and two inputs. One linked with 'open doors' switch, the other going to undo switch. And it worked fine for couple of years in a >million city.


Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact