Hacker News new | past | comments | ask | show | jobs | submit login
Siri, What Time Is It in London? (daringfireball.net)
726 points by jmsflknr 3 days ago | hide | past | web | favorite | 746 comments

At some point about a year ago I noticed that I could no longer ask my Google Home devices "what's the weather?". I'd just get a generic "I don't understand" response. But more specific queries such as "What's the weather in Seattle?" would work.

After a couple of weeks of this, I somehow got the idea that it was related to the devices' configured locations. And sure enough, telling the Home that I lived in the next city over fixed the problem.

So I started a binary search and eventually found that the issue was limited to my ~10x20 block Seattle neighborhood - basically the outline shown when I search for its name in Google Maps. I then also realized that it applied to weather queries on my phone as well, but since the phone uses GPS rather than a specific location setting, I could only reproduce the broken and working behaviors by crossing one of the neighborhood boundary streets.

Turns out it was some long-standing configuration issue with Knowledge Graph's entry for my neighborhood, and some recent code change in location-based weather queries began butting heads with it. Luckily I worked at Google at the time and was able to track down and pester people that could help fix the issue.

But if you had not worked at Google at the time, there’s basically zero chance you could’ve got anyone at the company to do anything about it.

Reminds me of when I worked at Yahoo, for not very long in the early 2000s. Yahoo had a consumer product called My Yahoo, a customisable home page with various feeds in it, and at the time they also had a My Yahoo Enterprise Edition, which they tried to sell to companies to serve as an intranet home page. I worked (in a junior sort of way) on the European deployments of MYEE.

Anyway, one of our customers - representing a company in Germany I think - filed a bug report that said something like "Weather module hasn't updated since January". They'd been going to their fancy intranet home page and seeing the same weather for months at a time.

And this bug report just sat there. For a mixture of technical and political reasons, there seemed to be nobody in the European office able to pick up this report and do anything meaningful with it. We knew about it, we knew that what we were serving to paying customers was hopeless, but we somehow couldn't get our hooks into the right point in the Weather feed to figure out where it was going wrong. Or, collectively, we didn't care enough.

You could replace Yahoo with any other company and this would be equally true.

If this had happened at some of the other places I have worked, the CEO would have made all the developers stay in the office until the problem was fixed. Why did no one care?

To say no one cared is going too far, but it is interesting to think about why something like this was able to happen.

There were various technical and structural factors making it difficult to fix. Weather feeds were known to be problematic (still are I guess) and this code would have been surprisingly low-level C/C++ with custom serialisations and limited logging. Structurally there were problems in getting attention from a team in California to support a problem experienced by a different team in London, especially since it affected relatively few users - a tension in supporting paid products in a company that is focused on non-paying users at far greater scale.

(I am assuming this bug would have needed some actual development work - I don't recall, but I think we were familiar enough with common ops problems that it wasn't just a question of kicking one of our own feed servers.)

But I do think there was an issue about lack of concern - at heart we didn't have enough confidence in our own product to motivate the personal pain of working through these problems and getting them solved. I think that, if you had gathered us together and asked our collective opinion, we would have suggested that this customer would be better off not using our product at all - it simply wasn't ever likely to be good enough. Once you reach that way of thinking about your own product, it becomes extremely hard to countenance fixing the most difficult problems with it.

That sounds like pretty bad places to work. And very bad internal processes.

Don't have to name them, but what kind of places are these that care? Big, small, tech, non-tech? Locations?

Well, for me, it was finance where you know each and everyone of your clients by name because each of them is paying millions of dollars a year for your product. The company had around 50 employees and a 200m valuation.

My address has been blatantly broken in Google Maps for years now. When you enter it, for some reason it deletes the house number and just looks up the street, which ends up pointing to somewhere about a 10 minute drive away. For example, if my address was "123 Elm Street" it just ignores the "123" and searches for "Elm Street" instead. We have to give special instructions all the time to delivery drivers and other people we give our address to, to warn them and make sure they don't end up going to the wrong place if they use Maps.

I've sent feedback and error reports about this repeatedly, and even had a friend that knows someone that works on Maps pass it on to them directly. It's never been fixed, and I've basically just given up on it at this point. It's really shown me how impossible it is to get any kind of support from Google for even an extremely obvious, straightforward issue.

My address has been blatantly broken in Google Maps for years now

I recently moved, and found out that virtually every single web site from my credit cards to my bank to the library uses Google to verify address entry on the fly. The problem is that Google's database entry for my address is wrong. So any time I try to enter the address "123 Oak Street, Apartment Q" Google unhelpfully corrects it on the fly to "Oak Street, Suite 1." No amount of keyboard jockeying can override Google's on-the-fly autocorrection.

The solution I eventually came up with was to turn off javascript, then enter my correct address, then turn javascript back on to finish the rest of the form.

Of course, there's no way to contact Google about its error. Maybe in Google Maps? I dunno. How do you find an address that Google Maps doesn't know to tell it that the address it has is wrong?

I had issues with google maps being broken for the address when I moved in. I reported it too many times to count using the website.

In the end what worked for me was registering as google maps client/customer, reproducing the issue via API, and then reporting it as an API issue. The underlaying data was fixed within a day or two, and I got my emails answered by google engineer within (literally) minutes.

Ironically your best bet to get it fixed is almost certainly to post your real address here as part of this thread.

Or post another address with the same problem, assuming it affects the entire street.

It doesn't. Even the address for the other half of the duplex (my address + 2) works fine. Sometimes we just provide that address instead and catch the delivery people when they arrive, because it's easier than worrying about the Maps issue.

I had an issue with my own address on google maps (not as severe as this, it just had the wrong zipcode). After trying to correct it using the tools built in to maps 3 times and having my correction rejected every time, I tweeted my complaint @googlemaps, did a quick DM back and forth and it was fixed shortly thereafter.

(please do not take this as an endorsement of google maps support, merely an anecdote of what did work for me that I hope might help you)

As a former Uber/Lyft driver, I probably ran into some kind of map error at least once every 8 hours of driving. These databases have a lot more garbage in them than I realized when I wasn't driving as much.

Same here. Delivery people who have never been to my house consistently go to my neighbor's house a mile away because that's where Google sends them. What's so frustrating about this is that both houses have very clear address markers on the street. But people now trust Google more than they trust actual address signs IRL.

Curious if the address is correct in Apple Maps?

With Apple, I've submitted perhaps 5 corrections for 5 different (usually minor) problems in 5 years. Problems like a place claims to take Apple Pay when it doesn't. Or the actual place is across the street from where Maps claims it is.

In each case, Apple sends back a notification within 2-3 weeks saying they've fixed the problem, and when I've checked, it has always been resolved. Pretty happy with the service.

I would suggest navigating to your home in maps and end up on the wrong address. When it asks for your feedback (nowadays it usually asks for it based on where you live), give it the lowest rating (the frowning smiley). In some regions, it will take you to another page asking you to write in detail what happened. Mention your story in there, hopefully, someone looking at bug reports will read through it.

Well, this can be a feature rather than a bug if you like privacy. I knew a house that was on a large, mostly wooded piece of land where Google Maps would direct you to the wrong side of the property via a dead-end street in a development. If you went there, nothing but trees for a thousand feet or maybe two. Probably at least a 10 minute drive if you actually knew the street that the driveway connected to.

A couple years ago, I sent a report to google maps about a small museum in a remote corner of Norway. The map had the location of this museum about a hundred meters off. Not long thereafter, I received a thank you note saying that indeed, it was wrong, but now they fixed it. And indeed they had. I suppose I should have been surprised at this success. Or maybe they were more responsive before.

Huh, I've forgotten that I have the same problem with my address. If you look up my address, Maps will point to my house (let's label it X St. 36), but the street segment in front of my house has the name of the parallel street (Y St.) a block south of it. On Google Maps, X St. incorrectly changes to Y St. 100 or so meters before reaching my house...

As annoying as that sounds for a home address, imagine it being a business address. It would cost that business a lot of money.

Yes. In this instance you're probably correct.

Write a blog and post it to HN, hopefully you'll gain traction

What if you asked Google "what's the weather?" and it gave you a dictionary response of "weather" as an answer? Or with more snark, responded with, "I don't know. Look outside".

"What is the weather report for today/this week?" is a more accurate question, despite an annoying amount of verbosity. But answers are still given relatively. "Cloudy" could be an accurate answer, for now. But it will be "Sunny" this afternoon.

Some people will prefer a one word answer to "What's the weather?". Others, will want an hourly breakdown of the day displayed on their screen. Others, might prefer a week. It's hard to give an ideal response for every situation.

Ironically, if I say "Hey Siri, weather" I usually get what I need.

> What if you asked Google "what's the weather?" and ... with more snark, responded with, "I don't know. Look outside".

Now that you mention it, I find it strange that voice assistants don't give natural responses when they encounter an error. It would make them seem more real, and it would be less frustrating.

When you ask a human what the weather is and they tell you to look outside, you don't try to rephrase the question in a way that will make them give you the right answer, you just realize this is not the way to get an answer and you look for another way.

Maybe this isn't the best way for it to work in this case, since there is a different thing you can ask to get the response you want, but maybe this would make interactions better if, say, the phone can't detect your location. "Where did you want the weather for? I think I'm lost."

Of course, the other problem is that if it gives the same response every time it'll get grating. The hundredth time you hear "Look outside," it has probably lost its charm. I wonder how possible it would be to generate responses that take into account all previous conversations, so that this doesn't happen.

Put another way, joking or snarky responses from a human aren't just jokes, they're a form of social communication. If voice assistants used those forms without intending to communicate the same thing, that would just be frustrating in a different way.

Or with more snark, responded with, "I don't know. Look outside".

One of my meteorologist friends always answers "What's the weather?" with, "The state of the atmosphere."

“What's the time?” “What's up?”

The possibilities are endless.

I've actually had this with Siri. I've asked about the wind and Siri gave me the definition of wind.

Other times it will tell me the wind speed.

Other times, for the same query, it will tell me the wind speed AND direction, which is what I want.

But it's always random what I get: wind speed, wind direction, a combination of both, or (occasionally) the definition of wind.

they spend a lot of time with these requests i think. “sing me a song”, “are you insane” etc

For your neighborhood or was this a more structural problem and did your collegeas find more locations with similar issues?

IIRC it had something to do with the parent hierarchy of my neighborhood. It was parented by both a postal code and a city, or something. I'm probably making that up, but I'm sure you can imagine how difficult it is to create and maintain abstractions for every possible geographic situation.

I wasn't aware of any other occurrences.

Not only a zip code can span cities, a single city/zip can span multiple tax authorities e.g. parts of Redmond, WA zip code 98052 are under Regional Transit Authority and others are not.

I think that pretty much any assumption we're making about strict hierarchy are bound to be broken at some point.

I worked with zip codes on a project. Zip codes are a nightmare. They cross everything. There are zip codes which cross state-lines. Zip codes which are not a single contiguous area of land. So many issues.

I feel your pain. It turns out there are even a group of ZIP codes that apply only to specific businesses, buildings, or even people (if you're important enough).

That's because zip codes aren't defined by an area, but by postal routes.

Zip codes don’t denote geographic regions; they denote nodes in the postal routing network.

That’s a fun experience to debug a problem, I bet if that problem goes thru customer service, it probably will not unfold and let alone a fix.

Siri doesn't know that my front door is called "FRONT DOOR".

I only have one smart lock, which works perfectly, and it is called "FRONT DOOR" in HomeKit.

When I ask Siri about my FRONT DOOR she responds that she cannot find it.

When I ask Siri about the status of my DOOR, she responds with "The FRONT DOOR is locked/unlocked".

I'll then say 'Alright Siri you literally just used the phrase "FRONT DOOR" five seconds ago and the text transcript on the screen says "FRONT DOOR" hey Siri is my FRONT DOOR locked'

Siri: WTF are you talking about? You don't have a FRONT DOOR.

"Hey Siri is my door locked"

Siri: Your FRONT DOOR is locked.

Google and Alexa handle things flawlessly.

About once a week Siri and I have this conversation:

Me: "Siri, turn off the bedroom lights."

Siri: "OK. Your 6am alarm is off."

For the most part Siri works for me, with the exception of the above and her insistence on adding "ginger ale" to my grocery list as two items.

/Native English speaker, specifically trained in non-regional diction because I used to work on-air in radio.

Me: "120 degrees Fahrenheit in Celsius"

Siri: "Contacting emergency services in five seconds"

To be fair I was in a noisy environment and Siri only got the "120" part but seriously, why would that be okay? My phone is registered in America with an American phone number and English set as it's only language. Why should it think 120 is equivalent to 911?

Looks like 120 is the emergency number for ambulances in China, emergency at sea in Norway, police in Guatemala, and national police in Bolivia [1]. Alternatively, maybe it interpreted "120 degrees Fahrenheit" as a body temperature reading for an extremely high fever?

[1] https://en.wikipedia.org/wiki/List_of_emergency_telephone_nu...

I was having problems with a phone service where I needed to "press 1 to continue," but it wasn't registering. Eventually I ended up pressing 112, and my iPhone displayed "Calling emergency services," even though it's an American phone in America.

I know some countries use 112, but that's too many edge cases colliding.

In fairness here, 112 is an international standard (I think it's in the GSM spec?) and is expected to work in the US.

I guess that makes sense. But it doesn't explain why the iPhone intercepted those digits when I was punching into a phone menu system.

>Native English speaker, specifically trained in non-regional diction because I used to work on-air in radio.

Well there's your problem (/s): https://youtube.com/watch?v=Avp9aUkM5g0

It could be worse. It could have said "OK".

In the past week:

me: "Hey Google, add half and half to the shopping list."

gh: "I've added those two things."


me: "Set an alarm for 2:30 tomorrow"

gh: <generic alarm set response>


wife at 2:30AM: "hey... HEY... why's the alarm in the kitchen downstairs going off?"

well you specifically said 2:30 tomorrow.. it sounds like it triggered at 2:30 next day

And never-mind the literal interpenetration of the command - it's far more usual to set up an alarm very early in the morning (got to catch a plane, unusual event) rather in the afternoon.

Very true. I hadn't specified AM or PM as to when I wanted to start the dinner roast. :-/

It's nice to see that using siri is a normal thing to do :) People at work make fun of me for using it.

That conversation sounds like Siri thinks you're about to go to sleep and wants to make sure you remember to enable your wake-up alarm.

No, that’s what Siri says when it disables your alarm lol.

She wants you to get more sleep so she's disabling your alarm.

Not in my experience. This is how 80% of my Google Assistant conversations go:

Me: "Hey Google, play Nine Inch Nails, you know, the one in my Google Play library"

Google: "OK, playing Nine Inch Snails, a band nobody on Earth has heard of and is definitely not in your library!"

Me: (Repeat a few times, trying all kinds of accents, eventually I get tired of songs about nine inch somethings, and I pull the car over and type in by hand what I'm looking for.)

This seems, to me, to be something else. Google Music consistently avoids playing the most popular version of a song. It's always a cover by no one, or a different song by no one with the same title. I've started to think it's a royalty thing.

Ah but Nine Inch Snails is a classic https://www.youtube.com/watch?v=S0qQ1pomYzk

And what bugs me the most about the Google Play Music Library is that it won't sync up with the YouTube Music Library... They have two different music services which I can use with one subscription but I can't have some of the songs on both of them or have one single library.

You're going to have to, as they're going to merge


Thanks for calling this out! I guess I really should check regularly for every Google service I use if it's being discontinued yet.

There's that Google Graveyard site, maybe they should make an aaS out of it, e.g. you can do an API call with services you use, and it will respond letting you know if any of them is about to be euthanized by Google...

Weekend project idea right there!

All these speech assistants are almost stateless, aren't they? I mean, they can ask you questions and go into a listen-for-an-answer state, but, at least last time I tried, you can't have a conversation about your conversation with them and improve their comprehension of anything. It's like talking to a command line, or those old Palm Pilots you had to learn a special alphabet to scribble on.

It's quite interesting to hear very small children talk to voice assistants. Probably not surprising, but it seems like in normally-learnt human communication you expect your conversational partner to remember the context of what you were saying, and carefully forming canned commands is a separate learned skill. It suggests these voice assistants have still got a way to go, and it seems more like a paradigm change, a big leap, than just incremental improvements.

When i made a meeting room reservation chatbot the hardest part was managing a conversational context without the user getting stuck.

In the end I had a “draft booking” and a conversation loop, where the bot would repeatedly ask to fill in missing parts (eg nr of participants) and then give you a summary and opportunity to correct things. It was hard to do, and definitely required a lot of contextual understanding of how people book meeting rooms. That approach doesn’t scale up well.

I think the basic problem is being stuck in a local optimum. The scripted bot approach doesn’t scale to complex conversations, and you need to start from scratch to do better.

Ahh the good old "conversation is a state machine" pitfall. Even linguists I work with do that sometimes, I guess it's how the simple models that we're taught with work.

Wanna have the simplest parser? Finite State Automaton to the rescue! So people automatically assume that a the simplest approach yo conversation is also something like a finite state machine.

Here's the thing. The only reasonable FSA would be a clique.

You can always move between nodes.

A much more feasible approach is the "actions competing for repelevance" one. Where you have global state manipulated by actions, and all the actions generate a "appiccability score" for the given user input. The system then chooses the most appropriate action, and it does it's thing. And on the next user input the cycle repeats.

Honestly, I can forgive the lack of context awareness. That's a hard problem. I have issues even getting consistent responses to the same query over time (even back to back in some cases). Sometimes, Siri will misunderstand me and fail to do the thing, but then I look at the text that's transcribed...and it's correct (I.e. the backend was replying to a different transcription than I saw on the frontend).

I've just been trained to not bother. Unless I'm setting a timer, I just don't try anymore.

> you can't have a conversation about your conversation with them and improve their comprehension of anything. It's like talking to a command line, or those old Palm Pilots you had to learn a special alphabet to scribble on.

Which is why I have no confidence calling it AI if its not even intelligent. Its just voice recognition on preprogrammed operations.

>Which is why I have no confidence calling it AI...

That's because it really should be called Simulated Intelligence, and would be a much more accurate description. The marketing team wouldn't like this though.

Are they even simulating intelligence. To the parent poster's point it's really just simulated voice recognition. No intelligence even gets simulated. This is more like running cucumber scripts based on voice recognition.

It's an artificial idiot, plain and simple.

your average idiot is intelligent. your average digita assistant can barely assist and has no fingers.

yeah, that's what I'm wondering about - is it going to be hard to implement things like this until LSTMs with better "memory" get good enough for consumer use?

You can‘t even have the „thank you“ „you‘re welcome“ part of the dialogue.

I don't know if it works there, but siri couldn't understand one of my contacts names

When it read back super incorrectly from an alias, I said something to the effect of "could you pronounce that correctly?" and it asked me to say it

Since then, it's understood that person's name. ¯\_(ツ)_/¯

It really needs to expose the option to train those easily

Indeed, you can say "That's not how you say that" to correct its pronunciation of any name.

There is a "Pronunciation" field in the contact where you can spell out how a name is pronounced.

This article clears everything up, https://discussions.apple.com/thread/8116586

Similarly, I have two smart outlets and one smart lightswitch in a room. One is called "Drew's LEDs", and the other is called "Drew's Heater". When I ask google to "Turn off the lights!" it turns off the switch and the "Drew's LEDs" smart outlet, but not the "Heater" smart outlet!

I definitely appreciate the effort they put into understanding the semantic nature of a device from the name I assigned it. Nowhere did I ever designate one of the smart outlets to "behave like a light."

Scary! I would never have anything involving heat (oven, space heater, etc) or water connected to the IoT. Way too much risk of fire or flood from a bug or a hack.

The people who cut my grass are in my phonebook, under, let's say "Lawn Cutting Corp".

If I say "Siri, call lawn cutting corp", she'll say "I'm sorry, I can only call a single person at a time."

If I say "Siri, call lawn corp", then it immediately opens the phone and says "Dialing lawn cutting corp."

What's most likely happening here is that any door can be classified as "front door", and Siri doesn't know whether you want to open the door which has been marked as "front door" (you have none?) or the one named "FRONT DOOR".

File a bug and it will probably be fixed.

My computer is a laptop, which is on my desktop, and there is a desktop on that laptop, and on the desktop is "My Computer".

I think I just made a palindrome of homonyms.

is it named in capital letters? I've noticed some systems will see capital letters and read it as F-R-O-N-T etc. Maybe try renaming the door to lower case

At this point better stop using Siri and just check by yourself no :D ?

This example illustrates how difficult AGI is and how far we are from it. We, humans, tend to take advantage of the context to make communication simpler and shorter. Just think about all the implications of this one simple question: what time is it in London? Or e.g. how can I get from London to Dublin?

If the person asking the question lives in Ohio, they may actually be talking about London OH (or Dublin OH). Some people in neighbouring states may mean the same, though they will be more likely to mention the state. However, how close should you be to London, OH even within the state to mean the Ohio one and not the UK one? How close is close enough? Is a few hours of driving close enough? A 3 hour flight? What if I'm roughly at 6 hours from London OH and 7 hours from London UK?

Further, if the person is a British expat in Ohio, especially if they are working for a multinational business (or not), they would more likely mean London UK. German expats, though? Russians? Or an Irish person who lives in Amsterdam having some relatives in Ohio US, looking to book a flight to Dublin. Etc. etc.

There are so many contextual layers here that even human assistants can occasionally get it wrong, and without the context the task becomes insurmountable for the "AI" algorithms. That is not to say virtual assistants are useless, just that selling them as "AI" is a big lie, bigger than even those who market these algorithms as "AI" think it is.

> If the person asking the question lives in Ohio, they may actually be talking about London OH (or Dublin OH).

I would seriously doubt this assumption. Why on earth should someone living in a state specifically ask for the local time in a different location within that same state?

On the contrary, this context information would make it much more likely that the person actually meant "London, England". Except if there is a timezone border going through the state, of course.

However, I obviously agree with your general point regarding the severe limitations of what we currently call "AI" and how little "intelligence" there actually is.

> Why on earth should someone living in a state specifically ask for the local time in a different location within that same state?

True, and that's another contextual layer to deal with: that e.g. the state of Ohio is in a single timezone and that - why on Earth should someone ask the time within the same timezone? - like you said. And then there may be contextual exceptions even from this rule...

you might not know where the timezone boundary is - they move, or you're new to the area. alternatively, there could be a daylight savings boundary in between, so it is only in the same timezone for half the year.

the fact that americans are inclined to say the state name as part of the name of a place could also help - since they might say london ohio, london might be more likely to mean the real london.

Flipping this around a bit: if someone in England asked "what's the time in London?" should Siri assume that they're talking about London in Ohio or Ontario? Everyone in England is on GMT, so they don't need to do timezone conversions to London time.

There's an extremely long-standing convention of referring to timezones by the major or capital city within that timezone, so Siri should of course assume that they're talking about London in England. In fact, this is probably more or less the correct, canonical way to ask this - asking about GMT would not give you the correct answer, since England is currently on BST which is GMT+1. Needless to say, this argument would not apply to asking about anything else about London. Context is complicated.

For similar reasons, anyone who asks for the time in Boston probably means Eastern Time regardless of how far they are from Lincolnshire, though I think the more usual and canonical way of referring to that is New York time.

Pedantic: London is currently in British Summer Time, not GMT (which it is in the winter)

I'd expect it to ask the first time around, and then remember the answer afterwards.

There are many European countries that have the same timezone, but I don't remember all of them always, despite living in Europe.

In this case it is totally normal for someone living in Netherlands to ask the time for a polish city.

> Why on earth should someone living in a state specifically ask for the local time in a different location within that same state?

Who said they were in that state when asking? People travel.

True. So substitute "living" with "currently located in". Makes the problem of correctly considering the context of that question just even more complicated ;-)

I find it unlikely that someone in that scenario would ask "What time is it in London?" rather than "What time is it in Ohio?"

I ask for the time in SF or LA all the time, rather than the time in California. Regardless, it's not hard to find cases in the same state that cross timezone boundaries. Cottonwood, AZ vs Cottonwood, AZ 86503 (on Navajo time) is an example.

SF and LA are major cities. London, OH is not.

I assume it triggers just on location, not in context of a time zone. I think Siri just hears tell me something about "location" and then just defaults to nearest.

> Why on earth

Just use your imagination a bit.

Maybe I live in state X at location Y while my parents live at location Z in the same state, about 250 miles away from me, maybe there’s a serious storm where I live and I wonder if I need to check on my elderly parents?

But you'd still be in the same time zone, no? For different queries, like the one you mentioned, that might be reasonable, but asking for the time in London means the assistant should infer England's time zone, if you're in Ohio.

Unless you live in a state like Nebraska that spans across multiple time zones…

True, there are definitely edge cases to account for. Maybe the problem is these AI assistants are trying to make everything location-aware when that may not always be the desired behavior. If there are ambiguities the assistant could ask you to clarify, but that might make for a worse UX.

Or answer both, like a human would do: "It's 3pm in London/England and 9am in London/Ohio"

> Why on earth should someone living in a state specifically ask for the local time in a different location within that same state?

Because they want to know what time it is in a different location.

14 states have more than one time zone. Do you know which ones?

> Do you know which ones?

No and that’s irrelevant trivia. What I know is if the state I live in is on that list. Oregon and it is. Don’t care what the other 13 are if I’m asking for a time in the state I live in.

The point is that just because you know the state doesn’t mean you know the time. They’re unrelated. It is reasonable for a person in Portland to ask what time it is in Ontario. They may know there are two time zones around eastern Oregon but where is the line?

Idaho is the same way. What time is it in Riggins? I’m from Idaho and I don’t even know the answer to that question. It’s a reasonable thing to ask siri.

That's a harder question. The important question here is "does your state have more than one time zone?"

No, it isn't important. What's important is how to answer the question that was asked. The number of time zones in a state has nothing to do with the time in a location. You just need to disambiguate what location is the subject of the question.

Well, it's no less important than this.

"14 states have more than one time zone. Do you know which ones? "

The point I was making there is not all states have one time zone so it is reasonable to think someone could ask for the time somewhere else because they don’t know what time it is there. Even if that somewhere else is in the same state.

I had hoped that would also demonstrate that the solution here is to just answer what was asked instead of saying “that’s not something people would ask.”

They would ask that and if you are designing a system based on what you think users won’t do then you’re gonna have a bad time.

All of these situations are fairly complicated. So the best thing to do ( even for a human assistant ) is to ask more questions. That is the best way to get clarification rather than just trying to figure out what the context is. The assistant should be able to ask a simple question: "Are you talking about the London in UK or Ohio?"

The problem is, for some subset of people, the question is ambiguous. Thus we can reduce the problem to finding that subset and asking additional questions only in those cases.

Exactly. And what about if I initially DO specify it's London Ontario that I'm interested in but five minutes later I ask again, referring to it simply as "London".. shouldn't a "better Siri" come back with 'assuming you still mean London Ontario, it's...' ?

I think it's possible that general acceptance of these non-AI gimmicks being referred to as "AI" will end up pushing genuine progress in true AI further into the future.

Agreed. These solutions seems gimmicky. AI as it is today is really a chain of "if"s. I think aside from some amazing results in niche areas, so much of the attention to ML in general (and DL in particular) is just due to marketing. It's a bubble in many cases, but since everyone is doing it, companies decide to do it as well.

I think you're making basically the same mistake that Gruber is in the article, and that all "location based services" are making. The context isn't the current location, it's where people's attention is.

To take Gruber's example, if you had an office in London Ontario, were talking about setting up a video call, then asked your assistant "what time is it in London", and they picked London England because it's the most famous, you'd question how smart they were.

The context is not where you are, or which "London" is largest or most popular or least driving distance away or where you grew up or where you lived once, the context is why you are asking about the time in London, it's all of the present brain/attention/conversation/local state bundled together.

> you'd question how smart they were.

If I'm physically in a city named London, and you ask me literally "what is the time in London" that's actually an ambiguous question. Why would you specify the name of the place you're currently in? Typically someone in that instance would just ask "What time is it?"

I don't hire people to make "educated guesses" on my behalf. If they detect an ambiguity then I expect them to initiate a dialog that resolves that, not just blithely pick the first thing that comes to their mind.

> If I'm physically in a city named London,

But Gruber is not in London. If he was, perhaps we'd have a discussion about what the right answer might have been, maybe we need some more clarity, that kind of stuff. If I ask a stranger outside (well, if I did before everyone isolated themselves) they would immediately give me the time in London, England, and if I actually wanted the time in Ohio I would have to clarify myself.

> I don't hire people to make "educated guesses" on my behalf.

Of course you do. Do you really want people asking you questions all day when there could be any ambiguity?

I have worked on that problem recently, and yes it is really hard.

You will be surprised on how many cities are named 'San francisco' in the world : https://en.wikipedia.org/wiki/San_Francisco_(disambiguation)

I often think city names must constitute some part of their citizens' identity. How is living in a fake San Francisco pleasant for people who live there?

I wonder which will come first: AI that is truly able to teach itself about our world, or that we are able to define algorithms or otherwise figure out software solutions to most of these these context problems. Of course, if an AI is intelligent enough that problem solves itself.

To me it seems that developing AI on that level will be here sooner than developing solutions to many context problems, given the difficulty the best funded algorithms in the world have answering this question which we humans see as very simple.

That's exactly what gave rise to the first "AI Winter". People had to hard-code things. But with NN and esp. Deep Learning, they thought there's no need to write programs for pattern recognition anymore; the "AI" just learns it on its own.

That wishful thinking has turned out to be dangerous tho, as we have moved towards an ML-dominated world where we don't even know how the ML algorithms produce specific results.

Add that to their bizarre behavior (like this example with Siri) and you'll realize chances of another AI Winter are not low. If we have to develop solutions to many context problems one by one, that may reduce so much of the hype and interest in "AI". We'd be basically back to square one.

> We'd be basically back to square one.

But with better tools! It might not be going from 0 to 1, but going from 0.1 to 0.2 is still progress.

Anyone want to guess where this sign is located?


I wonder if it actually points at Mecca…

It points to the town of Mecca which is close to the Salton Sea in Southern California.

Looks to be in Kentucky. Has both a Paris and a London, only 90 miles apart, so it's easy enough to pick a point on highway 64 that would be nearer to Paris either coming from Louisville or Huntington. Impossible to know which direction though without more context.

As a non-American, can someone explain why there are US cities with the exact same names as cities in Europe, Egypt, Greece, etc.?

Because settlers from Europe came to America and didn't give af what the Indians called their land. So they named places after ones they knew; New York, New England, Boston.

Toronto, Ontario, Niagara, Ottawa, Canada, Mississauga.

Canada kept a few.

I guess mississippi is one but most US places have white names.


Heading west from Clappison's corners near Waterdown.

Lol look at that, a London and Paris right next to each other in Ontario too... nicely done.

in the US, based on the sign style and the road striping, and the drainage grate

On the side of the 403.

somewhere near Versailles, KY ;)

It is THE area people have been working on in AI.

Multi-hop reasoning models have started working surprisingly well. ie. reasoning over multiple levels and conditions.

Common sense reasoning is also getting a lot better. By having huge knowledge bases, the model can actually learn some degree of human like general purpose context. Such as, returning the time at the London which has the most similarity with the user's hidden representation.

There's a higher-order set of contexts as well. Everyone in the conversation (the asker and the answerer) knows that "London" likely means "London, UK" - that's the Bayesian default. So the if the asker wants to inquire about London, Ontario (and they are not continuing a previous conversation about London, Ontario), they are very likely to be explicit in their ask: "What's the time in London, Ontario?"

The asker has a mental model of the answerer's default contexts, and if their question is likely to be ambiguous in those default contexts, they are more likely to be explicit in their ask. The converse is also true - if the asker is not explicit about which London, that's actually a signal to the answerer to lean even harder on default contexts and best guesses.

Humans do this without even thinking because though culture and conversation we are quick to arrive at shared mental models and lean on shorthands "in the other person's head". AI not only doesn't have it's own context, it doesn't have an estimate of it's user's context and where the two might differ.

I disagree. Should the resident of London, Ontario say it every time to a virtual assistant about the weather when it should be obvious they are interested in the place of residence? And so on, all the same context layers apply (British expat in London, Ontario, etc.)

If they were somehow calling a global hotline that could answer arbitrary questions for free, then yes, I suspect someone in Ontario would still clarify. If it was a local hotline, maybe not.

Which is Siri?

Some additional variations:

What if you were calling an individual human personal assistant who knew you lived in London, Ontario?

What if you had previously clarified to this person that when you said London, you meant London, Ontario?

I think both of these questions ought to be relevant to the digital personal assistants that we're creating.

How about asking for context if it's not clear? Any reasonable human assistant would ask if the context is not clear (enough).

Obviously virtual assistants should ask for clarifications. Now imagine every time you ask for the time, weather or a flight to London and you always mean only London UK, imagine how annoying that can be. Makes me wonder if VA's are more productive in such cases compared to typing.

"The time in London, England, is 8:23pm."

"No, I meant the time in London, Ontario"

"Sure. The time in London, Ontario, is 3:23pm."

You start by doing a best guess, and actually listening for a correction. For other kinds of requests, you reply with a best guess and ask for confirmation or for clarifying questions.

Hard, yes. Not impossible.

Right. A good test for a conversational AI is whether it could perform the Abbott & Costello “Who’s on First” sketch with you. It should make assumptions where the characters in the sketch make assumptions, and ask questions borne from confusion where the characters do.

"The average time in London is 5.53 pm"

I mentioned in another comment that with Siri, this exchange does work:

“What time is it in London?”

... gives answer for London Ontario

“No, London England”

... gives answer for England

However there’s no memory to this; the same thing happens next time you ask.

The problem can be reduced to finding the subset of people for which the question is not ambiguous (or the analogous).

The solution for it is, sadly, more data. So I imagine if Google or Apple can listen, "see" and has access to your every electronic communication do. They could eventually build a model that "knows" everything about you. I am pretty sure we have the technology to do it. But the privacy implications for this is terrifying.

> The solution for it is, sadly, more data.

I don't see why that would be necessary, since a human does not need to know everything about you or have access to everything you've ever communicated to anyone to guess accurately that when you ask about "London" you probably mean London, England.

>to guess accurately that when you ask about "London" you probably mean London, England.

It depends on your circles/bubble and the context.

Sure, for me and almost certainly the majority of people the majority of the time, assuming London = London, England is almost certainly the correct disambiguation. However, maybe not someone in Ontario or Ohio asking a question about "London." And I expect that the person who sees London, England as this far away place they certainly don't have regular questions about would find that always being the default annoying.

> It depends on your circles/bubble and the context.

But not on the intimate personal details and communications of the individual person, which was what the post I was responding to was about. Sure, there are going to be circumstances where London, England is not the most likely guess, but you don't need to have access to someone's entire personal history to know what those circumstances are, since a human can spot those circumstances without having that knowledge.

> The solution for it is, sadly, more data.

The problem is these "AI"s are plain stupid. The solution for it is moving on from gimmicky and hacky solutions to true AI.

> This example illustrates how difficult AGI is and how far we are from it.

Does it? AGI is very difficult, but I think this example only illustrates that Siri is kinda bad, given that DDG, Google, Alexa, and Bing all got it right.

TBH I always feel amazed about how worked up people get about stuff like this, especially people familiar with software who should know that there are millions, maybe billions of edge cases like this in a generic knowledge system, and thus at least it's easy to make a mistake like this. I mean, the time it took him to write his blog post is probably more than all the times it would take him to follow up with "What time is it in London, England?" It reminds me of someone who commented that "there must not be any black people who work at Apple" because it pronounced "Malcolm X Blvd" as Malcolm 10 Blvd.

I mean, if anything, just appreciate how amazing humans are at differentiating these ambiguities.

I agree with you, but then consider all the influential people who think the exact same category of technology--AI aka machine learning--will produce a self-driving car that is safer than human drivers, or take away every human job in X years.

I think it's very worthwhile to point out these seemingly basic errors as a way to maintain appropriate skepticism about the limits of our technology.

But the post points out that Siri from other Apple devices gets it right. Apple’s “generic knowledge system” can answer this. It’s only Apple Watch which has trouble.

That’s kind of weird. It’s not that Siri is especially bad. It’s that “Siri” is something different depending on how you query it. Other online search systems aren’t like that, and integration and consistency are typically Apple’s forte.

Seems like an easy plausible explanation is mobile devices (like a watch or phone) take your current location into account, while a stationary device (like a HomePod) does not.

Look, I'm not arguing it's not a bug, but I'm just really surprised at how software people, who I think should know better, are surprised that such bugs exist, or more importantly that completely eliminating all types of this class of bugs is basically impossible with current technology.

Answering (with a human voice!) the correct time zone for the wrong "London" is about the mildest possible bug you'll ever see. I might not even call it a "bug". Let's call it a mild inconvenience of modern life!

The aspect that's ruffling feathers, I believe, is that it's one of those cases where someone might have reasonably assumed something was built one way 'under the hood', and was confronted with an effect which forced them to see that it was not implemented that way at all. The issue isn't the 'bug'. It's the realization that their mental model was wrong.

Specifically, something has a name ("Siri") which might lead one to believe that everything from that manufacturer using that name refers to the same thing. (Isn't that the point of a name?) Clearly, it's not.

Your hypothesis sounds plausible, so I tested it. I have a Mac laptop, which has the same 'Location Services' that iOS has (AFAIK). I asked Mac Siri what time it is in London, and got a response for the one in England (further away from me). So that doesn't explain it, or at least not all of it.

Siri is often self-inconsistent. Ask a query again on the same device and you might get a completely different answer.

Exactly. I'll speak into my AirPods "Hey Siri, 30 minute outdoor walk" to start a 30 minute workout on my Apple Watch.

Half the time, Siri replies with "I'm sorry, you don't have an app for that. You can try searching for one on the app store."

Repeat the same exact query to the AirPods seconds later, and bam, it starts the workout on the Apple Watch.

Just because it's hard to implement right doesen't mean Apple should get a pass. I mean, they aren't forced to create a voice assistant by regulation. If they can't make a good one, (and this applies to Amazon and Microsoft too) they should have just left it in the lab until they can.

> they should have just left it in the lab until they can.

Totally disagree, because the only way these assistants get better is with real-world usage (which is why I can definitely agree that one should wonder why Apple isn't improving as fast as the competition).

It was only a couple years ago that using Google Assistant was an extremely frustrating experience. I'd say it got about 5-10% of my words wrong, which meant it got my intent wrong about 25-35% of the time. These days I find its accuracy uncanny - it almost never makes a mistake with most of my "standard" queries. No way it could have gotten that good without real-world feedback and data.

It is still an extremely frustrating experience. All of them: Google, Amazon, Apple. That it gets it wrong, ever, is too much for me. The first time I ever had to repeat myself getting directions on the road I was done.

I don't understand why anyone other than hobbyists can stand to use these things. They are so obviously years away from ready for serious use, and the novelty value wore off years ago.

Not shitposting here, these are serious comments about absurdly bad UX.

> That it gets it wrong, ever, is too much for me. The first time I ever had to repeat myself getting directions on the road I was done.

How can you stand to deal with humans?

Humans are so many orders of magnitude better at comprehension that your comment feels a bit disingenuous.

I was replying to the parent comment that said that basically any errors were unacceptable, and that ever having to repeat himself was a deal breaker. This seems like a ridiculous position to me given that, as you point out, humans are much better at comprehension, and even they can't hit this standard.

So they should go completely counter to the way that businesses traditionally function and keep something in a lab for potentially decades instead of shipping a "good enough" solution that can be fixed along the way? Especially one that would get valuable information and edge cases from a live system? It's not like it's safety critical software. It's neat to have, sure, but you won't die if you get the wrong London's time.

> shipping a "good enough" solution

I think the debate here is about whether anything has shipped that is actually "good enough". I don't think it's that controversial to avoid shipping stuff that's not good enough.

I think it's good enough. Sure, in this very cherry-picked example it fails. But for a lion's share of functionality it works just fine.

No way

This is the difference between a usable product and something that is not

If my voice assistant is going to make a significant % of errors, it either needs to be very cheap for me to correct it (it's not -- usually you retry and if it keeps failing what do you do?) or I'm going to stop using it

Steve Jobs made great, not passable, products. It's a shame that Siri is so far behind

Siri may be a marvel of modern technology. If the competition does better, than it's reasonable to complain about Siri.

But if you are Apple and building the Siri product, you think this would be one of the first use cases you would develop and/or test. I remember asking Siri to convert a currency for me and it just showed me search results for 'currency conversion'. Again... wouldn't this be a query you would test for?

That's actually a pretty typical blog post for this guy. So I'd say it's more like "some people", not just "people".

Ranking and NLP aren't easy. If you are asking a slightly related question (for example, "What is the weather in London"), and if you are living at some place where the nearest major town is called London, but is not London in England, you would expect it to give you the weather in "your" London. However, it you are asking for the time in a particular city, then the ranking should of course consider whether the timezone of the city you asked for is different than your own - it makes no sense to ask for the time in a city which lies in your timezone. Then again, if the distance from your location to that other London is greater than a certain threshold, the question could imply that you actually do not know whether the city lies in the same timezone as your location.

All these thresholds or ranking factors seem to come intuitively to humans (I would guess a good intuition for them is actually a sign of intelligence), but it seems to be incredibly hard to capture them in ranking.

As others have pointed out, a solution here would be to make Siri more conversational. A simple "Which London?" could've removed the ambiguity and given Siri the opportunity to learn something about that particular person (that London, England is more important to him than London in Canada).

> A simple "Which London?" could've removed the ambiguity and given Siri the opportunity to learn something about that particular person (that London, England is more important to him than London in Canada).

IMO I would be very disappointed if Siri started asking clarifying questions at a significantly higher rate. Siri is already a bit too chatty, and I never feel like having an extended conversation with her.

I’d rather she just say the wrong thing (but make it clear that the answer is for a specific London, e.g. “The time in London Ontario is...”) and I can correct her. It’s the same number of conversational “turns”, but in the happy path when she actually gets it right the first time, it’s one-shot and done.

It’s a lot harder to get signal on this for learning, but I feel like there are ways around this as well. (Maybe saying “thanks” can signal she got something right, and prefixing the next utterance with “no” could signal it was wrong...)

Alexa responds to "wrong", and many variations of that, with cancelling the previous action and thanking you for the feedback.

> Siri is already a bit too chatty, and I never feel like having an extended conversation with her.

Tell me about it. I have to unpair my bluetooth headphones every day, for stupid reasons that aren't Siri's fault. But when I say "Siri, open bluetooth preferences", it parses my command on screen VERY quickly, and then slowly enunciates "Okay! Let's take a look at your bluetooth settings." I'm just tapping my foot and waiting for her to quit talking.

Of course, then, 1 out of 3 times it takes me to the wrong settings page. Because if Settings has been opened recently, it can't deep link from Siri. /shrug

As others have pointed out, a solution here would be to make Siri more conversational.

But that would make it almost as smart as an Infocom game from 1981. Something, something, doesn't scale, mumble, something...

The problem sounds a little like collaborative filtering. If you have a certain affinity with cities A,B,C, then you can compute the expected affinity with a city X by looking at the affinities other people have with X, and their affinities with A,B,C.

Instead of looking at people, you can also scrape websites to get the relations. But here you may get a recursive problem because if a website speaks of "London", you might not know in advance which London they speak of.

I now have most of my adult years learning how to construct a phrase that gets the right results for Google, and later Assistant. It got to the point where I'm certain it must be a headache for whatever team is trying to support natural language processing in these - all proficient users ask for some artificial gibberish and get where they want to be.

Here comes my favourite brain freeze moment - recently my parents asked me to explain this to them. How do you construct a good search phrase? My brain blanked. I HAVE NO IDEA. It seems I have learned fluent Goonglish without noticing, and now can't explain the grammar or vocabulary of it.

My 2yo daughter reeeeally loves the Aladdin soundtrack, but the French version (we're in Québec). It took us way too many tries to get our Google Home to play the correct version with voice commands, even adding the language ("Ok Google, écouter Aladdin en français") wouldn't work. I now have a small note with the correct incantation posted besides the Google Home, because even forgetting a single word will play the English version instead. (For the curious, the correct incantation is "OK Google, écouter Aladdin bande originale française du film").

I would summarise it as: use separate keywords instead of sentences. "Change Light Bulb" instead of "how to change a light bulb". "Black Science Guy", "Kevin Durant height", "rails has_many api", etc...

Recently Google got much better in understanding full sentences and there are tons of SEO optimized pages for certain phrases. Nevertheless, using keywords is what I imagine advanced users do.

It also got much worse at keyword searches. It seems like the one capability came at the cost of the other.

That's definitely what people were referring to when they had good googlefu abilities. It was always odd to me when people would have trouble finding stuff and come to me for help. I somehow picked up that language to search effectively while growing up during search engines infancy.

It's weird that it feels like my tried and true abilities are getting worse. Or Googles algorithm is hurting some of us that became very proficient in very specific ways.

I wonder if anyone is working on special languages for talking to voice interfaces. Maybe a reduced grammar would allow for better recognition accuracy and reliability. And we could get more helpful corrections.

The problem reminds me of the difficulty of programming in applescript. In applescript, articles like "the" can be inserted optionally in the code, and there are lots of equivalent ways to write things, i.e. "if x equals y" is the same as "if x is equal to y". As a result I never remember the syntax, and error messages are less helpful.

From my limited understanding, even in lojban, a constructed language with unambiguous grammar, you can have semantic ambiguity.


The fact that there isn't a feedback mechanism to let the Siri team know that it responded incorrectly tells me everything I need to know.

Until they have real metrics around how often Siri fails they will continue to think that their correct response rate is great.

Companies usually hire editors to manually annotate the quality of the algorithms on anonymized samples. They don't need direct user feedback to gather quality metrics. You also can't tell which step failed in the pipeline: speech recognition, query understanding, or the actual search.

Why can't you know which step? It's their system..they can build as much tracing as they want into it.

Why would editors make user feedback any less valuable? It's hubris to think it's not.

I'm just waiting for my toddler to start repeating "Hey Siri, nobody was talking to you."

My preschooler does that whenever I’m working with a tool that requires me to bend my wrist backwards and the watch-Siri activates.

“Sorry, could you say that again?”

Nobody asked for you to interrupt my chopping, Siri.

They likely don't need the manual feedback as they would be able to tell that someone asked the same question twice in a row, and then look at the data to see what the issue was.

The craziest and most confusing behavior of Siri for me is:

Sometimes you can ask a question and watch it be perfectly transcribed in real time, but then receive a nonsensical answer from it. Ask the exact same question immediately after on the same device, transcribed exactly the same way, and get the correct answer.

Where does such unpredictability come from? How can Siri transcribe the words correctly but fail to deliver the right answer?

> Where does such unpredictability come from? How can Siri transcribe the words correctly but fail to deliver the right answer?

Voice assistants generally use both the text transcription and a bunch of contextual metadata as input. That metadata could include things like what's currently visible on the screen, your location, your recent queries, etc.

So even though the underlying algorithms powering the assistant may be deterministic, the input data between two seemingly identical queries could vary quite a bit.

For instance, Siri almost certainly has context around the previous questions you've asked. It would be reasonable to assume that if an assistant received two identical questions back-to-back the initial answer was wrong.

In that scenario, the assistant might decide to use the a different answer (perhaps one that had a lower ranking) in an attempt to get it right.

I looked up the last screenshot I had of this. What I asked was "In 2 days remind me to call FRIEND_NAME", and Siri created a reminder that just said "call". Transcribed perfectly, wrong content in the reminder.

I tried it again right after, and the reminder said "call FRIEND_NAME".

I don't think there was any previous conversational context or anything like that. Hard to fathom how that could happen.

Maybe some overly aggressive A/B testing?

Might as well comment, first employee at Siri here. This result “maybe” should provide ambiguity resolution, but where does it stop. The she/he who compared it to Google was right on. Siri provides singular results in most cases vs multiple search style results. We did use geo for locality based results in the past. This would solve the problem the OP mentioned, not sure if they call location for these requests now. The other person who mentioned we can’t/couldn’t train on data is correct too. Again, privacy first. Be proud, Apple cares a lot. When the Siri commercials hit (No one told us there would be commercials) when we launched, we got decimated, and couldn’t debug the issues, user utterances were not allowed to be logged. Luckily, after much sleep deprivation, one of my engineers (love you Stu) said, “hey, aren’t they running commercials” to all our surprise. We convinced the privacy team to let us log word parts. Then we started to see words that were present in the commercials. Fun fact, also happened when Tom Cruise was presenting at the Academy Awards. We had millions+ asks all at the same time, again word parts. “height”, “tom”, “foot”, etc.

Interesting. I can see how the basic algorithm might go "I've been asked for the time (or temp or whatever) in cityname. Citynames are routinely reused globally; what is the closest such cityname?"

But that fails completely when you get to names like London (or Paris or Moscow or Cairo).

But it happens with people, too. I'm from Mississippi, though I haven't lived there since I left for college. I now live in Houston. At a family reunion many years ago, I ran into a cousin I hadn't seen since we were kids. She asked where I was living, and I told her.

"Oh, isn't it terrible about that wreck?" she asked.

Baffled, I asked for more information. "Oh, you know, that wreck over on 406!"

I did not know. "I'm sorry, Houston's really huge. I don't know what wreck you mean."

"Oh, did you mean you live in Houston, TEXAS? I thought you meant Houston, MISSISSIPPI!"

I was, at the time, about 30. I grew up in that state, and lived there until I went to college. And until that moment, I had never even HEARD of Houston, Mississippi (a metropolis, it turns out, of about 3600 people in the misbegotten northeast corner of the state).

It just feels like what is the closest such city name is pretty obviously an insufficient test.

To approximate what a human would do, one would presumably want to start by ranking places on a range of dimensions:

* nearest

* biggest (or maybe size category: big city, city, town, village)

* how many times user has asked about this place before

* how recently user last asked about this place

* ...

If most/all these rankings put the same place in the top spot, go for it! Otherwise, ask the user for clarification.

I wonder if there are Chinese cities with names close to major world cities. Paris has 2.15m people, its metropolitan area has 12.x million. A medium sized city im China has more than that. If by coincidence it's also named Paris, getting that as the search result would be annoying.

Oh, I absolutely agree. But I can see how that might have come in as a first-assumption that was never revised. Which, if you think about it, is probably a huge class of bugs industry-wide.

To throw a tongue-in-cheek additional example in here: the population of London, Ontario is (significantly) greater than the population of the City of London, UK.

Reality is hard. And with machine learning (especially proprietary, remotely-hosted machine learning) there's rarely a way to pinpoint a line of code and say: "this is what happened and why you're now frustrated and firing hypothetical personal assistants".

Yes, but in the same way people don't usually mean "London, Ontario" when they say London, they also don't usually mean the City of London (which, for the benefit of people who may not know, is a tiny portion of London with a population less than 10,000).

A group of us once tried to rank cities in Europe by population only to realize that most of them are effectively incomparable.

Cities sometimes have clear legal boundaries that feel irrelevant to the question, like the City of London, but more generally have metro areas that sprawl well into an ambiguously defined countryside. There's rarely a "this block is city, the next block over is clearly not" situation, so the number of people you include ends up being pretty arbitrary.

While it is arbitrary, and not a city (in the sense that it hasn't received that status from the Queen), Greater London is absolutely a defined administrative area in the UK, with a governing body (the London Assembly) and a mayor (the Mayor of London -- not the Lord Mayor of London, who is the mayor of the City of London).

Anything can be ranked as long as you clearly define the ranking metric first.

When ranking by population it often makes most sense to use the population of the metropolitan area. That is, to ignore the administrative divisions, which vary too much, and focus on the physical reality of the urban area.

Just to make things a bit more complicated London has two cities. The City of London, and the City of Westminster, which is also a borough of London.

Two cities in one City; it shouldn't be allowed.

It's time to refactor London.

Already done. We're talking about "the next gen" and it's still dodgey.


Yep; my (indirect) point is that there are multiple possible reasons why Siri may have made the judgement that London, Ontario was more relevant when answering.

My guess is that Apple would find it difficult to provide robust references to John to explain why it happened, or how they've fixed it for him (and whether that fix is a one-off workaround for his complaint, etc..)

People always expect the 'obvious' interpretation, but sometimes it's difficult to define exactly what that means for everyone. As another poster said here, context is also a very important factor.

Precisely. It's the user's context that matters, and that can change even for a particular individual at different times and locations.

Remote, proprietary personal assistants tend to apply their own (generally unknowable and unaccountable, from the user's perspective) interpretation of the context.

Considering (IIRC) asking about the weather is what they show on the TV ads, they should've made sure their city selection logic returns the answer that most people would find acceptable..

There's a Woodland Hills in Utah and a Woodland Hills in California. If you ask Google what the weather is in Woodland Hills, it will ALWAYS give you the weather for California. Even if your current location is Woodland Hills, Utah and even if your address is set to Woodland Hills Utah in your Google account or Google home.

I live in Alton in Hampshire in England. A town of 20,000ish people. To be fair Google (Maps etc) tend to get the correct Alton, but a lot of mickey mouse site often thinks I am in Alton, Shropshire, a tiny village next to a more famous theme park, Alton Towers. Which is very annoying setting filters job sites as I am never sure which Alton it thinks I am in, local ads that don't make sense etc.

Worse there is an Alton, New Hampshire, which confuses even Google sometimes.

Even worse, Apple seems sometimes confused where I am as I have twice woken up to see a tornado warning on my IOS lock screen in the mornings. I live pretty far from any decent tornados. Unfortunately, I have been too sleepy to prevent myself from unlocking the screen before I remember to screenshot it.

Why would you need to specify your location? You can just ask "How is the weather?" or "What will be the weather like tomorrow?"

Maybe you're traveling and want to know what the weather is at home.

My usual dialog with siri:

Me: "Hey Siri, play Radiolab podcast"

Siri: Which Radiolab podcast, Radiolab or Radiolab: More Perfect"

Me: "Radiolab"

Siri: Which Radiolab podcast, Radiolab or Radiolab: More Perfect"

Me: "Radiolab"

Siri: Which Radiolab podcast, Radiolab or Radiolab: More Perfect"


Me: "The first one"

Siri: "I don't know >the first one<"

Me: "Siri you're useless"

Siri: "That's not nice"

Me: "Could be but it's true"

"Hey Siri, set a timer for 5 minutes"

90% of the time works fine, and it's essentially all I use Siri for. It's very convenient when cooking and my hands are dirty. But 10% of the time I get something along the lines of...

"I'm sorry, but you don't have the Timer app installed".

"I'm sorry, but you don't have the Timer app installed".

"I'm sorry, but you don't have the Timer app installed".

"I'm sorry, but you don't have the Timer app installed".

It's infuriating because I know Siri is dumb so I use the same exact simple phrases to avoid confusion. Sometimes it works, sometimes it doesn't. It always transcribes the command accurately though! I've actually lost my temper and smashed an Apple Watch before over this. This is in my house, on a very reliable network, always with my phone within a reasonable distance.

It's ridiculous how Siri is still this shitty. I have an 11 Pro and even on such an expensive phone I can't really trust it to do anything more advanced than set timers. Every few months I try to do something else and just get annoyed at how bad it is.

Before lockdown I even had it disabled entirely because it would get activated randomly from time to time, even if nobody in the vicinity said anything remotely close to "Hey Siri".

The problem is not just that it is wrong, nor that it doesn't have enough personal information, but that it lacks proper personalisation and the ability to learn.

You can't reply with "no Siri, not that London" and have it remember. It doesn't learn your voice among the people who normally use your Siri in your household.

"Artificial intelligence" is always going to make mistakes, as do real humans. Humans can perform unsupervised learning - in fact it's one of the key skills that employers like to select on! Until AI can learn in context it's going to be very limited.

Siri indeed never learns.

I've had to disable "Hey Siri" because my daughters name is pronounced vaguely similar to Siri. Worst thing is, Siri transcribes what it hears, and it transcribes my daughters name. So it doesn't hear wrong; it just activates on a different name than Siri.

I've tried telling Siri to shut up; but it never learns not to activate when I call out my daughters name.

That might be because the activation words are recognized by a separate chip (so it's low-power and works offline). Whereas the rest of the conversation is with the software service.

At least that's what I heard about how iirc Alexa works.

Activation words are fuzzy by design.

Siri is easy enough so we never looked much into it, but “OK Google” for instance looked like a real PITA, so we did some research before buying an assistant.

It appears a ton of people just intentionally say “Ok GooGoo”, “Ok Boogle” etc., whatever is easier for them to pronounce and it works perfectly fine.

When those designs strip your privacy, 'fuzzy by design' is cery much a bug to the user, and only a feature to the company mining the data.

I may be biased as I helped a voice recognition internal project in a previous life.

It’s a genuinely hard problem to solve, and I am willing to give the benefit of the doubt to Apple for instance when they have humans reviewing samples. There may be other motivations, there’s ton of people in any of these companies, any given feature must be seen from a different angle depending on the department looking at it.

But I think a lot of what we see as privacy violating is primarily an effect of the flaws and all the hacks needed to make the feature work at all (when it works).

It activated when I greeted my cat. My cat is called "Timmie"

Siri is not artificial intelligence-it is speech-to-text + a poor search experience with one result.

I had never used an Apple product before the company which I joined recently gave me a MacBook Pro. I am really surprised how bad the product quality is. The calendar notification is very random. Sometimes it fires, sometimes it does not. I have missed couple of meetings because notification popped after the meeting was over. Similarly the keyboard shortcut is random. Sometimes it opens the app, sometimes it does not. The laptop also gets very hot if you are not sitting in A/C. Not sure if it is this specific laptop or it is a general issue

Yeah it is the norm these days. I’ve been using macs for 15 years and they bave been fantastic until the very last iteration (after 2016, 17).

My latest MacBook (16") is so unstable that it is actually funny at this point.

So do people just buy these to look cool? I tried using a Macbook many times, but often got frustrated and went back to my good old Linux laptop for development. Doesn't look quite as slick, but certainly gets the job done.

No, I don't think so. There are multiple aspects.

I develop on this thing. It is running a great Unix os. I can't stand desktop Linux. The hardware quality was the best with a wide margin before the latest gen. Battery life is also great. I like them for development work when they are stable.

A lot of people are also really invested into the ecosystem. My entire photo collection is on iCloud. I use an iPhone. I can copy paste between my computer and phone. My Apple watch unlocks the computer when I'm near... List goes on.

But now I feel like Apple is a fantastic phone company that also happens to make some computers. They have been degrading pretty bad.

It's also that Windows/Linux has many of these issues as well. It's not as if Windows 10 notifications are clear and intuitive. When I go to my desktop after a day of work, Windows will slowly replay every single slack message and email I got all day, one at a time, for almost an hour, as single notifications.

I think it's less that OS X is bad now, but more that it's finally degraded to a level of annoyance that people just have gotten enured to with Windows. It's not to say that that's a good thing, but at this point, I have known bugs and annoyances with all of the computers I work with, no matter the platform.

Some of it is also that Apple has a "real" integrated ecosystem. To what you say, you can easily move things between iOS and OS X. If you're watching stuff on your Mac, you can throw it to an Apple TV or your Airpods. Windows doesn't have a version of that that "just works". The closest you get is opting into Google's ecosystem and going Chromecast/Android, but I'd rather not trust Google with even more of my info.

Honestly I'd dare to say most of Apple's market right now is purely from vendor lock-in. Both their hardware and software are getting worse, but not bad enough for people to switch their entire digital lives to a different ecosystem.. not yet, anyway.

My first Mac was an employer-provided MBP in... oh, 2011 or so. Before that I'd used DOS, Windows (3.1 and up, including NT4 and 2K) and Linux (Mandrake, Debian, Gentoo, Ubuntu, roughly in that order with a little Redhat and Fedora here and there). I'd seen some early OSX server edition thing, but not really used it, and I'd used pre-OSX Macs at school (hated them, "it's more stable and faster than Winblowz" my ass). Some exposure to Solaris, too. Used BeOS (loved it) and QNX on home machines for various purposes, as well.

The MBP was the first laptop I'd used that 1) had a trackpad good enough that I didn't feel like I needed to carry a mouse around to use it for more than 10min at a time, and 2) had battery life good enough that I didn't feel like I needed to take my power supply with me if I'd be away from my desk for more than an hour. It had every port I was likely to need for most tasks. In short, it was the first time I'd used a laptop that was actually usefully portable as a self-contained device. They kinda ruined that appeal by going all-USB-C and The Great Endongling, but that's another story.

It was also very stable, and over time I came to really appreciate the built-in software. Preview's amazing (seriously, never would have thought a preview app would make a whole computing platform "sticky" for me, but here we are, it's that good), Safari's the only browser that seems to really care about power use, terminal's light and has very low input latency, it comes with a decent video editor, an office suite I prefer over anything I've used on Linux, and so on. In short it's full of good-enough productivity software that's also light enough on resources that I don't hesitate to open them, and often forget they're still open in the background.

These days I like having a base OS that's decent, includes the GUI and basic productivity tools, and that's distinctly separate from my user-managed packages (homebrew) rather than having them all mixed up together (yes, I could achieve this on Linux, if it had a core, consolidated GUI/windowing system so various apps weren't targeting totally different windowing toolkits, but it doesn't, so separating a capable and complete GUI "base OS" from the rest of one's packages gets tricky). There are quite a few little nice-to-haves littered around the settings and UI. Most of the software is generally better polished UX wise than Linux or Windows, and that doesn't just mean it's pretty—it performs well and, most importantly, consistently. There are problems and exceptions to "well and consistently" but there are so many more issues on competing platforms that even if it's gotten worse, it's still much nicer to use.

Given the premium on hardware (that's come and gone—at times there almost wasn't one if you actually compared apples to apples [haha], but right now it's large) I'd rather use Linux (or, well, probably a BSD, but that'd mean even more fiddling most likely) but the only times that's seemed to function genuinely well and stably compared to its competition was when I either kept tight control over every aspect of the system (time-consuming, meant figuring out how to do any new thing I needed to do that other systems might do automatically, which wasn't always a great use of time to put it mildly) or in the early days of Ubuntu (talking pre-Pulse Audio, so quite a while ago) which was really sensible, light, and polished for a time.

I do still run Windows only for gaming, and Linux on servers or in GUI-equipped VMs for certain purposes.

It's not just Apple, I've got a Mi phone, sometimes reminders pop up hours after they happened. They've mucked around with the default android lockscreen to save power and I think this is causing the problem.

The devices are so complicated now that they cant do their most basic functions right.

> The calendar notification is very random. Sometimes it fires, sometimes it does not. I have missed couple of meetings because notification popped after the meeting was over.

I see something similar and assumed this happens because Mail / Calendar are relying on ics attachments (not sure what the behaviour is with the Gmail integration). I believe this means that if Mail is closed you don’t get Calendar updates until you open both and refresh.

Either way I find I have to refresh Mail and Calendar a lot to keep them in sync.

Heating is either a hardware issue or something like a broken program running continuously - on a normal setup that doesn’t happen.

Calendar / Todos depends on the backend. If you’re using Exchange, check the settings to confirm that it’s not set to poll every hour or something like that.

I had this problem too. Maybe try charging it on the other side. It sounds crazy but it's true:


It's ridiculous how poor in functionality all of them are... The best they can do is, what? Creating schedule entries, for me. All I ask Google is the weather, time, some search when I'm lazy and translate (the voice translate app itself is great btw). Feeling like a total corporate bitch saying "Hey Google" every time, too :D

This is supposed to be a personal assistant. And I have a whole list of what it could do for me, personally. But it doesn't.

I've been trying to figure out how to hook Google's speech recognition and voice into other apps, since they're great and it's 99% of what I need, hands-free control and feedback. Maybe they should make that easy, preferably offline and let other people create their own personal assistant modules or something.

> Maybe they should make that easy, preferably offline and let other people create their own personal assistant modules or something.

Like they would ever do that. Then you would no longer be their "corporate bitch".

Seriously, the one thing that stands between home assistants and being useful is opening the software up and letting it be used by regular OSS devs. Alas, every one of the four big providers (Apple, Google, MS and Amazon) treat them as their moat; they want control over the ecosystem. It's the same in many other places in the industry - we're technologically way behind where we could be, because everyone wants to be the platform and commoditize everyone else, which necessitates having total control.

I'm fairly sure they could still control and monetize such a product. Kind of like Android.

Maybe there's already something like that in the works, with all the talk and investment in AI, we should be seeing some real world results...

I agree, some things are just astonishingly bad given the immense effort and resources that are being put into machine learning.

Microsoft OneDrive tags my photos. It's mostly useless. For instance, I have some pictures of squirrels on the tree outside my window. Squirrels can really do the most amazing things on trees, but they are small compared to the tree.

Microsoft with all its AI muscle will invariably tag those images as something like #Outdoor #Grass #Plant #Tree.

It's the same problem with all of those benchmark beating AIs. They have no clue what's special about the picture and what just happens to be in them as well.

I’ve had Siri disabled for years. Even the basic “call home” works every 3rd time. I try it for a few minutes with every new iOS update only to see it’s still the same dumbster fire.

When I lived in the United States I frequently used Siri while driving to setup the gps or change what music I was listening to. But indeed, ever since moving to a country with a nice public transportation system, I’ve had it disabled. There just isn’t a compelling use case.

Meanwhile, my iPhone has gotten much smarter about adding meetings to my calendar, or guessing the person who is calling me. These seem like the real use cases for AI going forward.

TBH, Google Assistant is not that much better. In the last few months it has become absurdly racist against my Italian accent, replying to me in Italian after I ask stuff in English - and getting the question wrong anyway.

But yes, Siri is the worst.

Are you sure it's due to your accent? That seems like a difficult feature to even implement. I know Google loves to play language shenanigans based on your current IP address.

Mine will sometimes parse English, sometimes German. Sometimes it will understand English, and answer in German. It's a total crapshoot.

I live in England.

Google Assistant failed on me recently trying to set a timer. It correctly understood my "set timer for three-and-a-half minutes" request, says it's setting a timer for 3am instead and proceeds to actually set a timer for 30 seconds. How is there that much disconnect between the stages of the query?!

Screenshot: https://twitter.com/R1CH_TL/status/1252232170237640706/photo...

You consider that racist?

There’s nothing worse than being forced to use a language you didn’t explicitly request.

Well, maybe some things are worse.

Probably at least number 3 behind the Holocaust and Internet Explorer.

Yesterday I was washing the car while listening to Music with the air pods. I mistakenly clicked them and it launched Siri which for some reason it called a number on my phone through Facetime.

I always immediately disable siri because something similar happened the very first week I got my first iPhone. It seems to want to call the one person you haven't talked to in years.

Ha. Same thing for me. Made me rush to the phone as I heard the number it wanted to call.

It isn’t really ridiculous. The number of people that buy an iphone because of Siri could share a pizza.

It may not be the main selling point but it's certainly a factor for many people. And the fact that Siri is so bad makes the whole phone feel less high-quality.

And maybe I'm the exception here, but I have refrained from buying an Apple Homepod specifically because of how bad Siri is. If it was on the same level as Google Assistant I would have bought one by now.

Back when Siri was new and I was much younger this was something I really wanted…

I'm convinced it's actually gotten worse with the 11, I recently went from an iPhone 7 where Siri only really activated randomly a handful of times a year and could set timers (Only thing I ever use it for) with a shout across a room over the sound of an oven and fan.

Now it randomly activates multiple times a week and really struggles to even pick up me talking to it right next to it.

Convinced they've switched to a less capable microphone system because assistants were all the rage in the 7 era but now I think people have realized it's not really that important.

It even sucks at setting timers. I asked to set a timer for 50 minutes and it clearly said 50 minutes on the screen and then “corrected” it to 15 minutes.

For a while, it randomly decided that “call my wife” meant to “call my mom.” It clearly said call my wife on the screen and then switched to “mom”.

You have to know the Siri hacks.

Set a timer for fifty-one minutes or fourty-nine minutes.

Even siri can hear that

Ask it to set a timer for 5/6ths of an hour.

Like others have said - it's Apple's privacy policy, they don't know enough about you and can't use the recordings to train.

I think Apple should have been more honest about it in their privacy messaging -

"Hey guys, pretty please can we listen to your Siri recordings? We know it's not the privacy style you're used to from Apple, but if you want Siri to ever not be a piece of crap, this is really the only way."

I don't buy that anymore. There's a multitude of ways to improve Siri while respecting privacy. Apple doesn't have to go the Google way to improve.

Apple knows a ton about me, they have realtime access to my email, calendar, contacts etc. If they have guarded access, then I would accept a toggle. A lot of Siri processing happens locally on the device nowadays, which could be why the Watch, Mac, iPhone and Homepod all can give wildly different results.

Also, Apple could train it themselves, they possibly do, except we haven't gotten an update yet. A large portion of my personal training data could be stored in iCloud, I mean my passwords, mail, documents and my photos are there, right? The analysis of my voice data is sent to Apple anyway.

I think that up until the real AI appears that can mimic human personality and expand it, understand variety of languages, virtual assistants will remain what they are: virtual assistants capable of executing basic commands provided by a human (preferably in English). This technology isn't bad per se - in particular cases is helpful (blind people) but it's still far for what we dream about.

My Nokia 1320 with Windows Phone 8.1 come with a very basic VA that was capable of understand Polish but only if I drop all grammar and talk like a robot. The "call mum" is "zadzwoń do mamy" in its proper form, but I had to do "zadzwoń do mama" which sounds unnatural; not mention that stuff is also being read without proper Polish grammar; "calling mum" is synthesized as "dzwonię do mama", not "dzwonię do mamy". The grammar complexity is a problem for VA technology and probably that's why neither Siri nor Cortana supports Polish or other Slavic languages, not mention dozen of other languages.

I mostly use Siri to play Spotify, but most of the time she won't listen to me through my car's microphone. If she isn't working then I need to pull my phone out of my pocket and place it somewhere the phone can clearly hear me.

Every once in a while she will decide to listen through my car, but it's very rare. I don't have Apple Car Play so not sure if that's a factor.

Google has been far better and more consistent listening through my car even if it didn't get my query correct all the time, at least I could correct it without pulling my phone out.

I almost never use the "google assistant". However it has an habit of firing up while I'm driving listening to podcasts : "I didn't understand your command" (generally, at a annoyingly loud level, with that). I didn't ask you anything, you stupid. You've just misunderstood a sound coming out of the phone itself, where nobody said anything remotely related to "hey google" or "OK google". Puhlease.

I turn off as many settings for Assistant as I can, and disallow the Microphone permission to the "Google" app. That should help you out.

Siri is utterly useless. Like, I just don't. The only capacity in which I use it is to tell it to open an app on the phone, or set the timer.

Siri is like that nice employe who was hired by way of nepotism, and she's attractive, but she sucks at most things, but the organization won't fire her because of aforementioned nepotism in the organization and the only reason you put up with her is cause she's attractive and she, at the very least, makes coffee and makes photocopies just good enough, but you can't trust her with more advanced tasks.

What's worse is that the organization also won't hire her more talented and equally attractive contemporary Google Assistant because of aforementioned nepotism. The boss thinks there's only room for one assistant.


Same deal for me but with Google. I had to turn it off because it kept activating itself, despite voice training it to apparently just my voice. I'd be driving in the car by myself, playing music (via BT, from Google Play, on the same phone!) and it would randomly pause my music and give me an assistant prompt (after, I assume, hearing something in the song that it was playing itself that sounded like OK Google). It was absolutely infuriating and would come up a few times a week.

Still? Siri has actually gotten worse.

In the last year or so it's gone from correctly handling "add red salsa to the shopping list" to consistently adding two items "red" and "salsa". (It also fails on "buttermilk" and others.)

And around the time this started happening, Siri went from acting after a short pause to saying "just a sec" after a short pause.

Perhaps it's time to file a feature request to Apple to allow us to plug in alternative digital assistants in place of Siri.

"Timers"? You mean: the timer. You can still only set one timer. Yet another deficiency. Its like the people who work on iOS don't ever cook multiple things at once.

I get the accidental invocations about once a day, because my dog is named Maisie and apparently that's just close enough to "Hey Siri".

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact