

How Accurate is Apple Maps in Canada's Largest Province?  - bane
http://www.mtonic.com/applemaps/

======
jarek
Good attempt, but some of these results are wonky.

Right off the top of the list, the result for Allanburg is actually perfectly
right - administratively it's part of Thorold hence why GeoNames came back
with that. Dain City is maybe 500 m off the centre rather than outright wrong
as suggested in the list. Ayr comes back as "8 George St, Kitchener ON N0B
1E0, Canada <+43.29177700,-80.45398100>" - the coordinates and the postal code
are correct, the street and city name are not - I don't know how to rank the
correctness of such a result. Same deal with nearby St. Jacobs. Port Robinson
is also described incorrectly but has the correct coordinates.

The answer for Burlington has first coordinates that are "close" (well within
the city but not at centre/downtown/city hall), and the 'region' one is about
8 km off with a silly large error radius of 15 km. Ditto Kingston, Kitchener,
and Markham. Similar deal with Clarington, Clarkson, Falconbridge, "Sudbury",
and probably a bunch of others, though there's the added difficulty of Ontario
municipal organization thrown in for those ones. Bizarrely, Ottawa's point
result is perfect whereas the 'region' result is more than 20 km away with a
specified radius of 55 km, but I'm not sure if that warrants a 'wrong' rating.

Echo Bay points to the body of water rather than the settlement around it, and
I don't think it's fair to expect the search to know you meant the settlement
specifically. Jordan Station is marked "close" but is in fact 500 km off.
Wilno is marked "close" but is 10 km of forest away. The results for
Mississauga are unreasonably far away from the centre of the city but within
the limits.

Then there's search context - if you ask the service for "Baltimore Ontario"
with no further context it might get it wrong, if you ask it from a device
that knows it's at least in Ontario (or even New York or Michigan) it might
work better.

It would also be useful to somehow quantify the results by search likelihood,
popularity, community size, etc. A few names I recognize from the missing or
wrong list are fairly small settlements. Scrolling through the list (currently
on W), the largest results I saw that were flat out wrong so far are
Belleville, which is about 10 km off, North Bay about 8 km off, Whitchurch-
Stouffville about 6 km off, and
Burlington/Kingston/Kitchener/Markham/Mississauga 'region' issues as mentioned
above.

~~~
cjensen
Yep. Many of the place names in his "official" list do not have Wikipedia
entries, which cast doubt in my mind on whether the places actually exist. I
googled some and it seemed like Google doesn't think they exist either.

Also, it looks like he counted a server error as a bad map result. Maybe the
server was throttling him or the server is otherwise overloaded?

A better attempt would be to use Wikipedia as a reference for places which (a)
exist and (b) are non-trivial. Also he should cross reference against Google
since that is the current standard in mapping.

~~~
jarek
The server code 8 error appears to come up only for less significant
placenames (I don't recognize vast majority of code 8 results), lending some
credence to a theory that this error is returned for unrecognized queries.

Edit: It has occurred to me the 'region' results might be trying to
circumscribe the whole area of the municipality in a circle, which would
certainly give odd results for a lot of Ontario municipalities.

------
ChuckMcM
If this person is applying for a QA job with Apple they have the right mindset
for it. Its always great to come up with interesting ways to measure the
quality of something, I see a few problems with the methodology but hook it up
to an automated bug submission process and it could do wonders for quality
over all.

~~~
jsnell
I don't think it would. This particular method is likely to be unactionable,
due to comparing a base map data source to another.

The problem is that either Apple have rights to this data source, in which
case they should just be fusing it together with their other data. Or they
don't have rights, in which case they can't use the results of this comparison
to fix data without also risking tainting said data. I'm not a lawyer, so I
don't know where the line would be drawn. But automatically or manually filing
data bug reports based on results like these seems very dodgy.

Second, automatically generating and filing hundreds or thousands of bugs
would just render the submission channels useless. Just triaging geocoding
bugs requires ridiculous amounts of skilled human effort by people whose time
would most likely be better spent in fixing the geocoder. It might be
different for things known to be guaranteed data bugs, but then your
methodology certainly shouldn't involve a geocoder. And you'd run into the
rights issues of the previous paragraph.

There are good ways of doing automatic quality evaluation or semi-automated
quality improvement. But they're really going to be more sophisticated. And
unless Apple's expensively bought maps people are totally incompetent, they
are already doing such things.

(Edit: I don't mean to belittle the efforts of the original poster. It's good
that somebody tried to quantify the quality of Apple maps rather than relying
purely on anecdotal evidence. I just don't think that it's a method that'd
have much relevance to the way Apple works.)

~~~
ChuckMcM
I was thinking more along the lines that this person was passionate enough
about improving the quality that they were willing to code up a prototype way
of visualizing relative quality. Perhaps I've known too many engineers where
I've asked "How do you know that is ready for prime time?" and they have sort
of glazed over and said, "Uh, I don't it seems to work?"

------
fushi
I'll be needlessly pedantic and point out that actually Canada's largest
province, geographically speaking, is Quebec and not Ontario

~~~
Cbasedlifeform
LOL. Well for what it's worth, I looked up an address in Montréal and Maps had
it completely screwed up. I don't mind the beta quality of the product (Apple
has to gather the raw data somehow) but I do mind its hubris and the fanboys'
initial rave reviews of iOS 6.

------
imperialdrive
After just today reading news about the ios6 map app failing, and then this
article, I really am amazed at the deterioration of the company. Nothing too
surprising as all goods things come to an end, but still, this is fast.

~~~
owenjones
So all it takes to declare the "Most Profitable Company in the World"
deteriorating and coming to an end is some hiccups with a fairly new product?

I agree, this is fast, but not in the way you mean.

[http://money.cnn.com/2012/08/20/technology/apple-most-
valuab...](http://money.cnn.com/2012/08/20/technology/apple-most-valuable-
company/index.html)

~~~
jarek
I think "most profitable" and "deteriorating" are perfectly capable of being
orthogonal. If nothing else, the profit or its first derivative w.r.t. time
could be declining while still being the highest in the world. No opinion on
Apple.

------
terrywilcox
I'm not sure what the symbols mean, but there's a red circle next to Kingston,
ON. The Kingston info is correct. It's in the right location and Portsmouth is
part of Kingston.

The Belleville information looks mostly correct, though it places Belleville
city center a few miles north. Actual street addresses work.

And it place Picton, ON in Australia...

~~~
huxley
I wonder if was because Picton was unincorporated (I believe it happened 14
years ago)

~~~
jarek
Unlikely, the list of communities has a lot of unincorporated areas that are
formally part of a larger municipality and the results for some of them are
correct. Clarkson and Dain City off the top of my head.

------
mrng
Apple is using the cartographic data provided by TomTom (who bought TeleAtlas
a few years ago), notoriously less accurate in Canada that Navteq's (Garmin's
provider). This may be the reason (or Apple's own mapping algorithms -
couldn't tell, as I'm not using iOS).

