
The mysterious case of missing URLs and Google's AMP - mike22223333
https://sonniesedge.co.uk/posts/amp-urls/
======
SkyPuncher
Yes, Google owns a significant portion of search but I don't think it
qualifies as a monopoly.

* They have no exclusive rights to "search". Bing, DDG, Yahoo, and more all compete in the same space.

* They do not have a geographic right to any areas.

* They are not prevent me from using another search engine

I choose to use Google because it gives me the results I expect. Nobody is
forcing me to use Google over other search engines. If anything, Google has
raised the quality for all search engines by consistently producing the best
product available.

~~~
ChuckMcM
This needs to be repeated because it helps to understand the issue.

There are only four (4) search indexes of sufficient size in English, these
are Bing, Google, Baidu (Chinese), and Yandex (Russian).

 _Everything_ on the internet that calls itself a "search engine" gets its
search result data from one of those four indexes. Different front ends
provide different features around the results, whether it is privacy
enhancement, ad enhancement, Etc.

Google's English language index is generally considered the 'best' based on
the depth of the index with respect to its crawl size and that index's ability
to provide both good precision and recall.

You can use any search site you want but you will likely be using Google
search results. If you feel like you're results aren't quite as good as
Google's, and you are in the US, you are probably using Bing. Neither Baidu
nor Yandex have US data centers so latency to their indexes is pretty bad for
most front end web sites.

~~~
h2onock
This is incorrect, not _everything_ on the internet that calls itself a search
engine get their results from one of those four. There's also Exalead,
Gigablast, Mojeek and Findx for example.

Disclaimer: I work for Mojeek so know that it only gets results from it's own
index.

~~~
ChuckMcM
That is _awesome_ love that you've crossed a billion pages in the index. I
worked at Blekko (VP Eng/Ops) and at its peak it had about 5 billion pages
indexed, and that is pretty good for targeted search. It loses on the long
tail though for generalized search.

Something we did at Blekko which was really interesting/helpful was to feed a
search query to our index, Bing's, and Google's and then compare the results.
That was a great way to compare notes on ranking strategies between the three
engineering teams.

[1] _" In 2015, Mojeek's index surpassed the billion pages, the only search
engine in the UK to accomplish this and one of just a handful worldwide to
maintain an index of this size (English language). Although, passing this
milestone is only the first step and Mojeek is continuing to increase its
index size and improve relevancy."_ \--
[https://www.mojeek.com/about/technology.html](https://www.mojeek.com/about/technology.html)

~~~
h2onock
Thanks for the info. Marc (who created Mojeek) mentions Blekko a lot as he
considers yourselves to be the last ones so far to have a proper go at
creating a search engine from scratch. Also, we're now up to 2.2 billion pages
and it's only a lack of servers that stops us from increasing at a much faster
pace.

~~~
ChuckMcM
For what its worth, this is the ultimate calculus for a search engine; The
revenue you can produce is proportional to your query rate, your query rate is
proportional to your recurring user base, and your recurring user base is
proportional to the usefulness of your searches, which is proportional to the
number of pages in your index which is proportional to the number of servers
you have running which is proportional to the cost of running the service.

One of the things I would have done differently at Blekko is that I would have
created our own in house ad serving system (network) way earlier in the life
cycle (Blekko never did have an in house ad network). It gives you better
control over advertising latency and the quality factor of the ads. You also
get to keep all of the ad revenue rather than just some of it. And you can
deal directly with advertisers rather than the ad-tech crowd which can be
quite scummy at times.

I wish you the best of luck in your endeavor, if you can crack the secret of
surviving in a market where Google pays people over $4B/year just to send
their search traffic to them, then you will be one of my heroes.

------
skybrian
The first theory that some designers think URL's are bad for users seems more
plausible. Most people don't think of themselves as the villains in a movie.
What kind of conversations do you think Googlers have?

We've seen this before with Gmail hiding email addresses. You can still show
them with one click. These redesigns are annoying but world didn't come to an
end.

~~~
gowld
Hiding email addresses is really annoying, and a security hole because email
users can lie about their name.

Look at the problem with the telephone system where scammers can spoof caller
ID.

~~~
kyle-rb
People can already spoof email addresses pretty easily though.

~~~
skybrian
You can do it, but with SPIF and DKIM checking, it might be hard to avoid
detection, at least for some domains. How good is Gmail at detecting this?

~~~
Boulth
Good if you set up DMARC. Quarantine setting guarantee that fake email goes to
spam. Reject setting drops the fake email altogether.

------
jefftk
_> The hiding of URLs fits perfectly with AMPs preferred method of making
sites fast, which is to host them directly on Google’s servers, and to serve
them from a Google domain. Hiding the URL from the user then makes a Google
AMP site indistinguishable from an ordinary site._

Hiding URLs isn't necessary for sites hosted on the AMP cache to look just
like other sites; Web Packaging will allow this while preserving the URL:
[https://www.ampproject.org/latest/blog/a-first-look-at-
using...](https://www.ampproject.org/latest/blog/a-first-look-at-using-web-
packaging-to-improve-amp-urls/)

(Disclaimer: I work at Google)

~~~
wmf
Enabling Web Packaging and _then_ hiding URLs will produce a lot less blowback
than doing it in the reverse order.

~~~
jefftk
I was trying to explain why I don't think the people working on how URLs are
being displayed in Chrome are doing it for the benefit of AMP.

------
dagenix
> If this sounds all conspiracy theory, then good for you - you’ve just earned
> the Junior Meta Sceptic Badge.

At least the author seems to be aware that the article isn't anything more
than a conspiracy theory. What is not clear, is why they then published it.

There are plenty of things to complain about with AMP. But, it also does
address some very real problems - slow, inneficient, hard ti use web pages.
What I'd love to hear about is a different approach to address these issues -
and not something handwavy about how it could be done in a weekend. Actual
code.

~~~
ricardobeat
You don’t need a different approach. It’s absolutely possible to build fast
websites today without AMP - it’s just a collection of good practices and
strict limits.

In fact, you can even use AMP itself without any of the Google garden parts.

~~~
dagenix
So, your alternative is to do nothing. That's a valid alternative.

My experience, however, is that the web is clogged up with slow, painful to
use web pages. Despite AMP's short comings, I do know that when I click on an
AMP page, it's going to load quickly and behave consistently. IMO, those are
pretty big benefits it would be good to bring to a larger portion of the web,
regardless of it's via AMP or some other mechanism.

~~~
ricardobeat
Google has already made speed a stronger part of their ranking system, and
they could do more; when it comes to performance, they can prioritize fast and
bloat-free websites regardless of AMP - I see it being more about controlling
the platform, and the advertising revenue that comes with it than an
improvement in tech.

------
robotkdick
At first glance, AMP will make life easier for Google's search robots, but the
standard would also seem to benefit a competitor like Duck, Duck, Go by
standardizing code organization, making it easier to find the true 'intention'
of a website's pages. Is AMP contrived to benefit Google? For sure, but not
sure it's a terrible idea.

------
saudioger
They're paralleling the days of AOL in a lot of ways.

------
HIPisTheAnswer
Doesn't bother me. IPFS makes AMP obsolete.

------
pixl97
If you joined Google to make the world a better place, it's time to look for
greener pastures.

~~~
dmitrygr
Sure, please point to some that pay equally well and offer equally interesting
coworkers and tasks.

~~~
908087
Those sound more like things that would be important to someone who joined
Google because it paid well and offered "interesting" coworkers and tasks, not
someone who (naively) joined Google to "make the world a better place".

~~~
dmitrygr
If you _really_ do not care about being well paid and want to change the
world, Red Cross is hiring.

~~~
Nursie
They are.

However there is a sliding scale here, and a lot of people will willingly take
somewhat lower pay in order to work for places they consider reasonably
ethical and transformative.

Pay is not everything, but neither is it nothing.

(My partner works for the Red Cross in the UK, they are a great organisation)

------
iDemonix
This article, and the one it references about AMP, all throw the word 'force'
around a lot. No one is 'forcing' you to do anything with your website,
wanting to be top of Google SERP is optional.

~~~
justinph
I work for a large publisher. Google isn't putting a literal knife to anyones
throat, but they are not so subtly suggesting that search traffic is tied to
your usage of AMP. That qualifies as `force` in my book.

~~~
skybrian
But this is the opposite of force.

~~~
reitanqild
A threat can be a way to force.

Saying a threat is the opposite of forcing is definitely wrong.

~~~
skybrian
The original post talked about "a literal knife to anyones throat" which means
physical force.

But what is supposed threat about? We are talking about whether or not Google
links to a website, which has nothing to do with that. Linking is purely
voluntary and there is not even a contract.

Of course, at scale, financial incentives are still very important for
businesses, but let's be clear about this. There are no goons or threat of
goons involved.

~~~
reitanqild
> The original post talked about "a literal knife to anyones throat" which
> means physical force.

Here is exactly what _you replied to_ :

> Google isn't putting a literal knife to anyones throat, but they are not so
> subtly suggesting that search traffic is tied to your usage of AMP.

1\. Notice the use of isn't. I'm not a native English speaker but it still
seems clear that you are reading something that isn't written.

2\. The threat of being shunned is still a threat. Also there is something
about abusive relationships where the abuser will say: I didn't force anyone,
but everyone knows there would have been "consequences" if the victim didn't
do what they were told.

~~~
skybrian
You're implying physical force by comparing to an abuser situation where an
implicit threat of physical force is involved. If you don't want to imply
violence, don't use a violent metaphor.

