
AntiPolygraph.org appears to have been “de-listed” from Google on key searches - ap_org
https://antipolygraph.org/blog/2018/06/28/google-de-lists-antipolygraph-org-on-key-search-terms/
======
inglor
Well, to be fair the website looks like it does the most it can possibly do to
avoid good SEO and search engine visibility.

It uses `<meta keywords` and other techniques that apparently search engines
like Google do not take into account.

On the other hand the HTML of the page (view-
source:[https://antipolygraph.org/](https://antipolygraph.org/) ) does not
contain any guidance as to what's the page's focus. No header tags are
present, everything is `<table>` based and contains deprecated markup and
almost all of the links it contains are internal. The markup is HTML4 which
might be an indication to Google the page has not been updated in a while.

The website can easily be found in Google
([https://www.google.com/search?q=site%3Aantipolygraph.org&oq=...](https://www.google.com/search?q=site%3Aantipolygraph.org&oq=site%3Aantipolygraph.org&aqs=chrome..69i57j69i58.4647j0j4&sourceid=chrome&ie=UTF-8))
- it just appears to not be ranked very high as it aged.

Don't forget Google tends to favour newer and updated content to older
content. The website also doesn't have a sitemap as far as I know which
doesn't help with the problematic markup.

~~~
FreeFull
Unfortunately all these changes mean that it's now tougher to find old
websites with specific information that you're looking for on Google nowadays,
in general. It feels like Google used to be much more useful for that ten
years ago than it is now.

~~~
bicubic
Google's purpose seems to have changed from helping people find the most
useful information, to funelling people to information that Google wants them
to find on a particular topic, whether its the most useful or not.

Are we really still pretending like Goog/FB/etc. aren't effectively mass
information control mechanisms first and foremost?

~~~
rdlecler1
We have a blog on agrifood tech associated with our site and we’ve published
over 1,000 articles often interviewing some of the most successful
entrepreneurs, VCs, and senior execs at food and ag companies. We’re by far
the most authoritative thought leader in the industry and if you go to any
industry you’d hear our name a few dozen times but we get massively outranked
by Forbes articles, TechCrunch, CB Insights (which is a poor authority in our
space). It’s quite sad just how bad Google can still be.

~~~
throwawaymath
That sounds to me like Google is optimizing retrieval for deeper searches
within a narrower set of "authoritative" websites. The ideal solution would
involve putting in the logistical and theoretical engineering work to classify
and validate a huge number of websites good for fewer, specialized answers.

But for whatever reason, the reality seems to be a relative few websites are
selected to provide "good enough" answers on shallow queries for a huge number
of topics or objects, Wolfram Alpha style.

Maybe this is due to developer laziness, gradual dilution of cutting edge
engineering, explicit desire for simpler solutions or real practical
limitations, even at Google's scale. In any case, the result appears to be
that sites like yours are poorly surfaced because they're not very
generalizable. Whereas CB Insights claims it knows about your query already,
there's an internal API setup for exposing CB Insights already, and CB
Insights passes a basic sanity check on the answers.

This is really unforunate and I sympathize with you. I think there is a lot of
opportunity for a search engine that can optimize for very specialized queries
across many domains, including arbitrarily historical resources. But I think
such a search engine would have very different goals and incentives than
Google from the outset.

~~~
rdlecler1
People still seem to find us depending on the exact search terms you’re using
but it’s Google’s user experience that really suffers. If someone randomly
publishes “the top 25 Agtech companies” on Forbes it’ll jump to the top of the
list when searching for Agtech and stay there for half a year. Google is
embarrassing itself with these kind of shallow and superficial results.

------
dewey
When I do a search for Polygraph I get the results from Wikipedia, Vox, APA
([http://www.apa.org/research/action/polygraph.aspx](http://www.apa.org/research/action/polygraph.aspx)),
NYTimes on the first page. They are mostly critical articles about lie
detectors and they all look more like something I'd want to read than
AntiPolygraph.org that looks more like a conspiracy blog than well-researched
information. I'd say the problem lies somewhere else, not by google censoring
your content intentionally.

~~~
ap_org
I appreciate how the present blog post may strike some as conspiracy-minded.
Especially those not familiar with the history of Operation Lie Busters, a
U.S. criminal investigation that targeted the proprietors of websites that
provided information to the public about how to pass or beat a polygraph test.

As you'll see in this recent post announcing the leak of the U.S. government's
polygraph school's documentation about polygraph countermeasures, the federal
polygraph community considers AntiPolygraph.org to be a major threat, and it
features front and center in their training:

[https://antipolygraph.org/blog/2018/06/09/ncca-polygraph-
cou...](https://antipolygraph.org/blog/2018/06/09/ncca-polygraph-
countermeasure-course-files-leaked/)

None of this necessarily entails any state influence in Google's action to
delist AntiPolygraph.org on certain key search terms.

It is true that our web design is dated. But this doesn't seem to account for
the site being completely delisted from Google on such searches as "polygraph"
and "lie detector."

I am not familiar with any other instances of such partial delisting by
Google. Are you? If so, what were the causes for that?

~~~
dewey
> But this doesn't seem to account for the site being completely delisted from
> Google on such searches as "polygraph" and "lie detector."

It's not delisted, it's just not on the first page as other users also
reported and I just verified myself.

With every change of the ranking algorithm there's a bunch of sites going up
or down and nobody can tell you the exact reason why - it's easy to just call
it censorship but in reality it's probably something else. Like the top
comment already explained that your site markup has a bunch of missing
information that makes it hard for search engines to properly scrape you.

~~~
ap_org
What search term(s) did you use, and on what page of results did
AntiPolygraph.org appear?

~~~
e2e8
The google search "beat a polygraph" gives antipolygraph.org as result number
10.

The google search "pass a polygraph" gives antipolygraph.org as result number
11.

The google search "polygraph" give antipolygraph.org as result number 188.

------
rl3
Google has had terrible search quality as of late. I'm finding myself using
DuckDuckGo more and more just to find very basic things that used to be easy
with Google. This is just in the last couple months or so.

Not sure if that's the case here, but if it is I wouldn't be surprised.

~~~
obelix_
Once upon a time I could find blog posts of people reviewing things, travel,
software, equipment etc. Now I just find pages and pages of review
aggregators. The advantage of finding a blog was didn't take more than a few
clicks to figure out if the blogger and me shared thinking, lifestyles,
financial means etc. Now all that is gone. We just have ridiculous
aggregation.

~~~
dazc
The problem is exacerbated by search quality guidelines that need to be
followed if you're going to rank for any competitive search term, especially
when there are clear incentives for doing so - such as site speed and
deploying ssl.

Coming next is the 'insecure site' warning that's likely going to cull many
more hobby sites.

~~~
lawl
> Coming next is the 'insecure site' warning that's likely going to cull many
> more hobby sites.

Good. Setting up https shouldn't take more than an hour now with LetsEncrypt
even if you have no clue about the process.

I can totally understand outdated webdesign, can't understand not having
https.

~~~
ozfive
I can, because I just converted a site to https, which had a combination of
relative and absolute href and src parameters on it in a confused manner. This
was combined with html embedded in table records.

It wasn't a simple search and replace task since several admin functionalities
were also included in the site.

Think about all of the sites that have relevant information that aren't
selling anything that would love to get things like this fixed, but just can't
throw money at the problem.

Before you say oh it shouldn't cost that much. Think about where some of these
sites are hosted and for how little. Some people don't even have the means to
pay for such a fix, but they have important information to share.

This probably should have been a rule applied to sites that sell things and
not sites that just have information.

I am all for making money, but I also feel like doing some charity work for
this problem as well. I heard the deadline is the 24th. Anyone else interested
in doing this collectively for a couple sites?

~~~
lawl
> I can, because I just converted a site to https, which had a combination of
> relative and absolute href and src parameters on it in a confused manner.
> This was combined with html embedded in table records.

Just add an upgrade-insecure-requests header to your webserver config, boom.
No search and replacing needed.

[https://www.w3.org/TR/upgrade-insecure-
requests/](https://www.w3.org/TR/upgrade-insecure-requests/)

It's been added for _exactly_ your usecase:

> Most notably, mixed content checking [MIX] has the potential to cause real
> headache for administrators tasked with moving substantial amounts of legacy
> content onto HTTPS. In particular, going through old content and rewriting
> resource URLs manually is a huge undertaking.

I mean, uh sure, I'll volunteer to move your sites to https, but I don't think
giving a random dude on the internet root access to fix the webserver config
is a good idea ;-)

~~~
ozfive
This site also had mixed content due to it using a forward proxy with ARR in
IIS. Since ARR doesn't forward https requests it is truly turning into a mess.
There isn't an option to just move it to another webserver as that would be
it's own undertaking with the dynamic part of the site being ColdFusion.

> I mean, uh sure, I'll volunteer to move your sites to https, but I don't
> think giving a random dude on the internet root access to fix the webserver
> config is a good idea ;-)

In this point yeah I agree you don't want to let just random dudes have root
access to a site. On the other hand I run my own legitimate consulting
business and if you think about it. Every time I am winning over a new client
for all intents and purposes I am just a random dude. :)

~~~
lawl
You can always just put an nginx in front of it to terminate TLS there. Sounds
like a legacy mess nobody intends to maintain anymore anyways. Or hell,
cloudflare them. Kinda pointless since you won't have TLS to the backend
server, but the easiest "solution". I maintain that's it's possible to inject
a header and terminate TLS however messy the system is within an hour.

------
bradleyjg
I vouched for this story when it was flagged dead and I wanted to note why I
did.

I think it's more likely than not that the "innocent" explanation is correct
here. But even if that's the case it still says something important about how
google works and those things are being discussed in the comments.

Further even if the article is wrong, I don't see how it violates any of the
guidelines. I think there's a tendency at HN to be very protective of google.
I'm not sure if that's external fans or employees or what. There's nothing
wrong with that per se, but I think it should be primarily expressed in
comments rather than flagging or downvotes.

~~~
DTND
>I think there's a tendency at HN to be very protective of google

There's a tendency to be very protective of a wretched business model more
than there is of google in itself. It may seem people come to google's defense
a little too much on HN but that's only because google is the largest, most
important corporation at the front of the business model that matters most to
hners: advertisement and abusing user data. How else could any of the funded
startups that provide free online services survive if they didn't get their
money back through some other means?

There is a vested interest on HN and other similar places on the web to show
solidarity toward any individual or corporation whose livelihood depends on
this because the second you start questioning companies like Google and
Facebook, you start questioning the very livelihood of everyone involved here.

~~~
extralego
One of the first lessons activists (should) learn is the shortsightedness of
judging the content of an individual according to what they do for a living;
the means of survival they have trained and adjusted for. In an action,
strategy will often overrule this but as a conscious exercised and ideally
organized process. This along with overlooking petty symbolic straw mans like
using cell phones to organize against cell phone companies is like learning to
potty train; critical first steps toward maturity.

In other words, evaluate components in proper and sufficient contexts.
Software engineers might benefit from a similar conversation, to open more
room for discussion around actual problem solving.

------
vignesh_m
I saw the site on page 3 after googling for "polygraph"

Although searching for "the lie behind the lie detector" gives the first
result as the correct one from anitpolygraph.

Is it possible google didn't really do anything shady?

------
reilly3000
This is a complete falsehood. There is no mention of "Operation Lie Busters"
or any kind of program to suppress information about lie detection
circumvention in the linked document. No other sources were given.

The site has plenty of reasons to not rank well in search, and secret
government control over search rankings is at the bottom of that list in my
view.

------
dazc
Yet when I search for 'anti polygraph' [in uk] guess which site ranks top?
Like I would expect.

The site title is 'Learn How to Pass a Polygraph Test' yet the term doesn't
appear anywhere on the homepage?

I'd agree with the general sentiment that Google search is getting worse but
this isn't a great example.

~~~
asah
Adding to this:

\- search for [anti inflammatory] shows the site as first result and it's on
page two for [polygraph controversy]. At the least, the blog post is
misleading.

\- as per other comments, the site isn't mobile friendly, which has a huge
impact on mobile ranking.

\- searching for [polygraph] the first page includes several listings
discussing the controversy around polygraph results. The results are
reasonably high quality stuff, i.e. valid choice.

\- a more valid test search might be [polygraph accuracy] or [polygraph
controversy] to focus on the debate and less on testing centers and basic
information.

Disclosure: former Google engineer and worked at 3 search engine companies on
various search products. I've done SEO professionally outside of Google, with
success. I've also seen smart people botch SEO, myself included.

------
jmnicolas
I would never have heard about this website if it hadn't been "censored" by
Google.

We can be proud to live in democracies, now I know why they envy us in the non
free world! /s

~~~
arendtio
[https://en.wikipedia.org/wiki/Streisand_effect](https://en.wikipedia.org/wiki/Streisand_effect)
;-)

------
erric
The Streisand Effect is real and hopefully for AntiPolygraph this just gives
them a wider audience.

~~~
DarkCrusader2
Maybe Google is not trying to censor them, but just changed their algorithms
to not show antipolygraph when searching for polygraph as said by someone in
the google forum. Their site #1 result when searching "antipolygraph".

------
PSZD
Is it appropriate to rank extremely high for something you're directly opposed
to? Consider other polarized topics like abortion - should sites enlightening
the user on why it is murder, instead of providing useful information, be
presented front and center? I'm fairly sure this has come up as a controversy
recently, but can't think of the exact topic.

Certainly Google could decide whether it is appropriate on a case-by-case
basis, but at that point it starts to be very explicitly political. Once
you've stripped away the fig leaf of neutrality, their monopoly would look
outrageous to a significant portion of the population.

~~~
mmt
> Is it appropriate to rank extremely high for something you're directly
> opposed to?

I'd say that's an important question, which I would answer with "yes", so long
as it's understood that "rank extremely high" doesn't imply "at the exclusion
of all others".

> should sites enlightening the user on why it is murder, instead of providing
> useful information, be presented front and center

Possibly. Though, I fear you may have presented a strawman (while betraying
which way you, personally, are "polarized" on abortion).

The question of what is _useful_ information depends entirely on the keywords
the user entered. Just the single word "abortion" could easily, reasonably, be
understood to mean a request for information of general interest on the topic,
including moral implications. On the other hand, it would be a stretch to
expect that someone would want, say, instructions on how better to administer
sedation during the procedure, absent additional keywords.

Of course, there are going to be searches like "get an abortion" that are far
more ambiguous than "get an abortion in my area", but I think it's better for
something like a search engine (or government or society for that matter) to
err on the side of inclusion than suppression.

> Certainly Google could decide whether it is appropriate on a case-by-case
> basis, but at that point it starts to be very explicitly political

Indeed, and that's a can of worms they likely don't want or need to open.

~~~
PSZD
> while betraying which way you, personally, are "polarized" on abortion

I'm curious what side you think I fall on, seems to me it could be argued
either way. I have no strong opinion on it and picked it purely because it's
an evocative example. Perhaps too evocative!

Some other examples might be terrorism, animal rights activism, transgender-
as-mental-illness. Including both sides can be legitimately and understandably
very upsetting for some people; in other cases we demand equal representation.

Are situations like OP's an inevitable product of the tyranny of an algorithm,
one that errs to the safer side of excluding results?

~~~
mmt
> I'm curious what side you think I fall on

That abortion is not murder, as you made the example of the presentation of
such an argument in opposition to the presentation of something "useful". I
realize, of course, that the word could have been meant more narrowly (such as
"strictly pragmatic"), but that tends be less likely, even on here.

> legitimately and understandably very upsetting for some people; in other
> cases we demand equal representation. [...] > errs to the safer side of
> excluding results?

That's the the thing, though, I don't believe in a "right not to be offended"
or a "right not to be upset" (legitimately, understandably, or otherwise). As
such, I don't believe that these divisive cases ought to be treated any
differently than the other cases you mention.

None of this is to say that controversial topics don't attract a distinct
problem. They do: trolls (for various definitions of the term). However, I
believe that's more an issue with the quality of the content rather than the
content itself, since a troll could (sometimes just as easily) take either
side.

~~~
PSZD
I do agree with this, but think there's enough societal pressure these days
that it's very hard for a for-profit company to do.

~~~
mmt
I'm not sure being for-profit has that much to do with societal pressure as
being ad-revenue-driven.

However, this profit (or revenue) above all else model is something Google's
unusual stock voting structure was supposed to address.

------
astura
1) There's no such thing as "the first page of results on Google." Your
results depend on your location, previous searches, etc.

2) This site seems to think this is some mass conspiracy to remove sites
critical of polygraphs, yet when I Google "polygraph" the vast majority of
results I get are critical pieces. Maybe those sites are just as relevant or
better? A lot are stories in major publications, so that doesn't surprise me
they rank high.

------
ap_org
On 29 June 2018, we posted an question about this on Google's Webmaster
Central Help Forum:

[https://productforums.google.com/forum/?hl=en#!topic/webmast...](https://productforums.google.com/forum/?hl=en#!topic/webmasters/YO0jYLfb-
VM;context-place=forum/webmasters)

~~~
JackCh
Are you aware that your website cannot be accessed from many Tor exit nodes
without solving a google captcha?

~~~
ap_org
Yes, and we very much regret that. We're on a shared server, and our
webhosting company recently added security software called Imunify360 that
does this.

------
peter_retief
I cant load the site at all?

