> People seem to like it — we are seeing 5,000 hits to our API a minute to serve those cards that show when you hover over any link.
Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews.
> The original idea was conceived four years ago
When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the same thing, except I'm not sure if it included an image. (Mediawiki allows a user to define custom CSS and JS that get embedded in every page of their user session.)
I don't mean to express dislike or downplay the hard work that went into this feature, just to add some context.
> Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews.
When you evaluate features using engagement metrics, there are only two possibilities. If the metric is high, users love the feature. If the metric is low, users don't know the feature exists, and more alerts or "unread" badges must be added to help them learn.
The best way to run Windows Server is without a desktop installed, using PowerShell to manage it. Even one-offs. You might know this but it’s worth saying just in case. Also worth noting that some server applications assume/require a desktop during install or operation (usually because they’re trash).
But when the feature only needs you to hover over a link to activate (which people do anyway) you have no basis to claim that this is users "liking" the feature - it might just be noise from everyday regular navigation.
i was using wikipedia to with my son yesterday for a project and found it really annoying, so i'm in that list of people who hit the api but aren't in the list of people who seem to be "liking it"
> When you evaluate features using engagement metrics
No-no-no. Stop right there. "Engagement metrics" are the worst kind of metrics. Engagement means next to nothing. It just means that a user interacted with something. Was it good? Was it bad? Was it intentional? Was it by mistake? "Engagement metrics" answer none of that.
And yet... They are the easiest metrics to collect and too many companies use them as if they were meaningful.
I sincerely regret the use of that statement "people seem to like it" in my post. I've now removed it as I worry this confuses my message so thank you for pointing it out. This really trivializes all the work that went into A/B testing this and how we measured success. I really recommend a read of https://www.mediawiki.org/wiki/Page_Previews#Success_Metrics...
Side note: the volume of traffic was also wrong by several magnitudes... actually 0.5 million)
My main motivation when writing this post was to share how small changes require magnitudes of effort not to express the merits of the feature. As a developer who works with product owners a lot and often get asked how I can build things quicker (I'm sure many can relate). I wanted to provide something useful to other developers that easy looking things are not actually easy to build, so thanks for flagging.
With regards to the user script, yeh that's been around for some time and was the seed for this idea. It's just taken a long time to get that from such a small audience to a mainstream one. It doesn't downplay it in my opinion, just shows how far we've come.
I do mean to express dislike, as I'm one of the millions of users giving Wikipedia a false metric and sense of satisfaction. I've seen the pop-up hundreds of times, and have considered it a hindrance to my process every single time.
Something that just happens accidentally is a bug, no matter how useful it might be if it were not happening accidentally.
When you go from reading the Atlantic, The Economist, The Washington Post... any content site... then begin to read a Wikipedia entry everything you know about reading an article no longer applies. You are now forced to change what you do.
This is just the most basic no no in web design - unplanned interactions, forcing a user to interact, forcing a user to actually think about the interface.
The developers have just forced every person who wants to consume content on Wikipedia to do it differently than on every other content site.
And there are literally countless millions of people who will have no idea how to disable it, who use computers every day but have no idea how to change something.
I personally do not care for the link preview feature as I am one of those people who like to hover over a link so I can see the url leads (which is also why I detest URL shorteners).
I also have to wonder how much bandwidth Wikipedia is going to end up using to display unneeded previews.
> I personally do not care for the link preview feature as I am one of those people who like to hover over a link so I can see the url leads
But that's exactly what these preview popups are for! A bit heavier than the plain URL, but much more useful and friendlier.
(EDIT: I see there's a bug in that it displays the preview even if you don't stop the mouse there. Still, this can be lighter than requiring people to click through to the full articles, considering the preview can tell you what you need or that the link is not relevant for you, and slows you going down the rabbit hole to more links. Maybe it should be located at the bottom of the screen like the plain URL popup though?)
> I also have to wonder how much bandwidth Wikipedia is going to end up using to display unneeded previews.
This is interesting. There's an external arrow for external links, so I'm still good with this use case for inspecting links; for internal links I've found myself hovering where I previously clicked (and loaded) an additional page at least 50% of the time. Where I'm OK with a quick paragraph description and can then decide if the linked paragraph requires / inspires a deeper read, about 50% of the time.
I think this is a net reduction of bandwidth for my personal use-case. Wide A/B testing would reveal more, obviously.
That's less than one API hit per 17 English pages viewed - the overwhelming majority of the time, users do not use this feature at all, based on these numbers.
The sample size for those graphs is 1% so you need to multiple that by 100. I personally checked the access logs to check they are consistent. That was where the confusion came from!
That was also my first reaction, but I did not mention it because judging this (in relation to pageviews) would require more knowledge about the audience, e.g. how many users are using a mouse at all (vs. a touch screen, or pure keyboard navigation).
but how many of those views are on mobile devices?
this is a desktop feature. in general, mobile traffic > desktop traffic.
we’d need wikipedia’s browsing statistics be sure, since some sites see different ratios than others. What if 1:8 wikipedia visits are on desktop? That would make the 5K/minute much more significant.
"5000 hits to our API per minute" does not say anything about whether they're measuring in front of or behind the cache. For all intents and purposes, the cache's endpoint is the API endpoint.
And uncached hits only would be an even more worthless metric.
I, too, think either that number is wrong or this wasn't a feature available to all users.
I also agree that using this metric as affirmation the feature is liked is dangerous; count me among the many here inadvertantly contributing without enjoying the feature.
> To ensure that these rates were not artificially low due to usability issues, we also confirmed in a separate qualitative test that users were indeed able to find and operate the disable functionality if they desired to disable Page previews.
(From their AB article)
I just tested it and it's trivial to click. Are you actually having difficulty or just pretending it's hard so you can be condescending?
I think the best place for these controls are usually on the feature itself. I don't want to have to search in some independent preferences UI every I want to see if something can be tweaked, but it makes sense to offer it there as well.
“We thought the floating pop-up billboard showing you a preview of what’s inside when you glance at a building was a great feature as you walk down times square”
As far as I know the Lupin user-script was the foundation of the PagePreview feature. I guess the Reading group at WikiMedia rewrote the extension as part of MediaWiki core and tried to fix most edge cases on the way.
Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews.
> The original idea was conceived four years ago
When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the same thing, except I'm not sure if it included an image. (Mediawiki allows a user to define custom CSS and JS that get embedded in every page of their user session.)
I don't mean to express dislike or downplay the hard work that went into this feature, just to add some context.