In particular I've noticed a steady stream of woke articles being pushed my way despite me having little interest in that sort of thing.
These things are supposed to be fed by our preferences, but while I'm happy to read across the board brow-wise, most of my reading is of the deeper articles and yet genuinely intellectual essays are rarely offered up.
It's frustrating as a reader, and interesting to hear about it from the view of the writer.
Recently I've been reading well-cited/exciting academic studies for edutainment. Since you want more intellectual content you might also enjoy doing the same. Of course, I skip most of the methodology section.
The Google Scholar widget has a new feature where you can save articles for later perusal. You can also subscribe to alerts with new academic studies on your topic of interest.
Youtube can also be way more intellectual than most people appreciate. Because of Audible, Youtube, HN, Marginal Revolution, and Google Scholar, I no longer have the patience for most articles on current events. I just keep abreast of the major developments. I definitely ignore most local news (shudder). I'm usually surprised when it rains out. Who cares.
I know what you mean. After all the tracking data collected and ML models trained the primary metric for suggesting content still seems to be "stuff other people are reading/watching/playing/listening"
As someone that used to work on recommendations systems ... the problem is that the signal is just so robust. Stuff other people like really is one of the stronger ways to predict what you will like. Most additional data sources that allow us to customize further and identify what sub-groups you might belong to and thus prioritize highlighting things of relevance to your subgroups tend to empirically show lower performance for the overall population. The only thing I've seen work is focus on the "most popular" things for the most important demographics (such as those with money to buy things) which might overfit for that demo, and underperform overall, but still performs the best on some higher metric like revenue.
I commented a while back on someone asking "Why recommend me a ladder if I just bought a ladder, couldn't the system understand that people usually only buy one ladder?" But the fact is, the commenter was wrong about his priors. Actually someone who buys a ladder is indeed more likely to buy another one ... even if most people that buy ladders only buy a single ladder. The fact that you bought a ladder, puts you in the cohort of people that buys ladders, which means you are more likely to buy another one than someone who isn't on the record buying a ladder.
> the problem is that the signal is just so robust.
That's why you aren't seeing the better solution. You're only seeing what you know, which is the (tired old) signal you know how to detect. But you already detected it, and monetised it.
A better recommendation signal would be the one that you don't know how to detect. It's the signal people get unexpectedly from other people, when they chat about the thing they just bought. Call it the n+1 recommendation signal.
> The fact that you bought a ladder, puts you in the cohort of people that buys ladders, which means you are more likely to buy another one than someone who isn't on the record buying a ladder.
OK, but that's still the n recommendation signal, which has already been monetised, and won't be refreshed for quite some time, in the same way a Venus fly trap takes time before it will open and trigger again.
What can a person do with a ladder that she couldn't do before?
That question is what begets the n+1 recommendation signal. It isn't a question that even gets asked explicitly. It happens through association.
She says "I got a new ladder, and a paintbrush and some paint." and her friend says, "Oh great, I just used my ladder to hang a bird feeder from the sycamore tree!" and then she says "Oh! I've got a sycamore tree too! Where'd you get your bird feeder?"
> But the fact is, the commenter was wrong about his priors. Actually someone who buys a ladder is indeed more likely to buy another one ... even if most people that buy ladders only buy a single ladder. The fact that you bought a ladder, puts you in the cohort of people that buys ladders, which means you are more likely to buy another one than someone who isn't on the record buying a ladder.
Be that as it may it’s still ridiculous. There are very many other things I am going to buy before I buy another ladder. And Amazon should be able to know that because of having seen me search for very many other things in the past that I have not yet bought.
And if nothing else, couldn’t they let me at least click a little button next to any suggestions to tell their algorithm that no I am not interested in this recommendation?
That's because the optimization target isn't "what will let you grow best as an individual human being", but "what will make most money for the parent company". Which translates to what's most "engaging". Which translates to what's most dividing and most outrage-inducing.
That is what you get, and I thought in 2020, this would be an accepted wisdom: if you optimize for what gets the most engagement from people, you end up with things that cater to our most primal urges. It's a statistical phenomenon; you're more likely to ignore a nugget of deep knowledge than a headline which makes your eyes red before you even finish reading it.
I wonder what would it even take for the first "what will let you grow best as an individual human being" algorithmically even possible to judge? The spurious correlations alone from bad attempts to quantify it would be amusing.
For starters, I'd give people a honest and functioning search and filtering interface. Like: "Give me videos on topic X, order by date, then viewcount, filter out the blacklist". Let people save such queries for quick access in the UI.
Give the ability to provide negative feedback. One-click "blacklist this video/channel". One-click "don't show me things like this". One-click "show me less of these". In a way that actually works.
Allow to explicitly tune the recommendation algorithm. Like, modify the list of categories that get prioritized, and even their relative priority.
All that boils down to: give people autonomy to curate their own experience to match their own goals, instead of letting an algorithm do it in the background to optimize for company's goals.
As for "the spurious correlations from bad attempts to quantify it", yeah, I'd love to read about them too. But perhaps not on the front page of the news, in context of how it hurts people.
I think the pushing may also be an "editorial stance" essentially or reflective of the source production as they feel an obligation to do so let alone if the zeitgeist makes it a "generic winner" like clickbait.
Is it really surprising though? Woke culture is a predominantly west coast product and coincidentally that's where a lot of influential software offices are situated. I think what's more surprising is that people in our industry still pretend like American software practices are politcally agnostic.
As someone who worked at a company that pushed many stories a day to hundreds of millions there is an absolute bias towards “woke culture” and left. It’s evident here on HN sometimes, too.
I quite enjoyed this article and found the author's perspective on the inherent internationalism of art resonate with me.
However, I'm not sure that the issue he is discussing is mostly caused by algorithms. It seems to me that the internationalisation of newspapers and media organisations is the main culprit. As the author says, if you can read art reviews in the NYT, why bother with 'small time' critics? Personally, I share the author's view that having a local perspective is still valuable but it seems clear that a lot of people don't.
I saw this essay in Arts and Letters Daily (https://www.aldaily.com/) and added it to my blog's next links post, with a variation of this comment: I've never heard of this guy and yet his work sounds like just the sort of thing I'd like to read: I'm not interested in most of the standard political and pop culture stuff that's endlessly written and re-written. Unfortunately, he doesn't seem to have a link list of his recent works anywhere, at least that I can find. His website (https://russellsmith.ca/) appears to be pretty generic, and its RSS feed seems to have last been updated in 2015. How are we supposed to find his work and follow him, if we are interested in his work (and I am, after reading this)?
Can recommend highly, particularly his books, "Confidence" and "How Insensitive." I read his column in the Globe for years.
When you compare what he writes to the content that gets pushed to us like in that Social Dilemma documentary in another thread (https://news.ycombinator.com/item?id=24468533), smart people are writing, they're just getting drowned out. Content algorithms are like rating food on how efficiently it can deliver sugar and caffeine with no other criteria for "good."
Ultimately I think what this author is describing is actually more of a tale of one employer‘s crummy management style.
Interesting niche content like the author writes is exploding. In contrast, it sounds to me like this particular outlet is missing out on the opportunity to provide niche and offbeat content by bean counting.
In other words, I’d hesitate
to assume that all digital media outlets are blindly chasing “engagement” without making any other considerations. I would think that big legacy media companies are easily led off a cliff by “doing what everyone else is doing.”
We are in the infancy of digital media, as much as it seems like it’s been around for a long time at this point. Many outlets, especially traditional ones, are ultimately still doing “the wrong things” that digital media firms in 2050 will look back on as being primitive and crude.
One last point: while the “flip through the pages of the paper and discover odd topics on art and culture” is dead, that doesn’t mean the interest and readership is dead. What’s dead is the idea that institutions like The Times or The Globe have any relevance in acting as aggregators of that content.
Reading curated feeds WILL expose you to mass manipulation engines. The power they steal from you and others can then be rented out in bulk for a premium.
That’s why most companies don’t let you turn them off (or make it difficult to opt out.)
Most of the newspapers in Germany have half of their articles as 'premium content', and these cost some money. I wonder if they focus on the same metrics with this model. (I mean if the newspaper has a revenue model that is not primarily based on ads then its metrics might be of a different kind)
In the case where an article causes you to subscribe, the topic area can be sustained by a smaller audience.
Eg., this is how netflix maintains a large library of niche films. It is enough just that they profit on them, not that they are popular.
The issue in OP appears to be that his articles were always included with the printed newspaper, and some people were curious enough to read it. However those people would never think to seek out that content.
So it's a question of "how do you serve the intellectual curiosity" in an era where mere popularity is the main recommendation system?
I don't think it's as big of a problem as he seems to think, only, that newspapers now have no answer to it. Many institutions for the intellectually curious (newspapers, universities, etc.) don't serve that function anymore.
One thing a news recommender app could do is randomly insert a reference to an article that is not based on popularity. Sort of "giving it a chance" like a radio DJ used to do to introduce new music from an unknown artist.
The algorithms are motivated by ad impressions. Solution: seek out newspapers that are funded by subscriptions, not ads, the ones with stiffest paywalls. These would be Financial Times and WSJ. Chomsky called the FT "the only newspaper that tells the truth" because the readership and authors are aligned in their goal to make money from the news.
As a long-time FT subscriber (over a decade, with my own money), I believe this is no longer true. They have become very woke over the past few years and are extremely selective in their political reporting and the slant they put on things. It’s no longer possible to pay only for the FT and end up fully and accurately informed, except in pure markets or finance topics. For anything political you may as well be reading the Guardian, which is free.
Yes Nassim Taleb said the same. When you want to read unbiased news, stick to financial newspapers, because business people don't want somebody's coloured opinion but simply the facts.
The Financial Times is absolutely biased on certain topics, for example US politics where they are extremely anti-Trump. It’s not that I want them to be pro-Trump, I’d just like to be accurately informed about him, given he’s one of the most powerful people on Earth and what he does moves markets. I don’t read the WSJ, but some people say the reverse bias is evident there.
Assuming there should be "no bias" for fiancial geared newspapers is a mistake. Their bias should be pro-operable facts and those who work on the implications which may be followed and considered even if wrong to know the current Zeitgeist. That might mean saying "Brexit would be an economic disaster for the nation" no matter how impolite it may be.
I agree, but the problem with the FT is that they’ve stopped capturing the whole zeitgeist on some topics. Telling half the story all the time leads to the sorts of surprises that financial market participants really don’t like. It’s a very strange strategy for the business they’re in. They did get sold a while back, and their editors have changed, so perhaps their subscription model has been struggling and they’re pivoting to the clickbait model.
I personally never read articles of FT nor WSJ - I don't even try to "click" on those links/references as I know that I won't be able to read them => I'll never have any chance to "understand" if I would like to subscribe to those newspapers :(
Quite a few FT articles are free to read, and many others paywalled on the site are often syndicated to other, non-paywalled sites. A DDG search on the headline will often return such results. (I try, when submitting an FT article, to find a syndicated version as well and to post a link to it in comments immediately after submitting.)
Alternatively, and depending on your views about copyright etc, you can submit the article's url to archive.is - which will either return anything already "archived", or try to fetch it for you.
Thanks :) I didn't know "archive.is" => interesting/nice!
Still, I think that in the case of well-known newspapers I probably won't try to work around their policies (difficult to explain the reason as I'm not sure myself about the "why" - maybe a mix of ethics and other stuff specific to newspapers).
Wokeist trends usually originate in the WaPo, percolate through the NYT and eventually end up in the WSJ, all happening predictably with a time lag of 2-5 years
Interesting statistics! But I am not quite convinced use of various new terms necessarily conforms with advocacy.
All news outfits are biased, including the WSJ, but some try a lot harder than others to be unbiased, and some have given up all pretense of being news.
In particular I've noticed a steady stream of woke articles being pushed my way despite me having little interest in that sort of thing.
These things are supposed to be fed by our preferences, but while I'm happy to read across the board brow-wise, most of my reading is of the deeper articles and yet genuinely intellectual essays are rarely offered up.
It's frustrating as a reader, and interesting to hear about it from the view of the writer.