There will be some bias in what they choose to screenshot right? Meaning, the unrelated topic might show up in the feed but they don’t screenshot it because it doesn’t fit the narrative?
Also, what we’re interested is if the text changed what was shown. If I saw ads for X last week but didn’t notice them, then spoke to a friend and noticed them and took a screenshot, it would appear to confirm the theory. Even though I was always seeing ads for X.
Ultimately, I don’t think people who are convinced of this theory will change their minds so it’s a moot point.
Yeah, that’s the biggest flaw in the experiment I proposed, I think. This is the reason I try to have the hopefully independent grading of ad-topic-relevancy blinded to which system the topic was communicated over. It may be that one sees many vaguely related ads for the WhatsApp topic due to some selection bias but a similar number of actually related ads.
Also, what we’re interested is if the text changed what was shown. If I saw ads for X last week but didn’t notice them, then spoke to a friend and noticed them and took a screenshot, it would appear to confirm the theory. Even though I was always seeing ads for X.
Ultimately, I don’t think people who are convinced of this theory will change their minds so it’s a moot point.