Congratulations on getting your hands dirty and doing everything yourself like computing gradients manually, badly shuffling (non Fisher-Yates), badly js transpose (double swapping), it is a great way to learn.
Congratulations on completing a full pipeline, that's the hard part then it just swapping pieces for better pieces.
I advise non-technical readers not to attach much value to the results of this neural network as it is probably inferior to the even simpler naive Bayes.
The model of the neural network is simplistic :
The Concat operation mean it is especially sensible to dropping or adding a word as it will offset the remaining words and give a totally different vector.
Using word vectors mean it doesn't forgive any spelling mistake as a spelling mistake will usually correspond to <unknown> vector.
Using a feed forward neural network means formulaic titles with a single word substitution from a good positive example from the training set will often work.
It is trained by gradient descent using a squared error loss, on ~1000 examples one example at a time without cross-validation using a custom written neural network library. (Almost all these bad choices can be solved by using a framework).
It seems to have successfully over-fit as it return Good ~1.0 for positive examples from the training set.
When I learned about KNN, I made this project:
When I learned about Kmeans, I made this one:
When I learned about Perceptrons:
https://github.com/victorqribeiro/carGamePerceptron (the training is done when you run the project)
When I learned about MLP:
When I learned about Neural Evolution:
None of these projects are perfect, but helped me to better understand things. I did implemented a professional solution for a company using machine learning, but I used scikit-learn (back then). I have some experience with tensorflow also, but implementing things makes me fell like in control. But I know whena a solution is good enough for production and when it's not. I made clear on the README of this project that this is not at all good for production. I took the time to show every single step of the process and tell it how it is. I had limited time and limited resources, and with the little I had I made a "fun" experiment. I enjoy doing this kind of thing. I have to squish this kind of projects in my free time, cause I have a full time job and I'm finishing my final thesis (I'll defend it two week from now).
Sorry if I turn this into a journal entry, but I kind got hurt yesterday when the other guy shted all over my project That's not at all what you did and I appreciate that. Thanks again.
But this ends up on frontpage anyway. Welcome to HN.
You will overfit an NN trained on only 1000 examples.
Also a simple train/test split will tell you that. But the author failed to take any time to learn the basics before spewing out this drivel.
I'm sad to do that because you are knowledgeable about a number of things. Many of us could learn from you if you would share what you know without putting other people down. But the aggression subtracts more than the knowledge adds. We can't have users behaving like this sort of asshole in comments, least of all in Show HN threads, where the idea is to teach people things and expressly not to shit on them and their work: https://news.ycombinator.com/showhn.html.
Other users here know things and are willing to talk about them without being mean. GistNoesis modeled this wonderfully in the GP comment. We'll learn what we can from them instead. But if you decide that you want to use HN in the intended spirit, as described in the site guidelines and especially the one that says Be kind, you're welcome to email firstname.lastname@example.org and let us know.
1. "Apple introduces 16-inch MacBook Pro, the world’s best pro notebook" Bad: 0.9964 - Good: 0.0038
2. "Developing open-source FPGA tools" Bad: 0.3381 - Good: 0.6652
3. "Show HN: Can a neural network predict if your HN post title will get up votes?" Bad: 0.0598 - Good: 0.9307
4. "How internet ads work" Bad: 1.0000 - Good: 0.0000
5. "More Intel speculative execution vulnerabilities" Bad: 0.7413 - Good: 0.2306
6. "OpenSwiftUI – An Open Source Re-Implementation of SwiftUI" Bad: 0.9994 - Good: 0.0005
7. "How VCs Make Money" Bad: 0.9997 - Good: 0.0003
8. "OpenBSD: Why and How (2016)" Bad: 0.9988 - Good: 0.0013
9. "The Perl Master Plan: How to Put Perl Back on Top" Bad: 0.9997 - Good: 0.0003
10. "Jerry (YC S17) Is Hiring Senior Software Developers (Toronto)" Bad: 0.3142 - Good: 0.6800
So all in all, only 3 of today's top 10 had good titles... Either the titles could have been better but the content was too interesting, or this tool has very low recall.
"Warning: bad economist" Bad: 0.0001 - Good: 0.9999
"Warning: bad artificial intelligence" Bad: 1.0000 - Good: 0.0000
Kinda spooky even, who knows, social media totally might have already killed companies that sounded too different or even just political ideas that differ from mainstream (or sponsored) views?
Categorizing submissions to different types, and repeat the experiment, you'll find the program may predict blog/article and "Show HN" submissions with higher accuracy.
This is from the Github project.
6 characters Bad: 0.0085 - Good: 0.9915
EDIT: find a higher score than yours at one character less
6 characters Bad: 0.0025 - Good: 0.9975
for 5 characters:
"I won" Bad: 0.0031 - Good: 0.9976
at 4 characters:
"J ML" Bad: 0.0002 - Good: 0.9998
Bad: 0.0267 - Good: 0.9720
"this is just a tool for detecting buzzwords"
=> Bad: 0.9991 - Good: 0.0011
"this is merely a device for detecting artificially sophisticated words"
=> Bad: 0.0019 - Good: 0.9980
=> Good: 0.9986
"linux for sohpisticated when if contextual what but then what"
=> Good: 0.0201
Put another way, I think misspelled titles are well within the problem domain here.
Is it interesting for this project? Based on my skimming of the README, probably not. I don't think it handles misspellings intelligently.
Maybe it's sohpisticated because Linus is going to RIIR?
Bad: 0.0001 - Good: 0.9999
I'm honestly interested - do we hobble our neural networks with our choice of training space? If for instance, the HN-NN-input space including 'prepositions and word-frequency statistics', would the network train for sentence sophistication? Just because those stats were in front of it?
Yes, frequently. It is common in various areas of math -- mathematical background is not something "normal people" usually flaunt, but there are good reasons to expect programmers to have much more such background than average.
A good rule of thumb is that people are probably using the words they use because those words make sense to them. See the first panel here: http://www.basicinstructions.net/basic-instructions/2009/1/2...
Unfortunately far too often both in tech and in CS academia (and I'm talking about informal conversations ie. this is an 'orthogonal idea')
I made cheese sandwiches for a week, here's what happened
And got: Bad: 0.0001 - Good: 0.9999
Not sure what to make of that.
Can you pass the butter?
But exactly the same score with:
Cannot your passed their butter?
Couldn't hill-climb past that.
Bad: 0.0598 - Good: 0.9307
Can an AI predict if your HN post will go viral? The results will shock you
Bad: 0.9539 - Good: 0.0401
Seems to work!
>> Show HN: Can an AI predict if your HN post will go viral? The results will shock you
Facebook is down (Bad: 0.0079 - Good: 0.9921)
HN pg (Bad: 0.0133 - Good: 0.9886)
Zuckerberg (Bad: 1.0000 - Good: 0.0000)
At least it gets something right ...
I think the AI needs a little work
Well, this project doesn't offer this tool, but it tries.
"A mouse killed our network engineer" - Bad: 0.2068 - Good: 0.7895
"A rat killed our network engineer" - Bad: 0.9698 - Good: 0.0322
"A network engineer killed our mouse" - Bad: 0.0397 - Good: 0.9529
The distribution of points is very 0 heavy. It can be a problem to represent it and to model it.
"deep neural network ate my tuna sandich" Bad: 0.0000 - Good: 1.0000
"creating linux network sockets" -> Good: 0.01
Bad: 0.9988 - Good: 0.0013
Pretty sure HN would break if this actually happened.
Bad: 0.9999 - Good: 0.0001
Rust is 13x better over Go in this benchmark.
Bad: 0.1897 - Good: 0.7988
Looks like we have a winner.
in assembly, in VHDL, in brainfuck, in America, in McDonalds, in clickbait
but "in legalese" scores surprisingly high
A year ago I made a Hacker News submission score prediction notebook on the full HN corpus: https://www.kaggle.com/minimaxir/hacker-news-submission-scor...
Even with hundreds of thousands of data points, the R^2 was effectively 0.
I tried feeding https://rachelbythebay.com/fun/hrand/ to this. The results were... well, the input was very repetitive and the NN was only trained on 1.5k titles, so what can I say.
Some of the best:
1.0000 0.0001 The police found programming to raise sheep
1.0000 0.0001 The police found PS4 in the cloud
1.0000 0.0001 The future of API in the cloud
1.0000 0.0001 The economics of iPad in the cloud
1.0000 0.0001 My framework for Windows is patented
1.0000 0.0001 My framework for SDK is patented
1.0000 0.0001 My framework for Heroku is patented
1.0000 0.0001 I bootstrapped my Heroku in 2 years
1.0000 0.0001 Google kills Y Combinator in space
1.0000 0.0001 Coming soon: bigger blog is patented
Some of the worst:
0.0000 1.0000 Apple has Y Combinator in 2 years
0.0000 1.0000 China creating PS4 at a coffee shop
0.0000 1.0000 Choose Heroku for developers
0.0000 1.0000 Fixed: NoSQL at a coffee shop
0.0000 1.0000 Followup to the Arduino without warning
0.0000 1.0000 Followup to the marriage without warning
0.0000 1.0000 How I made Android at Stanford
0.0000 1.0000 How I made Obama on the freeway
0.0000 1.0000 How I made bitcoins in the cloud
0.0000 1.0000 How I made iPod at Stanford
0.0000 1.0000 How I made iPod on the freeway
I did this by porting this project to node (which was shockingly easy - took 3 minutes - because there were no JS libraries or frameworks used :D :D :D :D) and running it at the console. 5000 lines of output and repro instructions over at https://gist.github.com/exikyut/1714ad98a136d77d8674944410a4... (the output got pasted first for some reason, sorry)
Machine Learning and AI isn't about producing a perfect solution; that results in things like overfitting. Instead, think of Machine Learning as a human being. No person is going to be prefect and no single person is going to be able to please everyone. Instead, it's about building a solution that generally speaking, provides "good results". Subjective, 100%.
The project just started. Once the OP starts adding new features to the data set and improving the data set itself, I'm sure the results will start getting better and better. At which point, it'll be a good system to "test your subject lines" before posting.
What's the worse it could do? Tell you a subject is great when a bunch say it sucks? I've clicked on many head lines that I thought "sucked" but ended up finding the content very useful.
Great job to the OP and keep it up. This type of work isn't easy but certainly can be fun. Fk all of the negative opinions and keep on keeping on.
To be clear: this isn't a complaint about downvoting, I'm just pointing out the phenomenon. For example, its possible to go to bed with +7 and wake up to -4 (a rather stark difference) because a certain comment was posted during the European evening, and thus was 'exposed' to American HNers longer than it was to European ones.
Bad: 0.0005 - Good: 0.9995
Bad: 0.0004 - Good: 0.9996
Here is link I posted let's see. It's 100% bad.
Bad: 0.9917 - Good: 0.0076
I'm sorry guys!
"Show HN: Can a neural network predict if your HN post title will get up votes?"
Just in case, I've tested some more titles to make sure that "Show HN" doesn't (EDIT: typo, ugh) just boost any title when prepended.
"Internet": Bad: 0.4528 - Good: 0.5472
"Show HN: Internet": Bad: 0.9987 - Good: 0.0013
On the other hand, "Show HN" does seem to greatly affect the score, either positively or negatively.
Bad: 0.0015 - Good: 0.9986
Not sure what I'm going to build, but it's going to make me a lot of money.
It's a cool bullet point in the slide deck, and it gives you some metrics to graph on dashboards, but I'm unconvinced that it means anything.
This is nonsense. If the title is accurate, then it is closely related to the content, and the content has a lot to do with the upvotes you get.
Don't overlook "Should I submit this article?"
I'd argue that titles that are neutral and accurate have been edited from the original linkbait title that got the link to the front page.
all in all a great tool to analyse your titles, it will surely help this community grow and mature over time, finally. Thanks a lot for creating this.
Bad: 0.0598 - Good: 0.9307.
Bad: 0.9800 - Good: 0.0220
NB. Sorry for that, I just had to do it :)
Bad: 0.8083 - Good: 0.1910
"Scientists discover it is turtles all the way down" Bad: 0.9998 - Good: 0.0002
Bad: 0.9969 - Good: 0.0032
I am currently creating a filter that filters HN news and similar sources using a similar classfier. It learns on the fly and the accuracy of guessing my 'taste' is about 75%-80%. Better accuracy for this is explainable by the fact that my interests are more focused and the classifier has easier time predicting posts I would be interested in.
Hit me up at kevmod at gmail, would love to hear more.
"Elon Musk's advice on success"
Bad: 0.0000 - Good: 1.0000
"Artificial Intelligence Officially Solved" - Bad: 0.2722 - Good: 0.7477
"Singularity Achieved" - Bad: 0.4528 - Good: 0.5472
With "Show HN" they're universally bad:
"Show HN: Singularity Achieved" - Bad: 0.9999 - Good: 0.0001
"Show HN: Artificial Intelligence Officially Solved" - Bad: 0.9886 - Good: 0.0150
"Show HN: Artificial Intelligence Officially Solved, Singularity Achieved" - Bad: 0.9998 - Good: 0.0002
I've developed a cute Bittorrent client. Upvote, you leisurely butts!
Not sure, if I should give it a try?
More precisely, it received 3 votes, but then someone apparently was insulted by being called a leisurely butt and flagged the topic.
Good - 1.0
Bad. - 0.0
What do I win?
"Steve Jobs is resurrected. Buy bitcoin"
Good 0.0, Bad 1.0
Each word added, "bitcoin", then "buy", then "now", drops the score lower until it hits 1.0 bad.
This could easily be implemented as a (hover) tooltip.
Paul Graham Rust VC IPO growth
Bad: 0.0026 - Good: 0.9976
So far adding any kind of grammatical structure to the random list of keywords that turns it into a title that makes sense completely ruins the score...
-  - https://intoli.com/blog/hacker-news-title-tool/
"Show HN: Announcing styled-components v5": 0.9973 Bad
So, don't use "Show HN"?
BTW, I tried a bunch of single word titles (example: red, green, blue, title), and I always seem to get the same result: Bad: 0.4528 - Good: 0.5472
So, apparently, if you want to maximize your "score" with the lowest mental effort, just spam thousands of single word title posts, and then, it's a coin flip for each one :)
This seems very doable.
Adam Neuman got away with billions with WeWork bankrupt
Apple and Google help China's surveillance of dissents
Google Chrome monitors straight viewers
Solar panels can not slow climate change, only nuclear power can
SpaceX to to build new Silicon Valley on Mars
e.g: "Rihanna concert cancelled"
Bad: 0.0006 - Good: 0.9992
vs "Rust 1.33 released"
Bad: 0.9896 - Good: 0.0108
> spam can't ml neural network ai btc crypto
> Bad: 0.0013 - Good: 0.9986
If it can't, then it's just playing along
737 Max gun has flaw in SUV liberal IOT
Bad: 1.0000 - Good: 0.0000
a challenge .. the (non) prize is for reversing the outcome with the smallest diff.
i believe i am the current leader :)
If your model is "pick a title, and then predict how it will do", with the title being an independent variable, then the overwhelmingly negative assessment is quite correct. Most of the time, Stephen Hawking (or any other celebrity) hasn't just died, and that title would be a lie or a hoax.
To predict these obituary articles better, you'd need to build in an assumption that the content of the title was true, which would cause a lot of problems in the general case.
"someone ate my sandwich" Bad: 0.9999 - Good: 0.0001
Hacker news can't spell, or maybe associates sandwhich and someone eating it to some repository.
IBM buys Apple:
Bad: 0.9757 - Good: 0.0249
Only? If you want to make useful guesses on the time of a post, better take a much larger number, to ensure you get enough randomness.
That NN predicted.
"neural network fails Turing test" - bad: 0.9964 good: 0.0035
I think this AI might be a bit biased against AIs
>Bad: 0.1236 - Good: 0.8886
This says something.. though I'm not sure what :)
India bans the internet in kashmir. Bad: 0.9408 - Good: 0.0566
India bans youtube and whatsapp. Bad: 0.0059 - Good: 0.9930
Esp8266 USB input Manufacturing: Bad: 0.0775 - Good: 0.9290
Manufacturing Esp8266 USB input: Bad: 0.9992 - Good: 0.0009
Bad: 1.0000 - Good: 0.0000
It really disliked this one, even though it probably would get attention on HN.
Somehow I don't believe this. On the other hand, I believe this.
t. Google employee knowing what do with my stocks
>Bad: 0.1214 - Good: 0.8659
Can a neural network predict if your HN post title will get up votes? - Bad: 0.9917 - Good: 0.0076
I think it sums up all.
It's all about that Show HN.
Bad: 0.0002 - Good: 0.9998
"Bad XML Parsers" Bad: 0.9998 - Good: 0.0003
I might have to write that blog post.
0.94 Good: Facebook Literally Kills Puppies
Yup, that sounds like the HN I know.
This thing is broken.
Of course you will fail.
fluffy cats : Bad: 0.9921 - Good: 0.0090
>Bad: 0.0000 - Good: 1.0000
Bad: 0.1585 Good: 0.8280
Bad: 0.0012 Good: 0.9986
Bill Gates killed by a bug in Windows XP
Bad: 0.0026 - Good: 0.9975
=> Good: 0.9885
Good: 1.0000 :)
Bad: 0.0579 - Good: 0.9427
Bad: 0.0046 - Good: 0.9951
So... I guess the neural network has some learning to do?
the history of X scores:
power: good 0.91
wealth: good 0.99
sex: good 0.99
squirrels: good 0.99
cables: bad 0.97
keys: good 0.99
Maybe something to do with the wikileaks leaked cables?
"Planning sophisticated covfefe" Bad: 0.0002 - Good: 0.9998
Bad: 0.9879 - Good: 0.0129