
Show HN: Cyborg Writer - In-Browser Text Editor with Neural Autocomplete - antimatter15
https://cyborg.tenso.rs/
======
antimatter15
Hey HN!

This is our latest demo for TensorFire
([https://tenso.rs/](https://tenso.rs/)), a Javascript library for running
neural networks in the browser GPU-accelerated by WebGL.

Each of these models is a simple two-layer char-rnn (the kind explored in
Karpathy's "Unreasonable Effectiveness of RNNs" post
[http://karpathy.github.io/2015/05/21/rnn-
effectiveness/](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)).
When you hit "Tab" the editor feeds in the contents of your document letter-
by-letter into a network which is trying to predict the most likely next
letter. It samples from that probability distribution and feeds it back into
the network to generate an arbitrarily long string.

We've published code and instructions for training new models on Github
([https://github.com/tensorfire/cyborg-
writer/tree/master/trai...](https://github.com/tensorfire/cyborg-
writer/tree/master/training)). With about a megabyte of any sort of text, you
can train a model and easily embed it into any web application.

We (@bijection and @antimatter15) will be answering questions in the comments.

~~~
projectant
No offence, but with a megabyte of text per speaker I feel I could do better
with word n-gram Markov models (even bigrams), and random path choice.

I think deep learning is great and all ( and I'm meaning to learn it ) but
shouldn't it be able to do far better than Markov models, or other simple
things?

Image captioning? Incredible. Deep Mind winning video games? Incredible. Style
transfer? Incredible. With one exception ( I saw on HN ages ago, sorry I have
no link -- it basically generates novel text using deep learning, across all
sorts of genres, such as "academic paper", "math paper", "novel", "film
script" and I found the results remarkable and interesting ) I question if
many text applications are doing better than Markov.

I think the issue is there is something fundamental and sophisticated about
human language which our current deep learning models, with all their
omniscient benevolence ( or whatever ), are missing. There's something deep
about the structure of language that we are not modelling yet in deep learning
as far as I've seen. When we do .... boom ... computers that learn from the
internet and amaze us all. Then we'll have something to shine, smile about or
fear.

Sorry for the digression and what may be inapplicable comparisons. I can get
impassioned about this topic.

~~~
projectant
i think this, learning of hierarchies, could be the start of something
relevant to unlocking more workable language modelling

[https://blog.openai.com/learning-a-
hierarchy/](https://blog.openai.com/learning-a-hierarchy/)

~~~
timthelion
I certainly agree. After-all, each rule for creating a pair in a syntax tree
could be learned and these rules could be put together [1]

[1][https://www.google.cz/search?q=sentence+structure+chomsky&so...](https://www.google.cz/search?q=sentence+structure+chomsky&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjoq7CJ5pDXAhXkCMAKHbX7BowQ_AUICigB&biw=1262&bih=594#imgrc=c5TW8LQzC5gbzM)

------
jaster
Ha very nice!

I made a vary similar (but much rougher around the edges) project for fun a
few months ago using a smaller french corpus of press releases from _La
Quadrature du Net_ (a non-profit lobbying for net neutrality in France and
Europe).

There is a live demo at
[http://jorquera.net/quadra_neural/](http://jorquera.net/quadra_neural/). You
can press tab to auto-complete word-by-word, or just start typing and the
suggestions update on the fly. Instead of "Weirdness" I called my cursor
"Drunkenness" :)

I used keras-js ([https://github.com/transcranial/keras-
js](https://github.com/transcranial/keras-js)) to run my model in the browser.

If you are interested in this kind of experiments, do go for it! It's quite
accessible, and projects of this scale do not require heavy hardware (I
generated my models using a GTX 770).

If you need a kickstart, in addition to the repo from the OP all the code for
my project is Free Software
([https://github.com/tomjorquera/quadra_neural](https://github.com/tomjorquera/quadra_neural)),
and I tried to document the generation process in a jupyter notepad
[https://github.com/tomjorquera/quadra_neural/blob/master/gen...](https://github.com/tomjorquera/quadra_neural/blob/master/generation/model_gen.ipynb).
While I did it on a french corpus, it should work with any language.

I used two main resources for inspiration :

\- [http://karpathy.github.io/2015/05/21/rnn-
effectiveness/](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)

\-
[https://github.com/fchollet/keras/blob/master/examples/lstm_...](https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py)

------
westoncb
Pretty hilarious. I've found Tupac to work the best, e.g. (my starting text in
quotes):

    
    
      "I was reading Hacker News when" he gladed out
      Somebody excuse I went down today
      I wonder if I give a fuck, when I creep
    

For some reason Donald Trump always seems to produce garbage...

~~~
sajattack
"For some reason Donald Trump always seems to produce garbage..."

I think that means it's working correctly.

------
c3534l
Eh, it's scarcely better than a Markov chain. I was hoping for something a
little more useful, like the ability to scroll through many similar words or
concepts.

~~~
gwern
You might like this, then: a word2vec tool which suggests words of the same
rough meaning starting with a particular letter or spelling, allowing you to
write Genesis solely in words starting with 'a'
[https://llamasandmystegosaurus.blogspot.com/2017/05/alpha.ht...](https://llamasandmystegosaurus.blogspot.com/2017/05/alpha.html)
or generate large numbers of rhyming definitions (see comments), some examples
of which:

    
    
        Cat: purr fur
        Dolphin: bark shark
        Puppy: yelp welp
        Shetland pony: shag nag
        Mosquito: mighty bitey
       Shrimp: chill krill

------
nsomaru
Fun with Shakespeare:

It seemed, somehow mock'd; Unbashed as it was my father.

My lady shall have him, perchance and reasons will abide it. Upon his unspoken
thing, the devil shall see.

MISTRESS QUICKLY: Hide amongst you the gods? Wherefore comes here, sir?

------
dchuk
We have to be pretty close to someone having a system to generate mountains of
convincing/unique "content" using Neural Nets (or something comparable) for
the purposes of web spam/seo right?

Back in the day, SEOs would use spun content to generate lots of unique
variants. But at this point, surely a ML algorithm can crack this nut?

~~~
PeterisP
Sure, at this point building RNNs that generate unique text on a particular
topic isn't a big thing to do, it's something that can be given as a home
exercise for undergrads.

------
giosch
I was expecting it to at least generate valid words, but that doesn't seem to
be the case...

~~~
ballenf
Safari on latest MacOS gives me unintelligible words pretty often.

My input: "I went to the grocery store to pick up some milk for"

Output on wikipedia setting:

"I went to the grocery story to pick up some milk for ... proposes of such to
take a amplifier and the exporteruation of the player of former"

None of the options seem to key off of the text of my sentence except for
vaguely the last word (in my case a generic preposition). Am I doing it wrong?

Here's another example, with the ellipses where I pressed tab:

> As I walked along the forest trail, I came to a most amazing [...] the
> history of the town himself, the complete material was made to be a
> traditional

~~~
yorwba
You're starting the text like a story (in the first person even!). Wikipedia
articles aren't usually written like that, so the network is in unfamiliar
territory. Its only option is to quickly revert back to something closer to
the style of an informative article.

------
stmw
This is actually pretty neat, although requires more digging

------
sterex
Is there a technical reason why it generates text with lot of spelling
mistakes?

Ayn Rand seems very sloppy with her spelling!

------
novium
It's pretty neat, even got a t.co URL. Sadly, it didn't lead anywhere.

------
ttul
Mobile Safari just sits there...

------
_grep_
Hitting tab crashes Chrome for me.

~~~
bijection
Thanks for the tip! What OS and chrome version are you using?

~~~
snerbles
Not OP, same issue here. Whole browser just locks up, pretty impressive.

Windows 7, Version 61.0.3163.100 (Official Build) (64-bit)

~~~
_grep_
Same specs here.

