
Show HN: AI to help you write fast. modes: gen, legl, hp, research, got. [OSS] - jeffshek
https://writeup.ai
======
jeffshek
I built writeup.ai to learn more about machine learning. I underestimated the
amount of time it would take (thought 1 month, took 3, classic 3x off) ...
alas, the feature creep of my judgement was hard to manage.

I learned WAY more about deploying ML in production environments than neural
networks. ML DevOps was surprisingly hard. the ecosystem is a bit fragmented
with managing TensorFlow / PyTorch / Docker / Nvidia dependencies. since
there's SO MANY ongoing changes they have a lot of out-of-date documentation.

the backbone of the algorithms running this is from gpt-2 (openai's text
generation algorithm). i experimented with almost all of the other text models
including bert, transformer-xl, xlnet. i looked into ctrl (salesforce). most
of them had incredibly high memory requirements, speed wasn't the fastest, or
the output wasn't as good as gpt-2. this isn't a knock on the other
algorithms, they were designed with the focus of answering questions in test
sets (reading comprehension, etc).

i fine-tuned gpt-2 medium w/gradient checkpointing for five different
datasets: legal, harry potter, game of thrones, song lyrics, academic
research. training was relatively straight forward, but the data scrubbing was
incredibly tedious. i'm still training/updating the research models.

reddit has made some amusing harry potter and game of thrones fanfiction
w/this so far:
[https://www.reddit.com/r/FanFiction/comments/d5s9yh/i_made_a...](https://www.reddit.com/r/FanFiction/comments/d5s9yh/i_made_a_bot_that_writes_fan_fiction_with_you/)

app hosted on google cloud. everything is run via docker containers (which is
also hard w/nvidia). autoscaling on usage. the main inference runs off from
either cascade lake (cpu, surprising reasonable performance), k80s, or p100s.
i experimented with the speed and profile of them a lot.

open-sourced at [https://github.com/jeffshek/writeup-
frontend](https://github.com/jeffshek/writeup-frontend),
[https://github.com/jeffshek/open](https://github.com/jeffshek/open)

i'm going to write a detailed blog post about the
code/deployment/infrastructure soon if you're interested. love to answer any
questions and get feedback! thanks!

------
jeffshek
No one will ever see this self pep talk, but posting on HN and finding no one
cares about your project makes me quite sad.

Just have to pick your head up and continue.

