
Low-code, no-model recommendation system with GPT3 - satorii
https://medium.com/swlh/gpt3-empowered-recommendation-system-ba4bad29276b
======
superasn
I wish there was a light version of gpt3 with maybe like 100 api query limit
per day so just amateurs like me got to play with this API too on weekends.

I have no agenda of changing the world and I have nothing substantial to fill
in the gpt3 invite form but sometimes the best ideas come from tinkering and I
just wish they gave some limited access to everyone.

~~~
dharma1
I thought they have a free tier from October?
[https://twitter.com/hardmaru/status/1301362995356774401](https://twitter.com/hardmaru/status/1301362995356774401)

Not sure what the 100k tokens translates to in terms of query limit

~~~
dtech
That's 3 month max though

~~~
dharma1
would be nice if it was longer, but should have decent ideas in 3 months. And
having another go with a throwaway email probably not that hard

------
m12k
"Recommendation system is so successful in many products and services we
interact everyday" ->

"Recommendation system[S] [ARE] so successful in many products and services we
interact [WITH] every[ ]day"

I'm sorry, but four errors within the first sentence is just not good enough -
did the author even read this themselves just once before hitting publish?
Show just a little bit more respect toward the people who give their attention
to what you've written.

~~~
satorii
Thanks for pointing out the errors. I got them fixed in the blog. This is also
a good use case of GPT3, checking grammar and spelling. I might build an
application for this.

~~~
m12k
Thanks for fixing them - and sorry if I was overly harsh. Sharing your writing
with the world is nerve-wracking enough already without having the spelling
and grammar shot down on HN. Truth be told, my first thought was "maybe this
is some example of text created by GPT-3?" But I think you're right that it
would actually serve well as a combined spelling and error checker to catch
things like that - I hope you do build an application for it, that would be
neat.

------
anaganisk
Given that "Open" AI API is very pricey, Im waiting to see how economical is
it to use this as it is heavily marketed by all these blog posts.

~~~
rococode
For anyone else curious, the pricing according to gwern [1]:

    
    
        Explore: Free tier: 100K [BPE] tokens, Or, 3-month trial, Whichever comes first
        Create: $100/mo, 2M tokens/mo, 8 cents per additional 1k tokens
        Build: $400/mo, 10M tokens/mo, 6 cents per additional, 1k tokens
        Scale: Contact Us
    

Expensive but (unfortunately) pretty standard market price for cloud NLP
stuff. The price is about $40 per million tokens.

For comparison, cloud translation costs per million characters range from $6
(Azure bulk pricing) to $20 (Google/AWS without bulk discounts). If we assume
the average BPE token length is 5 characters, GPT-3 will cost about $8 per
million characters.

[1]
[https://old.reddit.com/r/GPT3/comments/ikorgs/oa_api_prelimi...](https://old.reddit.com/r/GPT3/comments/ikorgs/oa_api_preliminary_beta_pricing_announced/)

~~~
rvz
In one of the Reddit comments about a service (PhilosopherAI.com) using the
API outlining the possible total costs:

> That makes for 400 million tokens in 2 or 3 weeks, which puts me at like
> $4000/mo minimum

Oh dear, that means...

> Scale: Contact Us

So while they are using the API for free for now, by these metrics, they are
least going to be soon having a running cost of at least $4K+/mo. That is an
expensive toy. The same goes for AI Dungeon.

> App Store earnings will be used to cover OpenAI costs & develop new
> features. Thank you for your support.

Well I will be seeing a tsunami of complaints from OpenAI projects that they
can't pay up for their "game changers" since they will be priced out quickly.
OpenAI Wins, Everyone else loses.

If this is for simple projects, then I wouldn't dare build an entire startup
based on someone else's API. OpenAI will win by default here.

~~~
iforgotpassword
I was thinking about Ai dungeon the other day. I can't imagine it really is
sustainable with those 10$ subscriptions covering the free play folks too.
What happens whenever they get featured on some bigger website?

~~~
ChefboyOG
AI Dungeon has already scaled up to over 1,000,000 players and managed costs.
They have a whole article about it here:

[https://medium.com/@aidungeon/how-we-scaled-ai-
dungeon-2-to-...](https://medium.com/@aidungeon/how-we-scaled-ai-dungeon-2-to-
support-over-1-000-000-users-d207d5623de9)

An important note, as others have said, is that not every player gets the same
model. Free players, as far as I'm aware, only get GPT-2 ("only" is a strong
word, as up until GPT-3, GPT-2 was the state of the art in text generation and
is still extremely impressive in its own right).

------
dannyw
GPT3 really is remarkable.

If you told me in 2015 that I can give just a few examples of movie
recommendations, on a model trained for general text, and get perfectly
coherent recommendations, I wouldn’t have believed you.

~~~
rvz
Indeed. But now OpenAI has become more like another DeepMind, but starting to
rent out their APIs to its users, indicating that they probably won't ever
release models for their research.

They might as well rename to Standard AI.

~~~
calebkaiser
I don't fully disagree, but I also think there is a layer of complication here
that goes beyond OpenAI's hopes of monetization (which are clearly a factor).

I maintain an open source ML deployment platform, and I've interacted with a
bunch of teams that have used it to deploy GPT-2. It was actually the platform
AI Dungeon built their app on. GPT-2 is a beast to deploy—it's huge (almost 6
GB fully trained), requires GPUs, and scales fairly poorly. You need to
autoscale GPU instances aggressively to handle any kind of real time inference
situation with it, and even with spot instances, that gets expensive quick.

GPT-2 is 1.5 billion parameters, and at the time, was scandalously large.
GPT-3 is 175 billion. For a model that large, there's real questions around
whether it's even feasible for the average team to use it if it is not hosted
somewhere else as a third party API.

From that perspective, I think the value OpenAI captures with the API is less
about the exclusivity of the model itself, but the exclusivity of their
infrastructure. Because of that, I wouldn't be surprised to see them open
source the model for research.

However, I 100% agree that the fact that the model still isn't open is
concerning, and it casts some doubts on whether or not it will ultimately
happen in the future.

------
PeterStuer
Nice as a stunt, but feels like taking a 40 ton truck to get your lunch
sandwich from the bakers.

~~~
feral
If this turns out to be easy and convenient, it won't matter whether the tech
is overpowered.

A family car is really overpowered for most trips to the bakers.

Yet people take the car all the time, instead of walking or cycling, because
it's just more convenient and easy, and people are lazy.

You can't write just this off as a stunt.

If you described how YouTube renders cat videos to Tim Berners Lee in the mid
90s, it would also seem like a 40-ton truck.

And yet here we are.

------
sanxiyn
Serendipity is another GPT-3 based recommendation engine. Check its about
page. [http://serendipityrecs.com/](http://serendipityrecs.com/)

------
serendipityrecs
Hey, cool, you built the same thing as me but with less code:
[http://serendipityrecs.com/](http://serendipityrecs.com/)

One thing that's cool about GPT-3 is how versatile it can be, so I'm exploring
a natural language search interface, which is something that traditional
recommendation algorithms can't do.

Are you planning to incorporate GPT-3 into your startup? I'd love to hear
about what use cases you envision it solving. If you're interested in
connecting, my email is in my profile and my website.

~~~
satorii
Cool! I will drop you a message or catch up with you on openai-api slack.

There are a couple of features we see GPT3 stands out and potentially benefits
the product features.

Currently we are evaluating the end to end quality and cost versus our
traditional ML based recommendation and semantic search pipeline.

------
gitgud
Well it's a _no-model_ AI system if the interface is a HTTP endpoint that's
out of your control...

------
jwiley
Very creative use of the technology. I wonder when GPT3 will evolve to handle
more languages, and what if any steps they take to reduce cultural bias? If I
was in Japan these recommendations would certainly be less useful. :)

~~~
tastroder
Given this blog post I have no idea if it even works well for English. Sure,
the demo video contains sensible recommendations but what gives? Most of those
I've seen seem to be included in the training data shown in the image below.
If you have the term "AI" in your blog's blurb readers might expect the author
to actually perform an evaluation of stuff they write ads about.

------
fancyfredbot
Presumably when a new movie (not included in GPT3 training data) comes out the
recommendations are not going to be too good, unless someone spends a lot of
money retraining the underlying model?

~~~
feral
This will be the case for everything built on GPT-3, but you've got to imagine
it's possible for them to keep refreshing it, eventually training or fine
tuning it on a daily basis as their web crawl updates.

------
empiko
I am curious how much it actually thinks and how much it simply uses text from
recommendation engines that got crawled.

~~~
The_rationalist
A neural network can't think, please stop cargo culting neural networks

~~~
plutonorm
Panpsychism is a serious philosophical position.

~~~
The_rationalist
Could you please give an explanation of panpsychism applied to software that
make sense? Spoiler: you cannot and it's totally refuted empirically. If I
code a hello world, what kind of cognitive abilities does this have?

BTW I guess panpsychism researchers should start talking to rocks maybe one
day they'll find one that is an AGI! /s

~~~
MacsHeadroom
The position is that information not physical matter is ontologically primary
and that substrate independent integrated information processing is identical
to consciousness.

A hello world program and a rock are not integrated information processing
systems. The former is an automata which can be contained in an integrated
information processing systems and the latter is a more arbitrary construct
which does very little information processing at all (much less integrated).

~~~
bnlapp
Sounds like the ether in physics around 1900 or the four elements in Greek
philosophy.

Non-falsifiable dreams.

------
tgv
Intersteller (with -er, indeed)? How did it pick that one up as an alternative
for Inception? There is something with that title, but no information in IMDb.

Anyway, for the rest it looks similar to Netflix' recommendation: similarity
in actors, directors, keywords. A toy.

~~~
janekm
A2:B10 (including the results for Intersteller{sic}) are the prompts, it's
very confusing in the video. The results for "The Irishman" are from GPT-3.

------
nerdponx
No model? What? Of course this is a model, it's just a pre-trained one.

