sa-code's comments

On: Fine tune LLAMA3 on million scale dataset in consu...

[sa-code]

Thank you for saying this! The number of people that would need to fine tune vs just using RAG is really small. People that are not familiar with the source often jump to fine tuning as an option

[Foobar8568 → sa-code]

I am still unsure where to stand on this fine tuning vs rag. I feel that for live data, rag would be preferable but for daily/weekly updated one, then fine tuning.

Another aspect where I am unsure is multi user for a model e.g. can we have concurrency for a model or the queries have to be queued.

[bigfudge → Foobar8568]

Fine tuning doesn’t ’add content’ the way RAG does though. They’re not really comparable in that way.

[Foobar8568 → bigfudge]

So more to be optimized for specific tasks in a domain ?

On: Nvidia to Acquire Run:AI

[sa-code]

Run ai is not a mature product at this point though.

[intelVISA → sa-code]

Nvidia definitely overpaid an order of mangitude here, this setup's a month ish work for any semi-skilled SWE who is familiar with RDMA, SR-IOV etc.

On: Cloudflare CEO bought a newspaper to change report...

[sa-code]

This sounds like "things are bad so let them get worse", which is not a good look

[keybored → sa-code]

Is the PRC a people’s republic? Please don’t wish ill-will on the people of China by disagreeing.

On: Meta Llama 3

[sa-code]

EU actually has the opposite of draconian privacy laws. It's more that meta doesn't have a business model if they don't intrude on your privacy

[zmmmmm → sa-code]

They just said laws, not privacy - the EU has introduced the "world's first comprehensive AI law". Even if it doesn't stop release of these models, it might be enough that the lawyers need extra time to review and sign off that it can be used without Meta getting one of those "7% of worldwide revenue" type fines the EU is fond of.

[0] https://www.europarl.europa.eu/topics/en/article/20230601STO...

[taneq → zmmmmm]

Am I reading that right? It sounds like they’re outlawing advertising (“Cognitive behavioural manipulation of people”), credit scores (“classifying people based on behaviour, socio-economic status or personal characteristics”) and fingerprint/facial recognition for phone unlocking etc. (“Biometric identification and categorisation of people”)

Maybe they mean specific uses of these things in a centralised manner but the way it’s written makes it sound incredibly broad.

[mrtranscendence → sa-code]

Well, exactly, and that's why IMO they'll end up pulling out the EU. There's barely any money in non-targeted ads.

[sebastiennight → mrtranscendence]

If by "barely any money", you mean "all the businesses in the EU will still give you all their money as long as you've got eyeballs", then yes.

[ben_w → mrtranscendence]

Facebook has shown me ads for both dick pills and breast surgery, for hyper-local events in town in a country I don't live in, and for a lawyer who specialises in renouncing a citizenship I don't have.

At this point, I think paying Facebook to advertise is a waste of money — the actual spam in my junk email folder is better targeted.

[latexr → mrtranscendence]

> IMO they'll end up pulling out the EU.

If only we’d be so lucky. I don’t thing they will, but fingers crossed.

[extraduder_ire → mrtranscendence]

If it's more money than it costs to operate, I doubt it. There's plenty of businesses in the EU buying ads and page promotion still.

On: Mixtral 8x22B

[sa-code]

Fewer parameters at inference time makes a massive difference in cost for batch jobs, assuming vram usage is the same

On: Mixtral 8x22B

[sa-code]

Is this the best permissively licensed model out there?

[ru552 → sa-code]

Today. Might change tomorrow at the pace this sector is at.

[imjonse → sa-code]

So far it is Command R+. Let's see how this will fare on Chatbot Arena after a few weeks of use.

[skissane → imjonse]

> So far it is Command R+

Most people would not consider Command R+ to count as the "best permissively licensed model" since CC-BY-NC is not usually considered "permissively licensed" – the "NC" part means "non-commercial use only"

[imjonse → skissane]

My bad, I remembered wrongly it was Apache too.

On: Embed Crystal code directly in Ruby

[sa-code]

I think Python users are more on the mojo bandwagon rather than Nim now

On: A MySQL compatible database engine written in pure...

[sa-code]

Missed an opportunity to can this uSql!

On: Llm.c – LLM training in simple, pure C/CUDA

[sa-code]

I see it as an excuse to talk about it more, and I'm happy to read it all

On: GPT Chatting with Xfinity support getting discount...

[sa-code]

I would down vote this if I could.