On: Takeaways from hundreds of LLM finetuning experime...

[knorker]

If you write it "LORA" then people won't know if you mean "LoRA" or "LoRa".

Microsoft already created this huge confusion, so I'd recommend you help not make it worse by using the relevant capitalization.

[carbocation → knorker]

Both this article and Microsoft are referring to the same thing (low-rank adaptation, LoRA, which I have lazily miscapitalized). What is the confusion that you think will be addressed by my fixing that?

[wlesieutre → carbocation]

LoRa is a LOng RAnge radio system https://lora-alliance.org/

LoRA is LOw Rank Adaptation, the LLM tuning technique

[carbocation → wlesieutre]

So you are saying that in the context of an article about low-rank adaptation, where I was talking about applying this to ResNets, there was actually confusion that I was referring to long range radio?

[xd1936 → carbocation]

Sample size of one, but I clicked the article thinking that someone was using an LLM to "tune" the range performance of LoRa radio somehow. :(

[enjenye → xd1936]

Make that a sample size of two.

[drakenot → enjenye]

Had you heard of LoRA in the context of machine learning before?

[abraae → enjenye]

Make that 3. I imagined somehow using an LLM to even further increase LoRa range, maybe sifting through noise.

[snitty → carbocation]

As someone who is familiar with LoRa, but not LoRA, I was confused as to what they could have possibly been using LoRa for to finetune an LLM. The internet has a lot of acronyms, and a lot of jargon, and a lot of people doing seeming absurd things, okay?

[wlesieutre → carbocation]

I'm not the one who originally commented to complain about your capitalization, but sure.

Even if not in this specific comment thread, maybe having correct capitalization here means someone else knows how to write the one they mean to talk about in another conversation later, and then it will have been helpful.

If everyone went around calling both of them LORA all the time absolutely that's going to sometimes have people talking past each other about different technologies. Even with them capitalized correctly it's still going to, but not as much.

Too bad Microsoft didn't call theirs LoRaA and it would've been more obvious.

[ajsnigrutin → carbocation]

Yep, i clicked on it, because i thought it was about LoRa (the radio protocol). LoRa has many capabilities and many uses, from simple signalization and telemetry data to full on mesh networks for chat based communication, and, well... there are many things that could be fine tuned there :)

[HerculePoirot → ajsnigrutin]

There is "LLM" in the title ...

[ajsnigrutin → HerculePoirot]

Well yeah, and meshtastic is a chat app over lora (radio)... why not have a LLM somewhere on lora?

[knorker → carbocation]

My comment was more in general, that people who are aware of both, and read carefully, will spot the distinction by capitalizing it unambigiously.

They (and I) may need to do a double-take if they process it wrong, and you reduce that risk by being as clear as possible, despite Microsoft's incredibly poor naming choice.

By analogy, if I refer to MICRO soft, and their operating system, I'm sure there's no confusion to you. But it is harder to read, and may (for longer sentences) require re-reading the whole sentence.

(and, of course, the other replies you got, and other people being confused every single time there's any announcement or article about LoRA)