Hacker News new | past | comments | ask | show | jobs | submit | jpwagner's comments login

At some point the bibliography needs to reference the LLM itself which would need to be hosted indefinitely.


I do not think it is feasible to make sure prompts are reproducible. Considering that a LLMs are large you can not host every version of the model indefinitely.


ChatGPT when asked this question answers that its responses are probabilistic, so the responses aren't reproducible. I tested that myself, of course. Since it gave me 2 different (but overall equivalent) answers from the same prompt I'd have to agree.


That’s because it’s configured to with non-zero temperature. I’d you use the underlying model API, or the playground, you can get repeatable results when temperature is zero.


Mostly, yes, but there have been reported cases where even with zero temperature, you get nondeterminism, probably due to accumulation of errors due to different operation order on floating points.


You can also generally set an RNG seed to get reproducible results.


if it's not reproducible, it's not science


Some would argue that, due to the unfortunately near-universal deprecation of paper authors after 70-90 years, the actual process of writing any particular paper is not and has never been reproducible. As opposed to experiments, which are reproducible and are not generally contained within the operating weights of a LLM nor the thoughts of a human.


Your observation, while accurate, is a tangent. The point of the bibliography in the context of an academic paper is to reference the academic merit of the work. In the case of science, this would be reproducible experiments (ideally).

Perhaps you would prefer to include the generated text source as an author.


I'd rather not include it at all, for exactly that reason. It's just a writing tool- the paper is either correct or incorrect on the same basis as any other paper. We include bibliographies to ensure that the relevant scientific data is present, but I don't think there's any reason to say that a non-reproducible abstract isn't science


It belongs in the acknowledgments, along with Bob’s wife who did a bit of proofreading, Steve McProfessor who had a chat with the authors once, and whatever software was used for the figures.


This. It’s crazy that people are thinking we should _credit_ the model as if it were an author. It’s a tool, and should be usable without limit.

It would however be nice to have a mode where any output that matches some existing text in the training set could be highlighted, to help one avoid unintentional plagiarism.


Plagiarism will undergo a change in definition.


Plagiarism used to be norm until mass similarity checking has been available.


In published writing?

If so, we could pattern match historically, and discover the base rate of plagiarism.


^ nailed it


A long time ago I did high school math instruction. I used Sudoku for an initial introduction to proofs - something where the student knew they were implicitly going through a "proof" process.


Ran a consultancy for years

Most important advice I'd give is to liberally turn projects down and there are two positive outcomes

- you don't take on projects that don't make sense for you

- the best projects are the ones where the client is resilient enough to negotiate through your initial "no"


This is terrible fun, or dare i say "shitty-witty"


is your framework a reference to David Brooks' Ted Talk?

https://www.ted.com/talks/david_brooks_should_you_live_for_y...


I had not seen this video, but it clarifies my thoughts on this topic quite a bit. Thanks for sharing!


i don't think you're imagining the selection process properly.

i think to get the right mental model, imagine instead that they only wrote "send a one-minute video and convince us to invite you".


That exactly how it looks, and this approach seems wrong here. For somebody who can articulate - on video, no less - you have some who can't. Not sure who's more promising.




If you haven't clicked through to read the above commit message, please do. It is amazing. This is a glimpse into the mind of a mad scientist (and I respect and love mad scientists more than anything and aspire to be one)


> It is a tragedy, but I cannot imagine a language better than Spiral. Not within real world constraints facing me. A language better than it would pretty much have to be magic.

> So, in this respect, my programming skill has reached it's limit and the most I will be able to do is refine it.

> And it is embarrassing to me because I want to do better. I am ashamed, but the one to come will have to follow up on what I could not manage with my meager human abilities.

This reminds me of the fictional inspirational process of dwarves in the Drizzt Do'Urden series of books [0] by R. A. Salvatore [1]. In that setting a dwarf craftsman gets only one moment of exceptional divine inspiration in his life to forge a legendary artifact. It is both a blessing and curse for them, because they realize that they will never be able to surpass that work for the remainder of their long lives.

I wonder if other people here have had that moment in their life as well?

[0] https://en.wikipedia.org/wiki/The_Icewind_Dale_Trilogy

[1] https://en.wikipedia.org/wiki/R._A._Salvatore


The dwarves should be thankful. Many people would have this moment and not realize it has passed them by.


https://github.com/mrakgr/The-Spiral-Language/commit/1dee010...

That one is really in the "mad scientist" mindset. I love it, I wish him and this project good luck.


The funny part is the diff.


I didn't even see that! Yep. When all you have left to polish is some white space in a test script, the thing is pretty much complete.


No, he uses commit messages as log of his thoughts, and commits need to have changes, thus each "log only" commit deletes or adds those same whitespaces in a never ending cycle.

In between you find some actual code changes (with also long commit messages).

It's not that he's finished - I hardly guess so because he complains that he'd need 10 years of time to polish everything out perfectly (in his opinion) and he only has one - it's just his style for a log book...


I can't actually program all the time. Sometimes I need to study machine learning or think about design. During those times I use commits much like journal entries.

I actually keep a separate journal and sometimes paste from it. I've been using LibreOffice Writer for it and decided to drop it recently because it would take so long to save a file - like 5s or more. Around every 3 month I'd fill in about 1000 pages of it and it was forcing me to move to a new file every time that happened. Now I just use VS Code for this sort of thing. Raw text is the best after all, but it is too bad I can't paste images into the journal anymore.


If you do wish to use commits to simply journal your thoughts, and would like to do so without this requiring code or file changes, you may find the --allow-empty option useful when committing.

This allows you to create a new commit, even if no files have changed.


Or, write your journal entries to a file and commit that? That's what I do. Most of my personal projects have a "notes.txt" at the top level for this purpose. Sometimes I just need to write freeform about the problem I'm trying to solve, from different angles, to work out my strategy. Sometimes I just write the despair that I'll never figure something out.


How would one enable this in Visual Studio?


Unfortunately, as far as I am aware, Visual Studio does not directly expose this option.

However, it is quite likely you can use Visual Studio's "External Tools" feature to be able to achieve this. (https://msdn.microsoft.com/en-us/library/76712d27.aspx)

You could set up an external tool that did something like the following:

Title: Commit with Selected Text as Message

Command: path\to\git.exe

Initial Directory: $(SolutionDir)

Arguments: commit --allow-empty -m "$(CurText)"

This will allow you to create an empty commit, with the selected text becoming the commit message. (Just bear in mind if you have any files staged, they will be committed as well)

I unfortunately do not have Visual Studio currently installed, so please take the above as a rough sketch, and my apologies if it doesn't work as is. But, I hope it is of use to you.


> commits need to have changes

Ah, yes, I remember wanting to be able commit the identity transformation (_i.e._ no change) at some point. I don't remember why. It might have been a matter of principle.

PS: I just learned about the `--allow-empty` flag. [0] I'm happy again.

[0] https://git-scm.com/docs/git-commit#git-commit---allow-empty


Oh; I missed the back and forth.

But he said he was done, because he no more time for language design, and he must study AI now.


what should we call this new form of web log ? c[ommit]log ? v[ersion]log ? g[it]log ?


FØCAL | Cambridge/Boston | ONSITE, REMOTE-possible | f0cal.com

---FØCAL---

We are building a platform for development and testing your code on simulated and/or benchmarked hardware - with computer vision applications as our beachhead market. We liken it to "CAD for CV".

We have a great team and are growing (right now!) in a lot of areas:

- Product management

- Developers (full-stack, image processing, compiler experts, hardware, proserv)

Feel free to reach out - email in profile


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: