I love this. It reminds me of [1], which I wrote years ago to do something similar with undefined method calls: Rather than erroring, a modified `method_missing` triggers and searches Stack Overflow for code snippets with the same method name, then injects that code into scope and walks back the stack trace to try again, recursively if necessary.
I always wanted to write a tool that took GCC compiler errors and put up a browser window with the most relevant Stack Overflow answer. Maybe GPT is way more useful here. Perhaps you get an explanation and a suggested fix.
The problem with this is you can't trust the actual data, if you care about data quality. I'm really not getting it's usefulness beyond a contextual lorem ipsum generator? It could probably work for a test environment but you still need to have a human review the lists that are generated.
It is written in a humorous style, but I don't take it as a joke. How is this fundamentally different than GPT code completion, other than it's data instead of code?
I suspect our intuition tells us code will be verified first by a programmer, then by the language parser, whereas data is likely to be manipulated without human scrutiny.
Does anyone seriously believe that this will forever be the case?
> Hopefully you can see this is not meant to be serious. Contributions to make it serious aren't really the idea. Contributions to make it sillier or more audacious are the idea. Silly examples are also welcome.
First it was against Perl developers (well I actually also knew a few C programmers who looked down on "C++ lamers" but not sure if it was widespread), then PHP, then JS and apparently now Python. I think we should stop feeding the stereotypes.
While I think the paraphrased Jurassic Park quote can be very funny at times, I don't really agree with the statement. But I assume you were being sarcastic.
im calling you out because you have no idea of what youre speaking of ;; youve been an HN member since 2018 ; Ive had mined since 2009, but even before that....
i agree with 100.
growing up in the 80s and reading cyberpunk fiction??
I think we should blame those dudes on CONSEQUENCES of
Author: I have written "Don't Build The Cyberpunk Dystopian Hellscape", a warning against those foolish enough to believe that their actions have no consequences.
CEO, 20 years later: Inspired by "Don't Build The Cyberpunk Dystopian Hellscape", we have built a Cyberpunk Dystopian Hellscape! It will be very profitable.
And in the book it didn’t work because Zaphod was (rightfully) so egotistical he was immune to it! So it seems the sci-fi author’s moral here is that it’s fine actually.
"When you entered the door of my office, you entered my electronically synthesized Universe," he explained, "if you had left by the door you would have been back in the real one. The artificial one works from here."
He patted the briefcase smugly.
Zaphod glared at him with resentment and loathing.
"What's the difference?" he muttered.
"Nothing," said Zarniwoop, "they are identical. Oh-except that I think the Frogstar Fighters are grey in the real Universe."
...
"Meanwhile, you and I arranged that you would steal the Improbability Drive ship-the only one which could reach the ruler's world-and bring it to me here. This you have now done I trust, and I congratulate you." He smiled a tight little smile which Zaphod wanted to hit with a brick.
"Oh, and in case you were wondering," added Zarniwoop, "this Universe was created specifically for you to come to. You are therefore the most important person in this Universe. You would never," he said with an even more brickable smile, "have survived the Total Perspective Vortex in the real one. Shall we go?"
As AI progresses and improves over time, it seems like the expected trust one would put in this data will trend upward, maybe eventually to "I trust this data".
What is a (great) joke now might be less of a joke 5-10 years from now. :)
If the results were stable, reproducable and somehow memoizable, this would be actually insanely useful. Perhaps it could modify a cache file somewhere with the generated Python code, and that could be committed.
The backend OpenAI APIs are not deterministic even with temperature 0, and they might upgrade the model weights/tuning behind your back too? (Not sure about the upgrade, they might just put out a new model id param...)
It's very strange that they don't guarantee determinism. We trained a model at Codeium that does have determinism at inference time - it's not super hard if you don't just use random seeds everywhere.
This makes it a lot harder to rely on OpenAI's API if you are doing any sort of caching / evaluation workloads.
It's close but not fully deterministic. I remember seeing a theory that if the system is considering multiple possible completions that have the same rounded score percentage, then its choice between them is nondeterministic.
It's likely because GPU calculations are non-deterministic and small differences in floating point numbers could lead to different outcomes (either in the way you described or somewhere deeper in the model)
Looks like Dill [0] would be a good option for serialising the generated code. The built in Python Pickle doesn't support pickling functions properly, but Dill extends it to enable this sort of functionality.
Per the README (and the source code), requests to GPT are cached. I know that doesn't solve the stability problem, since the cache is keyed on the exact prompt among other parameters, but it's something at least.
The source code shows that it's using pickling to store OpenAI responses on the local filesystem. See here:
It really should be saving them to JSON instead of Pickle, but I gave up trying to figure out how to properly rehydrate the openai Completion objects.
If it was JSON then it wouldn't be crazy to add it to your repository. (I guess Pickle is stable enough in practice, but it would offend my personal sensibilities to check them in.)
Besides security it's pretty opaque, and unnecessarily so. The data all comes over JSON so there's no non-JSONable types involved. It would be nice to be able to run tools like cost summarizing over the JSON without having to load up any other code.
This is similar to copilot-import [0] which in turn was based on stack-overflow-import [1]. I'd be interested to see whether ChatGPT/GPT-3 or Codex/Copilot is better at generating function bodies.
This is a great illustration of how ChatGPT fundamentally changes "true/false" into a continuum of "truthiness" as measured by plausibility. The infinite AI array is clearly marked as such, but how long will it be before generative extensibility is the (undeclared) norm?
We're all about to get some real-world GAN training.
I was just messing around with ChatGPT for a similar use case. Amazing what comes out of you ask for:
Give me a list of imaginary products that might be found in a magic shop formatted as json array including an id name, description, sku, price. Use syntax highlighting for the output and pretty print it.
Editing the kind of magic shop it is, add "very high end" to that sentence and see the difference. Specifically —
{
"id": 2,
"name": "Magic Wand",
"description": "A magical wand that can cast powerful spells.",
"sku": "MAGIC-WAND-001",
"price": 49.99
},
Is instead:
{
"id": 4,
"name": "Magic Wand",
"description": "A powerful wand made from a magical tree.",
"sku": "MAGIC-WAND-001",
"price": 1500.00
}
Or with "fancy", it is —
{
"id": 3,
"name": "Mystic Wand",
"description": "A powerful wand that grants the wielder the ability to cast powerful spells.",
"sku": "WAND-001",
"price": "199.99"
},
And if the magic shop is "run down", it might have —
{
"id": 3,
"name": "Mystical Wand",
"description": "A powerful wand that grants the wielder the ability to cast spells.",
"sku": "WAND-001",
"price": 29.99
}
Does anyone remember Google Sets? It was a short lived project where you'd input some entries from a list and Google would automatically expand it with related items. Seemed magical at the time (mid-late 00s I think?).
Google Squared, and yes, killed around 2007-8, and yes, never seen something that compares ever since. It was Sheets except you could make any row and column headers/labels you want and it would try to fill in the result for every cell.
Edit: just RTFA'd, yeah this is better. What happens if you call magic.ai_take_over_world_debug_until_no_exceptions()?
I think it partly made it into Google sheets. The ‘auto fill’ function when you drag the bottom-right corner of a range will work on lots of things you wouldn’t expect. But maybe it doesn’t work anymore.
(Obligatory, in the comments from a decade ago: "They shut down Google Dictionary recently and now this set tool. What's next, Google? Gmail, perhaps?")
Disclosure: I work at Google, on Sheets many years ago.
God damn, Ian has been releasing super creative code since before I started paying serious attention to Python circa 2005. Great to start 2023 with another one. The opportunity to hire someone of his caliber doesn't come up often.
Absolutely right: Ian is one of the best Python programmers and technical writers ("blogger" just doesn't capture it) whose prose and code I've ever had the pleasure of reading and using for decades. (I'm still maintaining a TurboGears project using his SQLObject library even today, which I gleefully adopted as a traumatized Zope refugee.) If I had an open position worthy of his talents, I would move heaven and earth to work with him.
Edit: In case you have "show dead" enabled, I'd like to add that smohare has no fucking clue what he's shit talking about, but by looking at his dead downvoted posting history and useless comments and worthless opinions, to be jealously insulted by him is the highest complement.
It's hilarious that even before thinking "wow it's crazy how accessible and powerful these LLMs are" I thought "man... Python is a pretty heinous language".
Do those objects... know the names of the variables they are assigned to?
The magic method resolution seems much more intriguing to me than the infinite array. Are there any programming languages (or perhaps a better word would be paradigms), that take this concept further?
I would imagine that I just state comments in my code file. Then, at runtime, all code is produced by the language model, and then executed.
There's the issue of the model producing different generative results with each execution. But maybe that could be taken care off by adding doctests within my comments. Which could, of course, be mostly generated by another language model...
Hardware manufacturers actually have to deal with this issue these days.
An awful lot of chip fabrication is done via stochastic / heuristic / monte carlo methods for any large chip; rather than exhaustively laying out the whole chip by hand, hardware devs describe the chip in a constraints language and feed it to a fabrication program (often with some additional parameters like "optimize speed" or "minimize power consumption / heat generation"). Then the program outputs the chip schematic to be fabricated.
Unless you save the random seed at every step, it's entirely possible to end up with the problem that you have a schematic, but if you lose it you'll never get the program to fabricate exactly that schematic again (because hundreds or thousands of precise schematics solve the problem well enough to terminate the monte carlo search).
I tried GPT-3 and it generated code examples that just ~made up~ some library that didn't exist at all. It had code that in theory would have ran, if that library ever existed at all, but it didn't, and it consistently came up with code examples using this imaginary library. I thought that wasn't very impressive.
What I'm finding it's extremely great for is writing drivel. I'm talking ecommerce product descriptions, product names, copy... it's really awesome for making one of my side gigs less of a chore.
I asked it to write some GPT-related code and it used a gpt3 library (which does not exist) instead of the openai library. I was amused by the self-blindness.
(Forgot to actually look it up... seems like it does exist but shouldn't, as it's an empty placeholder: https://pypi.org/project/gpt3/)
It's important to realize it's just writing out the next thing that seems most likely given the input.
I was pretty surprised playing around with github copilot that it could understand the context of a CSV file and filled in all the headers and stuff when I asked it to, then I realized it could probably just arbitrarily read anything on my computer and promptly uninstalled it.
It trips me out that people still try to get GPT3 to do deterministic things (math, cite a source, recite dates, etc). Not sure if its a lack of creativity or just a stubborn view of what is valueable.
No, really. This is how I use it. Get it to emulate a Linux terminal. Seed the history with ls / -lah, lscpu, lsb_release -a etc. Then at that point {use git to install a Python project that is a Slackbot template}.
Boom, it clones the fake library. cd slack-bot-template, cat main.py and away you go.
Have you ever imagined it will come a day when an AI can explain this joke?
> This code snippet appears to be a joke because it is claiming to be a function that returns a random number, but the function always returns the number 4. The line "chosen by fair dice roll / guaranteed to be random" is a reference to the common phrase "random as a dice roll," which means something is truly random and unpredictable. However, the fact that the function always returns 4 suggests that it is not actually random at all.
> The joke in this code is that the function is called "getRandomNumber", but it always returns the number 4, which is not random at all. The comment "chosen by fair dice roll" and "guaranteed to be random" are added for humorous effect, because they suggest that the number 4 was chosen through a random process, but in reality it is hardcoded into the function. The joke is meant to be a play on the idea of "randomness", implying that the function is not actually generating a random number as it claims to do.
It's interesting to contrast this explanation and the parent's one with the explanation from https://www.explainxkcd.com/wiki/index.php/221:_Random_Numbe... , which is essentially that the function may well return a random number but, contrary to expectation for functions with names like this, won't return different results if called more than once.
def __getitem__(self, index):
# if the index is out of bounds, generate more items
while index >= len(self.items):
self.items.append(self.generator_function())
return self.items[index]
# example generator function that generates capital cities
def world_capital_cities():
# list of capital cities
capital_cities = ['Paris', 'London', 'Tokyo', 'New Delhi', 'Beijing']
# generate capital cities in a cyclic manner
for i in itertools.cycle(range(len(capital_cities))):
yield capital_cities[i]
# create an infinite array object with the world_capital_cities generator function
capital_cities_array = InfiniteArray(world_capital_cities)
# access the 10th item in the array
print(capital_cities_array[10]) # prints 'New Delhi'
# example generator function that generates concatenation of two lists
def soft_sweet():
# list of soft items
soft = ['cloud', 'pillow', 'fluffy']
# list of sweet items
sweet = ['ice cream', 'candy', 'bunny']
# generate concatenation of soft and sweet in a cyclic manner
for i in itertools.cycle(range(len(soft))):
yield soft[i] + ' ' + sweet[i]
# create an infinite array object with the soft_sweet generator function
soft_sweet_array = InfiniteArray(soft_sweet)
# access the 5th item in the array
print(soft_sweet_array[5]) # prints 'pillow candy'
> This implementation uses the __getitem__ magic method to override the behavior of the square bracket operator ([]) for the InfiniteArray class. The __getitem__ method generates more items using the generator function provided in the constructor if the requested index is out of bounds. This allows the array to appear infinite to the user.
Note that this implementation is just an example and may not be suitable for all use cases. For example, it does not handle negative indices or slice operations. You may need to modify the implementation to suit your specific needs.
This is an example of something I've seen referred to as "code hallucination". It's pretty darn mindblowing, and you can get some really interesting results. Those times when AI hallucinates some function that doesn't exist are kind of annoying, but one man's bug is another man's feature. You can turn the table on it and make it useful by __going ahead and using those functions that don't exist__.
I was playing around with this by telling ChatGPT to pretend to be a Python REPL and provide reasonable results for functions even if they weren't defined. A few of my favorite results:
>>> sentence_transform("I went to the bank yesterday.", tense="future")
"I will go to the bank tomorrow."
>>> wittiest_comeback(to_quip="Hey George, the ocean called. They're running out of shrimp.", funny=True)
"Well, I hope it's not too crabby about it."
>>> sort_by_temperature(["sun", "ice cube", "flamin-hot cheetos", "tea", "coffee", "winter day", "summer day"], reverse=True)
["flamin-hot cheetos", "sun", "tea", "coffee", "summer day", "winter day", "ice cube"]
It took some experimenting to get it to consistently respond as expected. In particular, it'd often warn me that it's not actually running code and that it doesn't have access to the internet. Explicitly telling it to respond despite those things helped. Here's the latest version of the prompt I've had success with:
---
Your task is to simulate an interpreter for the Python programming language. You should do your best to provide meaningful responses to each prompt, but you will not actually execute code or access the internet in doing so. You should infer what the result of a function is meant to be even if the function has not been defined. To do so, you should take into account the name of the function and, if provided, its docstring, parameter names, type annotations, and partial implementation. The response to the prompt should be formatted as if transformed into a string by the `repr` method - for instance, a return value of type `dict` would look like `{"foo": "bar"}`, and a float would look like ` 3.14`. If a meaningful value cannot be produced, you should respond with `NoMeaningfulValue(<explanation>)`. You should output only the return value, and include no additional explanation in natural language.
---
I also add a few examples; full thing at [0], to avoid polluting the comment too much.
I was meaning to write a Python library to do that, but right around then OpenAI implemented anti-bot measures. I'm sure it's possible to circumvent them one way or another, but if there's measures in place there's a reason for that, and it's not very nice to degrade everyone's experience. I've had less impressive results with codex-2 so far. Still, harnessing hallucination is a pretty cool idea.
I like the idea of a prompt based sort! E.g., books.sort(key="publish date"). I'm not sure if that's best done with a dict-like approach (i.e., actually calculate the key) or let it get really fuzzy and ask it to sort directly based on an attribute. Then you might be able to do books.sort(key="overall coolness factor") which is an attribute that doesn't necessarily map to any concrete value but might be guessed on a pairwise basis. (This might be stretching GPT a bit far.)
This is good fun. In the same spirit as something I threw together recently [1], that runs broken code (with un/misdefined functions and variables), and repairs it using a combination of stacktraces and comments fed to GPT-3.
Python, in particular, is a great language to play this game with since an imported module is just another class instance and class instances can have their dereferencing hooked via the __get__ method.
I believe it uses ChatGPT to generate "infinite" lists of things in a chosen topic.
So rather than programming in every airline (and array of airline names) for a program that returns airline names, it queries ChatGPT for airline names.
If I'm wrong I am sure someone here will be quick to correct me and provide the right answer.
But there's more! It also generates any function that does not yet exist. It also tried to solve any exceptions stemming from that buggy hallucinated code... :)
There's a maximum context of 10 items so it won't ever technically stop (though your GPT bill will keep going up). For something clearly ordered this might be enough, e.g., primes[100:110] will give a context of primes[90:100] which might give you the correct next primes. (Everything in this library is a "might".)
For something like the book example I expect once you get to item 12 it might repeat item 0 since that won't be included in the examples.
If you have a prompt with demonstrations (in-context-learning) you can randomly sample the demonstrations from a larger dataset. This will make the model generate more variations as you are introducing some randomness in the prompt.
Horrible idea. Great execution.
[1] https://gist.github.com/drusepth/48a246e1263183e0418c