I once worked on AI Dungeon and we had a similar idea to parse the story so far into a graph, so that we could manage long-term memory outside of the context window (which was only 2048 tokens).
Coreference is hard. ("he took the sword"... who is he?) Updating the graph is also hard. (As the story progresses, new facts contradict old facts. Jenny was dating Tom, but now she's dating Mike.)
And knowing what to do with the knowledge graph is hard too, especially if you don't know the schema up front. The only thing we could think to use it for was... programmatically turning relevant sections back into text and prepending it to the context window. (There were easier ways to get a similar effect.)
It's really fascinating hearing about this and what the issues were. I have played a lot of AI Dungeon on and off and this always felt like part of what was missing: some way for it to keep a structured view of the story to help consistency. The biggest problem has always been that it keeps contradicting itself or lose track of the plot. It's gotten a bit better with the manageable context being fed back each step, but it's still not nearly good enough.
Handling state (especially long-term) is really a struggle for LLMs right now. This issue should become easier to work with as context windows scale up in the next couple years (or months, who knows!).
This is the most excited I've ever been sequence models! If the claims the H3 (and S4) authors are true then we are on the cusp of something very big that will provide another quantum leap in LLM performance. I worth that the claims may come with a hidden catch, but we just have to work with these systems to know.
I'll venture that once truly long range correlations can be managed (at scales 100-1000x what's possible with current GPTs), all the issues about logical reasoning can be answered by training on the right corpus and applying the right kinds of human guided reinforcement.
Using tokens as context still sounds to me like you're asking someone to read back text that someone else wrote and continue the story. It might work but it's not the best way to get a coherent narrative.
It's possible to model by annotating facts in the database with a timestamp (Wikidata has this, as well as qualifiers for e.g. the source of a statement, or that it applies within a restricted context) but you still need to somehow integrate the information if you want to know the state right now. E.g. if you have (Jenny, date, Tom) from a year ago and (Jenny, date, Mike) from yesterday, does that mean (Jenny, date, Tom) is no longer valid? Or are both simultaneously true? Or is (Jenny, date, Mike) invalid too, because yesterday was like ages ago?
You could have some heuristics to handle this and then you add another relation "has met" and suddenly you need a whole new set of heuristics.
you can have a date_start and date_end to handle this ambiguity. but yes the complexity lies in the interpreter/reasoner that has to deal with these facts and evolution of this (meta)schema.
But rdf style and labeled property graph data modeling approach have multiple ways of dealing with this.
Correctomundo. See RDF-Star for progress about state-in-time. During summer 2022 there was extensive discussion/consideration in the W3C working group about different state-conditions.
It makes me sad that many people will use this as an opportunity to write off Buddhist practices. Please don't! It has personally helped me greatly. Just remember:
1. The goal is to end suffering, so if a practice is making you suffer stop doing it.
2. Do what works for you. The only truth is what you can directly experience in the laboratory of your life. All the other teachings are just suggestions.
And if you'd like a concise overview of Theravada Buddhism (which is somewhat easier to grok without the added teachings of Mahayana, Vajrayana, Zen, etc.) I highly recommend [1] and [2], the second of which can be read in a day or two.
> It makes me sad that many people will use this as an opportunity to write off Buddhist practices.
This would be a huge misunderstanding of the article. I read the whole article, and I found it to be positively re-affirming of Buddhism. Unfortunately the submission title is potentially misleading and possibly even clickbaity.
It's a very good article. I've read many critiques of Western Buddhism and they can mostly get a bit samey after a while. However -- after the first few paragraphs, which are admittedly pretty run-of-the-mill -- this article elevates itself to a much more interesting level. It also raises solutions, anecdotes, and references that go way beyond what you normally hear.
I disagree that it's a very good article or even a good article. Its conclusions stand in stark contrast to the fundamentals of Buddhist practice. It's message is confusion from an author who is not accomplished in practice but writes eloquently.
I think it's unfortunate its gone to the front-page because it makes a mess of the dharma and communicates Buddhism as a mass of contradictions.
People will latch on to anything that confirms their biases without practicing and seeing for themselves what it really is. I encourage more people to go out and practice the way we have the teachings preserved not in Zen, not in Tibetan but early Buddhism, where we're closest to the teacher. Start there, practice ethics first, then move on to stability of mind, then real insight practice always with an eye to, "is this thing improving my relationships to myself and others?" If it doesn't, if you become more egoic, more like Sasha, abandon it, seek help. Do not declare enlightenment, do not foster a following of people who will then spread your confusion to the four corners of the earth.
I've never seen so much disagreement about "what is Buddhism" as I have in online Buddhist communities. Everywhere I turn there is conflict, misunderstanding, and people proclaiming "this is not real Dharma, but I know what the real Dharma is".
I do however agree that a lot of folk seem to get into esoteric stuff like Tantra and Dzogchen very early before understanding basic things.
> Its conclusions stand in stark contrast to the fundamentals of Buddhist practice. It's message is confusion from an author who is not accomplished in practice but writes eloquently.
Some famous Buddhist teachers are a bit like this to me. Chogyam Trungpa, for example... I can't make head or tail of the man. Sometimes I think Crazy Wisdom is great, other times I think he's been a disaster for the spread of Buddhism to the West. I raise the spectre of CTR because the article we're discussing has a Crazy Wisdom feeling for me, and that's why I like it.
> I think it's unfortunate its gone to the front-page because it makes a mess of the dharma and communicates Buddhism as a mass of contradictions.
The whole Internet is a mess of contradictions regarding Buddhism. It is literally the worst place you can go to learn about what Buddhism really is, because the only way you learn properly is to trust a teacher by knowing them on some kind of personal level or teacher-student level.
For a tradition with such emphasis on sangha, practice, and lineage of teaching, it's ridiculous to submit to the thoughts of online Buddhists who you don't even know. I think this is a reason for all the disagreement; arguments are conjured up out of thin air with no reference to the actual practice of the person who says it.
I'm not saying the article is correct in all aspects, but rather that it goes beyond the trite and predictable criticisms of Western Buddhism, and for that alone it's more interesting than most articles. It raises anecdotes that I haven't much of before, it's slightly abrasive, and I like that.
> I encourage more people to go out and practice the way we have the teachings preserved not in Zen, not in Tibetan but early Buddhism, where we're closest to the teacher.
This is a very Western Buddhist attitude. In the Western tradition we love to get as close as possible to the primary canonical source when learning something, possibly because Western Buddhism tends to be quite dry and academic and admits very Protestant attitudes. I think a lot of Western Buddhists have this mistaken notion that Theravadans have some monopoly on what Real Buddhism actually is.
My most recent teacher said it doesn't matter whether you start out in Zen, Pure Land, Tibetan, or Theravada, just pick one and stick with it for a while otherwise you'll get your wires crossed and end up getting a jumbled message that is inconsistent with any sect. He acknowledged the desire for people to study early Buddhism as the "purest" form of Buddhism, but he said that IF you want to study early Buddhism or Theravada, then you need to stick with it. So, much like you're saying, yes the Theravada teachings are more direct and simpler, but maybe it's not necessary to start out with this form. Maybe it is, I don't know. I'll figure it out as I go along. But I don't disagree in principle.
Personally I like the simpler, warmer approaches to Buddhism, and I do often appreciate irrational, supernatural aspects when I encounter them. I do appreciate Theravada, not because it's the purest form, but because the practitioners I've met so far seem to be a bit warmer. Plus, Theravada is a bit more like, "do this, do that, keep it simple".
I've been very curious about Zen but I notice that a lot of Zen practitioners tend to deflect real-life problems into clever Buddhist aphorisms, which is just evasive and doesn't really engage with the world. Plus I don't really love meditation that much so maybe it's not for me. I'd prefer to do active physical exercise in my limited time (and when I'm not writing on HN) because I know this benefits my mind and body more than extended sitting. On the other hand, I like Alan Watts a lot, I think he had had a much more positive influence on Western Buddhism than CTR had. So AW a good, positive advertisement for Zen.
Pure Land sounds cool because I like chanting, but I like Nichiren chanting more (plus I met a lot of cool people in Nichiren circles). Tibetan Buddhist sanghas around where I live seem to be a bit, oh I don't know, I just can't really get into it.
Yes, I hope that most readers take away this sentiment as well.
I will say that some of Sasha's criticisms don't vibe well with my understanding, though. For example, we don't renounce pleasure because Buddha tells us we should, but because we directly observe that clinging to desire inherently feels bad. Our subconscious mind, when confronted with this info, naturally drops the object like a hot coal.
Telling newbies that traditional Buddhism is about giving up everything that makes you happy is a mischaracterization imo.
I also think his understanding of clinging is more narrow than what's described in the Abhidhamma. The interludes of TMI[1] give a pretty approachable introduction to this model of conscious experience.
> 1. The goal is to end suffering, so if a practice is making you suffer stop doing it.
That's correct: the goal is to end suffering - by means of understanding it.
Thus the understanding is important to distinguish there's indeed suffering that leads to the end of suffering -- like Ajahn Chah using analogy of going to the dentist (in itself is a suffering) to end the suffering of dental pain.
I've been curious about this in my own practice, but I haven't yet been able to find an example of increasing suffering as having a positive impact on my insight.
Do you remember the book or teaching that that analogy came from? Ajan Chah comes highly recommended from others as well.
Edit: oh I guess the exception is clinging to the Dhamma, like the sutta about the raft.
> Do what works for you. The only truth is what you can directly experience in the laboratory of your life. All the other teachings are just suggestions.
We should all follow this, and also use it as a basis for being more kind to others - what worked for us might not necessarily work for others, let’s be less judgemental and more understanding.
That's not what I got from the article at all. I got that Bhuddism is vastly more complex than I've realized, and that there's good and bad parts about it, and the author himself gives pointers on teachings/directions that he likes.
Criticising Buddhism has it's place but the average westerner is not at a level where it's appropriate to do it. 99% of people need more buddhism, not less.
the goal is to end suffering if you believe that suicide won't work because you'll just get reincarnated. their real goal is to end reincarnation. if you don't believe in reincarnation, buddhism is useless.
> The goal is to end suffering, so if a practice is making you suffer stop doing it.
That would run counter to what Zen practice is about. Suffering can be, among other things, an unconscious experience of pain. If the practice brings pain, whether physical or psychological, the way to deal with that is to continue with the practice and be unshaken by whatever comes, just staying with the practice (i.e. counting, or staying watchful) - while avoiding a desire to be in a better state or questioning the current state, just keeping on doing it under a firm belief that it will lead to an improvement in the long term (and not necessarily right now). Doing so leads to a development of an internal balance, sort of a way to operate beyond thoughts and emotions, which allows to blow away unnecessary mental clutter that contributes to suffering here and there on an ongoing basis, as well as having more mental strength to deal with unpleasant things in life with less suffering.
I think we are mostly on the same page. Pain and suffering are different things, and often fully embracing pain and letting go of aversion decreases the amount of suffering in that moment. So yes, it's very important that practice is motivated by wise consideration of the true nature of suffering, and perhaps I should have qualified that.
However, given that qualification, I still believe that practice should be a smooth descent of the gradient of suffering. You may increase your conscious awareness of pain or unwholesome states, but you should always feel less suffering after you do that. This is because letting go to the reality of the present moment is the opposite of clinging. So you should always feel better* when you practice than when you don't.
I will admit, though, that perhaps my perspective is colored by my experience and this is not true for all people.
*by feeling better we mean less suffering (dukkha), not more pleasure and less pain
Anyone who benefits positively from Buddhist practices will be unperturbed by this article (as you seem to be). That's fine; but suggesting that _everyone_ give it a chance; even in the face of (frankly valid) criticism, is toxic in my opinion.
> Do what works for you. The only truth is what you can directly experience in the laboratory of your life. All the other teachings are just suggestions.
This is just your own opinion at this point. It's not buddhism. Buddhism has specific teachings that it posits as being true, regardless of whether you've personally experienced them.
And a few other mental disorders, yeah. "Depersonalization" is the relevant search term.
This was the real "sudden insight" I got from trying meditation off and on for a few years: that the complete detachment they try to achieve and the complete detachment I've been feeling for most of my life are the same thing. The only difference is they did it on purpose.
I have been enlightened, and it is the biggest source of suffering in my life.
Not detachment from pain, you misunderstand. You will feel all the pain just the same. It's just that pain won't imply the same amount of suffering, as you experienced before. In this context, pain is what you feel, suffering is what you make of it.
What you are reminded of, regarding ptsd and detachment, is called dissociation (in psychology). It's an unhealthy, but working, coping mechanism for something that a mind decides it cannot face. A major difference with the detachment I'm trying to explain is that the Buddhist / stoic practice is conscious, but dissociation is unconscious.
I think you can get the effect you're looking for by using the previous cell as an init image and only repainting the character.
As for consistency of character details, I think that will depend on how many images you use to train dreambooth etc. and how varied those images are.[1]
I haven't used a feed reader in a long time, but I had a brief period when I was obsessed with Fraidycat. Worth a look if you're interested in a different approach to keeping up with people.
We do this with Pinecone, but we use CLIP embeddings of images, and they work incredibly well. It's kind of crazy how easy it is to get semantic search of images these days.
CLIP also does caption embeddings, so you can lookup images via both images and captions.
Seconding the recommendation of CLIP embeddings, especially compared to image histograms + requiring OpenCV.
I wrote a naive, minimal dependency Python package to calculate image embeddings (https://github.com/minimaxir/imgbeddings) with some lookup demo notebooks and it works well in a pinch, although it's due for an upgrade.
On macOS (display: 15.4-inch, 2880 × 1800), it's really difficult to read. I set the font to ''400 1.2rem/1.5 "Fira Sans",sans-serif'' and color to #111 in dev tools, way better readability.
(sidenote: is Fira Sans a default installed font on Linux systems? I'm on macOS and don't have that, and don't see a font embed anywhere in your source code. So that might be the issue - 'sans-serif' at 200 weight is way too faint)
I once worked on AI Dungeon and we had a similar idea to parse the story so far into a graph, so that we could manage long-term memory outside of the context window (which was only 2048 tokens).
Coreference is hard. ("he took the sword"... who is he?) Updating the graph is also hard. (As the story progresses, new facts contradict old facts. Jenny was dating Tom, but now she's dating Mike.)
And knowing what to do with the knowledge graph is hard too, especially if you don't know the schema up front. The only thing we could think to use it for was... programmatically turning relevant sections back into text and prepending it to the context window. (There were easier ways to get a similar effect.)