Does the massive context still needs to be dragged around? Until get a neural network that adjusts weights in real time without relying on a huge clump of context being cycled there, I don't think there will be an AGI or even an impressive "agent". Current agents are just LLM looping lmao sold to people with no knowledge how they work at all.
I've been wondering about context compression, actually. I remember a bunch of prompt compression tricks from a couple of years ago that must be usable. If you were to say "anything past half context gets compressed, including the previously-compressed context" would mean you'd still have reasonable workspace and potentially infinite recollection of the most important things. Then yes, you'd be dragging a massive context around, but you're maximising the return on using it. I presume somebody's got a toolkit for this that I don't know the right terms to google for.
Compress first half (even recursively) is a nice idea, but you'd have to sort it by importance first – ie. you don't know if the most important sentence didn't exist at the beginning (ie. the whole goal could have been stated at the beginning, you don't want to loose it – you want memory to fade in terms of importance instead).
I don't know that, but the LLM might be able to make that judgement. If it's involved in the compression process, it can choose to drop cruft from the middle. Maybe "summarise then compress" is a better description.
There's been some work on memory lately like Transformer² and Titans. But that may not be necessary for decent agents. Even the "context in a loop" is getting better as general reliability of tasks increases, and as people find better ways to manage context. Like Cline has the Memory Bank prompt where there's a folder of markdown files at diff levels of abstractions of info on the project, current feature, current task progress, etc. And so it can update those files and read from them as a consistent memory.
There is https://www.rwkv.com/ which is an LLM based on RNN's, thus having "infinite" context length, it comes with its own tradebacks though. (Notably that its impossible to actually store infinite information in the network, so it prunes based on which information it finds more important.)
Yeah they really like to mention it everywhere, like yeah it's good but imo not as good as some people make it out to be. I have used it recently for libgdx on kotlin and there are things where it struggles, and the code it sometime gives it's not really "good" kotlin but it takes a good programmer to know what is good and what is not
My mom just passed from ALS and now this. It's so unfair that we have essentially no working treatments for these, even though they always tout how advanced medicine is but I feel like it really is not advanced at all
It's mostly nocebo, they read about it online for weeks and then try it, and at that point they are so psyched out that they keep watching for every little thing, effectively making it a self fulfilling prophecy.
I started dutasteride when I was 20 and have been on it for many years and it really stops hair loss, because my hair loss was very fast and aggressive.
So if someone after two weeks had their semen consistency change to be completely watery, would it be their eyes that produced the nocebo effect or? Or how is the doctor who went on it for a few months, lost erections, then gained them back after quitting explained by a nocebo?
Again, I've heard at least 5 of these first-hand, which is kind of wild for a drug I've probably only talked to 20 people about in total.
We're terrible at detecting changes over long periods of time. If anything I'd think long term users would be much more liable to bias.
I don't really get it though? So he is basically arguing that there is nothing we can do and the maximum age is set in stone? I mean yes, without intervention it kind of is. But I wouldn't bet that in some time there will be an intervention or effective suite of interventions that can reverse aging. Will it happen soon? No probably not, I don't believe that in my life time and I am in my mid twenties, but at some point yes.
It would be nice though if they could come with stuff so we could all die like Queen Elizabeth, very fast at old age without long term illness. I read it is quite common with centenerians.
My mom just died from ALS some months ago and there has been lot of money put into that research and only stuff there are is few lousy treatments that barely do nothing, same with Alzheimers so I am nowadays very skeptical of any medical research and mentally prepared that if I get something I am good as dead
I don't really get the military part tbh. When I went to army back in my country where it was only mandatory for men (but they always love preaching about equality), many of the men were such slobs that random women would have been much better suited for it.
I really don't understand these men who send the dick picks. Like does it actually work with some women? In my experience women tend to be very different to men in that regard, but I guess it has work some time because so many keep always doing it. I just find it really repulsive.
The purpose is not for it to “work”. The men who do this have not had success with women, and have grown frustrated and resentful. They feel powerless in this area. The only way they can reclaim power and feel better about themselves is to harass, intimidate, or harm women.
I'm going to copy my answer from zellyn in a thread some time ago:
"It’s been obvious even to casual observers like myself for years that Waymo/Google was one of the only groups taking the problem seriously and trying to actually solve it, as opposed to pretending you could add self-driving with just cameras in an over-the-air update (Tesla), or trying to move fast and break things (Uber), or pretending you could gradually improve lane-keeping all the way into autonomous driving (car manufacturers). That’s why it’s working for them. (IIUC, Cruise has pretty much also always been legit?)"
reply