No doubt there are many, many other varieties of humans which we'll never know about because we'll never find any bones.
What chilling is that shows very clearly that humans can go extinct. In fact, humans go extinct all the time: 50,000 years ago, we shared the earth with at least 5 different varieties of humans (Homo Neanterthalensis, Homo Denisova, Homo Naledi, Homo Florensiensis, and Homo Luzenenensis). Probably also some late surviving populations of Homo Erectus and Homo Rudolfensis. Almost certainly more we'll never know about.
Since the last ice age we've been dropping like flies. In fact, we're the only humans left. And what with climate change, accidental nuclear war, or the singularity looming, we are arguably in a more precarious position than we've ever been.
If you're new to this then just download an app like LMStudio (which unfortunately is closed source, but it is free) which basically just uses llama.cpp under the hood. It's simple enough to get started with local LLMs. If you want something open source ollama is probably a good place to look too, it's just a CLI tool but several GUIs integrate with ollama specifically.
As for your bonus question, that is the model you want. In general I'd choose the largest quantized version that you can fit based on your system. I'm personally running the 8bit version on my M3 Max MacBook Pro and it runs great! Performance is unfortunately a loaded word when it comes to LLMs because it can mean tokens per second or it can mean perplexity (i.e. how well the LLM responds). In terms of tokens per second, quantized models usually run a little faster because memory bandwidth is a constraint, so you're moving less memory around. In terms of perplexity there are different quantization strategies that work better and worse. I really don't think there's much of a reason for anyone to use a full 16fp model for inference, you're not really gaining much there. I think most people use the 4bit quants because it's a nice balance. But really it's just a matter of playing with the models and seeing how well it works. For example, some models perform okay when quantized down to 2 bits (I'm shocked that's the case, but I've heard people say that's the case in their testing), but Mixtral is not one of those models.
I once commented that HN is the most wonderfully diverse ecosystem and here's my chance to prove myself right! I'm a cork 'farmer' in Coruche, right where this article is situated. I wasn't expecting to read a puff piece about it today. I just did my novennial harvest last year. For anyone not in the know, cork is the cork trees' bark, and it's stripped from the tree without harming it every nine years. Undressing the tree is properly medieval work and you need to be very skilled with a hatchet to do it. Do a poor job and you'll ruin the cork and scar the tree for decades.
The harvest is tough work but it's the only well-paid trade left in agriculture. I doubt it has much future beyond fodder for high peasant magazine articles. Trees are dying left and right from multiple climate-related problems no one has a handle on. Divestment from the traditional montado like mine into intensive production units with better water management and automated extraction is the likely future. The billion-dollar outfits have started experiments with high-density groves, inspired by the olive oil industry's success. It's a finicky tree though, so conclusive results are taking a few decades more than you'd expect to materialise. They're stuck having to buy cork from thousands of traditionalist family farms for now.
But that's assuming the industry even grows enough to justify the investment into better plantations. Legitimate uses for the stuff apart from wine corks are scarce. We're all hoping that our phenomenal ecological footprint will see us grow as an industry into everything from insulation and roofing to shopping bags and umbrellas (hence said puff piece I imagine). We'll see, it really is a phenomenal material and the carbon math makes sense at the source. You can almost see the tree sucking out stuff from the air and soil to build thicker layers of bark. I joke that we've been doing regenerative farming for generations, we just didn't know it until someone told us.
If anyone on HN is ever in Portugal and wants to visit a montado, happy to take y'all on the most boring tour of your life. But we can have a nice picnic! It's lovely country.
Urgh. And it gets worse from there. The bugs list on the repo has a closed and locked bug report from someone claiming that their code is using teacher forcing!
In a normal recurrent neural network, the model predicts token-at-a-time. It predicts a token, and that token is appended to the total prediction so far which is then fed back into the model to generate the next token. In other words, the network generates all the predictions itself based off its own previous outputs and the other inputs (brainwaves in this case), meaning that a bad prediction can send the entire thing off track.
In teacher forcing that isn't the case. All the tokens up to the point where it's predicting are taken from the correct inputs. That means the model is never exposed to its own previous errors. But of course in a real system you don't have access to the correct inputs, so this is not feasible to do in reality.
The other repo says:
"We have written a corrected version to use model.generate to evaluate the model, the result is not so good"
but they don't give examples.
This problem completely invalidates the paper's results. It is awful that they have effectively hidden and locked the thread in which the issue was reported. It's also kind of nonsensical that people doing such advanced ML work are claiming they accidentally didn't know the difference between model.forward() and model.generate(). I mean I'm not an ML researcher and might have mangled the description of teacher forcing, but even I know these aren't the same thing at all.
Yes. Gato was a good proof-of-concept that the Decision Transformer approach of 'just model literally everything as a sequence' scales well and doesn't exhibit some sort of catastrophic interference and can successfully imitation-learn from all the expert datasets, and a bit of transfer. But they need to push it at least another OOM or 2 to show major transfer, some emergences, and ideally do both from-scratch learning and additional learning on many new tasks. We continue to wait. :(
I hope it didn't all get rolled up into Gemini and become a state secret they'll never publish on again, or lost in the shuffle in the chaos of the DeepMind/Brain merger/liquidation.
It also allows you to search for trades by stock, instead of needing to parse through thousands of disclosure forms.
I do want to mention one weakness of my data, which is that I don't currently parse hand-filed disclosures. Most politicians do electronic filings, which are easy to scrape, but some still file by hand. Working on a solution for that, which should hopefully be live before too long.
However it just sits there asleep at an extremely low wattage most of the time, then pops instantly to life the moment a page is sent to it - as bgirard pointed out, without even needing to install a driver - and then back to sleep. And this is on wifi.
Loved it enough to buy another one when the first one went to my ex.
I'll use the full duplex occasionally, it's worth having years down the track from purchase time for the tiny price bump over the base.
A related line of work is "Thinking Like Transformers" [1]. They introduce a primitive programming language, RASP, which is composed of operations capable of being modeled with transformer components, and demonstrate how different programs can be written with it, e.g. histograms, sorting. Sasha Rush and Gail Weiss have an excellent blog post on it as well [2]. Follow on work actually demonstrated how RASP-like programs could actually be compiled into model weights without training [3].
I'm not sure about chicken cells but my experience with growing human cells (WI-38 fibroblasts)[1] is that they want to grow in a monolayer.
The way we grew them normally was in those roller bottles [2]: you have some tissue culture medium in the bottle (containing fetal calf serum, which is still required to grow mammalian cells AFAIK, another reason most of these companies can't do what they say) and other components. Then the bottles are put on their sides on a roller mechanism, in an incubator with a controlled CO2 concentration.
The cells want to grow in a one-cell-thick monolayer all over the inside of the bottle. What they won't do is to grow a double layer. This is because they have "contact inhibition" which is a major mechanism for preventing tumor-like growth: when cells are touching each other, they stop dividing.
So these guys are trying to grow tissue and what they seem to be doing instead is growing monolayers just like this on the inside of roller bottles, scraping them off, then putting them together again.
One thing that disturbed me was that they need to coat the bottles with porcine gelatin. In that case, a lot of those "chicken" cutlets are actually pork!
There is one: "The agreement between GPT-4 and humans reaches 85%, which is even higher than the agreement among humans (81%). This means GPT-4’s judgments closely align with the majority of humans. We also show that GPT-4’s judgments may help humans make better judgments. During our data collection, when a human’s choice deviated from GPT-4, we presented GPT-4’s judgments to humans and ask if they are reasonable. Despite different views, humans deemed GPT-4’s
judgments reasonable in 75% of cases and are even willing to change their choices in 34% of cases."[1]
Interestingly, at Google the typical developer workflow (google3) is very cloud native.
Most devs write code in VS code in the browser.
Many (most?) devs don't have a physical desktop any more, just a cloud VM.
The code lives in a network mounted filesystem containing a repository.
The repository is hosted remotely (everyone can see edits you make to any file nearly immediately).
Builds are done remotely with a shared object cache.
Tests typically run in the cloud (forge).
Facebook has similar infrastructure, although more pieces run locally (builds were mostly done on your VM circa 2020)
For my personal projects, I try to do most development on a cloud instance of some kind, collocated with the rest of the infrastructure.
Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc.
Rolled out a machine learning model and trained it on the database. 99% of them vanished.
Next day, the machine didn't work and success rate was around 5%.
Found out, they have learned the trick and now using symbols from different languages to make it look like English.
Trained again, success rate went up again.
Next hour, success rate fallen.
This time, they mixed their content with other valid content of our own blogging platform. They would use content from our own blog or other people posts and mix it to fool the machine learning.
Trained it again and was success.
Once a while such content appear and machine model fails to catch them.
It only takes couple of minutes to mark the bad posts and have the model get trained and redeployed and then boom, bad content is gone.
The text extraction, slicing through good content and bad content, finding out symbols vs sane alphabet and many other thing was at first challenging, but overall pretty excited to make it happen.
Through this we didn't use any platform to do the job, the whole thing was built by ourselves, little bit of Tensorflow, Keras, Scikit-learn and some other spices.
Worth noting, it was all text and no images or videos. Once we got hit with that we'll deal with it.
edit: Here's the training code that made the initial work https://gist.github.com/Alir3z4/6b26353928633f7db59f40f71c8f... it's pretty basic stuff. Later changed to cover more edge cases and it got even simpler and easier. Contrary to the belief, the better it got, the simpler it became :shrug
Hi setib, great question. The original idea to use heuristics for preview cost reduction came out of a Hack Week project. This led to an initial brainstorm meeting between the ML team and the Previews team about what this might look like as a full-fledged ML product.
From the beginning the ML team's focus was on providing measurable impact to our Previews stakeholders. One thing that helped us collaborate effectively was being transparent about the process and unknowns of ML (which are different from the constraints of non-ML software engineering). We openly shared our process and results, including experimental outcomes that did not work as well as planned and that we did not roll into production. We also worked closely with Previews to define rollout, monitoring, and maintenance processes that would reduce ops load on their side and provide clear escalation paths should something unexpected happen. Consistent and clear communication helps build trust.
On their side, the Previews team has been an amazing ML partner, and it was a joy to work with them.
PostgREST is great and really reduces the boilerplate around building REST APIs. They recommend implementing access rules and extra business logic using PostgreSQL-native features (row level security, functions etc) but once you get your head around that, it really speeds things up!
If you're interested in playing around with a PostgREST-backed API, we run a fork of PostgREST internally at Splitgraph to generate read-only APIs for every dataset on the platform. It's OpenAPI-compatible too, so you get code and UI generators out of the box (example [0]):
The first trained multilayer deep neural network was presented by the Soviets in 1970 in a paper titled “Heuristic Self-Organization in Problems of Engineering Cybernetics“
http://www.gmdh.net/articles/history/heuristic.pdf
I used to work at a web agency, and my biggest impactful was without a doubt setting up something similar. Instead of starting from scratch at each new project, or copying config from another project, I wrote a "project generator" that asked you a few questions (Django or symfony? React or vue? NextJS? Terraform?) then generated a fully-configured project for you, including a staging and a production environment.
It's incredible how much time is lost on config: AWS is a big one, but having to configure Django, Black, Flake8, Webpack, ESLint, prettier, Typescript, Sentry, react-router... It easily takes days, and copying from other projects meant we were propagating some less-than-ideal setup.
Automating setting up a React-native app was clearly the most challenging, I don't work there anymore but I don't think it's a smooth process even now
Yes to this.
I often counsel my younger friends to have a "mission". Then once that is completed, to find another one. In fact, one can have multiple missions running at the same time.
It's rarely said that marriage, kids, job, degree, a house, car, and most physical things are not missions. And in fact, most of these things -- unless accompanied by a "why am i doing this" and attendant missions/goals do not lead to happiness.
Marriage without goals = divorce.
Kids without mission = abandonment.
Money without a plan = poverty.
Degree without a plan = debt/sadness.
Relationship without a mission = break up.
Programming without a mission = abandoned code.
Hey, that's me, might as well answer questions here.
I[1] do performance tuning for Microsoft SQL Server. The licensing costs on MSSQL Enterprise Edition are $7K USD per CPU core, so a typical single high-performance server costs $250K-$500K. Factor in hardware, multiple replicas (high availability, disaster recovery, scale-out reads) and it's not unusual for a company to have spent millions of dollars on their database server. (I know, I know, you think you can get a 4-core MySQL VM to perform the same. Let's set that aside for now and just focus on the article & the work.)
My basic sales pitch is that in 2 days, I can tell you why your queries aren't going as quickly as you want, and how to make them go faster. It's $5K USD done during the week, $15K weekends. Aside from the Christmas & New Year's weeks, I'm booked out until mid-January (with a couple of weekend gigs already scheduled.) For my clients, $5K is an irrelevant rounding error compared to how much they're spend on software, hardware, staffing, and how much revenue they're losing when their web site or enterprise app goes down.
I don't subcontract, I don't wear suits, I don't network with executives. I just have a very clearly defined product, sample deliverables online, a blog with a couple thousand posts dating back to 2002, and a very active community presence: I present at a lot of conferences. (I'm speaking at Data Saturday Holland this weekend.)
The linked post talks a lot about short calls and canceled gigs, but that's easily avoidable in this market: just require nonrefundable prepayment. In this high-value tier, clients tend to understand that they're buying a specific window on your calendar. If they don't show, I lost that time and can't get it back any other way, so to lock that day down, they have to prepay.
The best resource for this kind of work is the book Secrets of Consulting by Gerald Weinburg. Can't recommend it highly enough.
Feel free to ask any questions you want - I'm really transparent about the business. I'm on vacation in Amsterdam this week so my answers will be a little delayed during the day.
The definition's a working one, and subject to modification, but I feel it's a good start.
I've been collecting / compiling / commenting on similar veins for a while.
There's Robert Ewashuck's paging philosophy (Google SRE):
Pages [alerts] should be urgent, important, actionable, and real. They should represent either ongoing or imminent problems with your service. Err on the side of removing noisy alerts – over-monitoring is a harder problem to solve than under-monitoring....
There's the Japanese awareness and response following the bombing of Hiroshima. Short version: the first indication of the 9:15 am attack was a cessation of all communications. Military command wasn't aware of the scope of damage until an overflight was arranged, in early afternoon, and didn't know the actual cause until told by the US the next day.
In many large-scale natural disasters, an ominous sign is no reports, especially from outlying areas.
Initial responses to disasters often follow early stages of the Kubler-Ross stages of grief model: obliviousness, denial, sometimes anger. My sense is that grief is another form of major world-model collapse: that some reality that has been internalised and is often a sense of identity is shown to be wrong or inoperative.
Again: it's not that catastrophes and disasters are similar to grief, it's that grief responses are a response to catastrophes and disasters.
And finally, a quote from a ... modern Cupertino ... mystic, Adyashanti:
Make no mistake about it -- enlightenment is a destructive process. It has nothing to do with becoming better or becoming happier. Enlightenment is the crumbling away of untruth. It's seeing through the facade of pretense. It's the complete eradication of everything we imagined to be true.
On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
-Charles Babbage, Passages from the Life of a Philosopher, 1864
What I did a while back is download the raw data and run it through https://promethease.com/. Will tell you a lot more about risks, drug interactions. What it does is it compares your data to that of all the journal papers and research around that gene. Interestingly I have a SNP that is correlated with lack of empathy and another one that is the "warrior" gene. The BRCA SNPs would be identified.
License compliance is easily verified using 3rd party services like Black Duck.
You basically install a scanner on your machine, feed your sources into it, it hashes them line by line and sends hashes to the mothership for analysis. It then spits out the report that file X, line Y matches something in the open source package Z. At least that's how it worked a decade ago, when we had to do a pre-acquisition source code audit.
What chilling is that shows very clearly that humans can go extinct. In fact, humans go extinct all the time: 50,000 years ago, we shared the earth with at least 5 different varieties of humans (Homo Neanterthalensis, Homo Denisova, Homo Naledi, Homo Florensiensis, and Homo Luzenenensis). Probably also some late surviving populations of Homo Erectus and Homo Rudolfensis. Almost certainly more we'll never know about.
Since the last ice age we've been dropping like flies. In fact, we're the only humans left. And what with climate change, accidental nuclear war, or the singularity looming, we are arguably in a more precarious position than we've ever been.