> we are inspired by the recent advancements in reinforcement learning (e.g., o1)
It is interesting to see what the future will bring when models incorporate chain of thought approaches and whether o1 will get outperformed by open source models.
Some currencies use more than 2 decimal places. For instance, the currencies of Algeria, Bahrain, Iraq, Jordan, Kuwait, Libya, and Tunisia are subdivided into 3 decimals.
Bureaucratic malfeasance, error, or just plain bad luck, can loose people their accounts, even with government not silencing them.
e.g. a fly landing on a sheet of paper, blocking the print head long enough to generate "Tuttle" from "Buttle", resulting in a long chain of violent events for some unassuming individual…
This article and many people attempting to analyze this topic (and many others) are missing a key element: "incentives"!
Many developers are more passionate about technology itself – coding, solving complex technical problems, and creating efficient solutions. Their formal rewards (like promotions, bonuses, and recognition) are often tied to technical achievements. If a developer spends 40 hours on customer research, this effort might not be as formally recognized or rewarded as much as a technical achievement like developing a new authentication solution that benefits the entire team.
Shifting this dynamic is not just about encouraging developers to spend more time on product management. It requires a systemic change in how organizations value and reward.
Changing the incentive structure to be more user-centric and less technically focused would significantly impact middle and upper management roles. A shift would challenge the very foundations of their roles and responsibilities, potentially leading to a loss of control and influence.
It's likely that such changes would face subtle yet strong resistance from middle and upper management. These individuals often have significant political power within an organization and a vested interest.
This isn't that hard. Just have your developers train people with your software. In every craft you learn the moment the rubber hits the road. A film director can watch the film in the editing room a thousand times, but the one first time watching it with a real audience will teach them more than those 1000 hours together. A carpenter can work away on a piece of furniture merrily all week long and be happy with themselves, but the moment of truth comes when the furniture is placed into the customers room. The same is true for software. If you want your software engineers to care about users they have to get first hand feedback what works and what sucks, preferably with their own eyes.
Every software engineer worth their salt has a certain degree of pride about doing things well. If this only ever is about the code, the commit history or other abstract concepts, but never about how well it can be used by the actual people it was made for, then how would they even start caring?
I can attest to that. I had worked on a product for a couple of years and received feedback from our support team, but never seen how the product was used in real life. One time, we had a specific bug that the customers were complaining about, and no matter how they described it, we couldn't figure out what the problem was. That was until I went to one of the customer sites and saw how the users were interacting with the product.
When they tried to demonstrate the bug, it was immediately clear to me what the issue was. That was the first time I really grasped the difference in how we approach software products. It also helped me see how the product could improve their job by watching how the process was in real life. Gave me a lot of ideas on how to make their job easier by using the product.
Watching what and how they do after a month is as valuable. User interfaces have two sides in this regard: initial experience and boring repetition experience. You can optimize for the former, the latter or both.
> That was the first time I really grasped the difference in how we approach software products.
For me as well. I've started my "career" this way, building something very specific for a client (family member) whilst being on prem and having constant and direct feedback, so that's how it was done I thought.
Almost twenty years later, having worked different roles with teams in office and remotely, I have to remind myself, and others around me, that we are building something to be used by real people with real problems and my/your software must not be one of them.
“Incentives” if you do this you better make it pay so I never have to leave. If I have to leave to get a payrise then I am getting on the Omega Star team and slowing down their ISO timestamp roll out to get GraphQL on my CV, so I can hit levels.io and get that extra $100k TC. /s but Poes law applies.
No but "those people" is all people, to some extent. All people have life outside of work, like to make more money, see how unaffordable a house is and say "OK I need to earn $200k a year to raise a family the way I want". While they won't be so brazen, there is some effect of "doing thing thing that leads to the carrot". So the company needs to change that carrot - but it is hard because the whole industry has the carrots.
So what you are saying is you have to live with people who demand not dealing with the result of their work, because the rest of the industry is allowing for such behaviour?
Never before have I heard such a good argument for formalizing the job. If architects or civil engineers were to do the same we had building collapsing left and right.
Heh it's funny that your takeaway is "managers hate this". I know a lot of developers who would hate such a thing, thinking the only thing developers should do is write code.
Thanks for the comment. And I completely agree. Incentives shapes so much of how people work. I tried to capture that in the 3rd bullet, but your comment is more on-point :)
Unification should mean creating a single interface that seamlessly interacts with multiple underlying services without needing custom adaptations for each case.
The issue with allowing for customization is that it leads to fragmentation. This defeats the purpose of having a unified API in the first place.
I'm not sure whether the convenience of unifying leftover common functionalities like authentication outweighs the complexity and reliance introduced by adding another layer to the tech stack.
In practice, most customers we serve have use cases that could not be supported by traditional unified APIs, but are still better off not rebuilding an integration infrastructure from scratch.
We also offer professional services to outsource the customization, so you still have an off-the-shelf unified API, but specific to your company.
Fast Twitch Neurons (FTN) - running at nearest user location at $0.125 / 1k neurons
Neurons are a way to measure AI output that always scales down to zero. To give you a sense of what you can accomplish with a thousand neurons, you can: generate 130 LLM responses, 830 image classifications, or 1,250 embeddings.
Who came up with this? This is ridiculous. I understand the underlying issues but would still prefer a metric like seconds of utilization multiplied by the size of worker.
Besides this, the expected pricing doesn't talk about the expected pricing but just the pricing model. Have the feeling that this is not going to be competitive to platforms like Vast.ai
Depending on how many tokens a typical response is using, pricing will vary wildly but a rough estimate put the fast one as more expensive than chatgpt3.5 and the cheap one as way cheaper.
Quality will likely be heaps worse than chatgpt3.5, given it's llama 7b
It's 0.96$ per 100 fast chat responses
It's 0.0076$ per 100 slow chat responses
Chatgpt 3.5 with 50 tokens input, 50 tokens output will give you 0.02$ per 100 fast responses
If the llm responses are 500 tokens in and 500 tokens out then you get 0.2$ per 100 fast responses
I presume people will flock to the cheap version for when they can't afford the price and quality of chatgpt3.5.
On the other hand, if it reflects their costs, I'm very happy to have an option that is 100x cheaper, rather than a more strategic one that raises the lower price by 10x.
It is interesting to see what the future will bring when models incorporate chain of thought approaches and whether o1 will get outperformed by open source models.