This is just science fiction. To mention "recent developments" in the introduction is somewhat misleading considering how far the current state of technology is from their hypothetical superintelligence.
We don't have superintelligence, we don't have the remote idea of how to get started on creating it, in all likelihood we don't even have the correct hardware for it or any idea what the correct hardware would look like. We also don't know whether it's achievable at all.
That's the mainstream opinion on every. single. revolutionary advance. That you and everyone else believes it's not going to happen ever has almost no predictive power as to whether it actually will.
It's not so much "opinion on a revolutionary advance". When it comes to AGI-related stuff, we are quite literally like contemporaries of Leonardo da Vinci, who have seen his plans for the helicopter and are postulating that helicopters will cause big problems if they fly too high and crash into the mechanism that is holding up the sky above us.
Also, this is not the mainstream opinion on e.g. fusion, or electric cars and smartphones (20 years ago), or a computer in every home (50 years ago). Those have been arguments about money and practicality, not about "we don't even know how such technology would look or what it would be based on".
I think we fear ourselves (rightly) and super-fear a super-self (rightly). If this theoretical thing has the capacity at all to interact the way humans do then it will probably be brutal to at least someone and possibly most or everyone.
Difference is, people were able to propose detailed mechanisms for realistic flying machines long before we actually achieved powered flight - that was mainly a matter of increasing the power to weight ratio of the propulsion system. For AGI, do you really think there are detailed proposals out there today that can achieve AGI but are only missing the computational power?
Actually the existence of human brains with their (comparatively) extremely low power consumption, indicates that we need something radically different from current silicon-based processors to achieve AGI.
> For AGI, do you really think there are detailed proposals out there today that can achieve AGI but are only missing the computational power?
Yes, I really do. It's neural networks, nothing more. All that is required is more power. Despite its lower power the brain is much more computationally powerful than even the largest supercomputers.
Although it is not necessary, imo, to have a much more efficient computing substrate to achieve GAI, there is work in this direction. Look into optical reservoir computing. Or more generally thermodynamic computing. https://arxiv.org/abs/1911.01968
Machine learning is progressing rapidly mostly because the computational power is increasing rapidly. Read this for an understanding of how important computational power is to progress in machine learning: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Time and time again computational power trumps complicated technique. It is fairly obvious to many now, after the continued scaling of the GPT-x series models, that genuine intelligence is an emergent property of the kind of systems we are building. It is an emergent property of systems that are driven to predict their environment - no more secret sauce to discover.
I think a major objection people have to the idea that machines are already a little intelligent is that they cannot understand how intelligence can emerge from pure computation. They imagine that there must be some magic between the operation of neurons in their mind and their experience of the world. Qualia where does it come from, how can it be just neurons, just inanimate physical matter? Combine this philosophical objection with the emotional response to the threatening nature of AGI and you have a milieu that refuses to see what is directly in front of its nose.
There are plenty of philosophical positions that allow you to understand how consciousness might emerge naturally from computation. Consciousness is emergent... Panspermia, consciousness is found in everything. To imagine consciousness in the movement of 1s and 0s within a machine is not too hard, to see the consciousness inside the "chinese room" is not really so difficult.
But none of this is relevant to the original argument I was making which is: trying to tell the future is very hard. Because it is simply a fact that before most great breakthroughs there is a common opinion that such a thing is, if not impossible, then in some distant future.
I think there are very very good reasons to think that a general intelligence of genuine utility is not more than 10 years away. I think people will look back at GPT-3 as the first proto form of the general intelligences that will exist in the future.
> It is fairly obvious to many now, after the continued scaling of the GPT-x series models, that genuine intelligence is an emergent property of the kind of systems we are building.
I respectfully disagree. GPT-x series models are performing interpolation on an unfathomably massive corpus. It is not hard to find cases where it directly reproduces entire paragraphs from existing text. When given a prompt on a topic for which it finds multiple existing texts with similar degree of matching, such as different articles reporting on the same topic, it is able to blend the content of those articles smoothly.
I mean, GPT-3 is around 6 trillion bits of compressed data. The entire human brain has 0.1 trillion neurons, and it obviously has a capacity far beyond GPT-3 - even in the extreme case if we assume all the neurons in the human brain are used for generating English written text.
In my view GPT-x is very, very far from any kind of general intelligence.
You want to be thinking about synapses. There's about 7000 synapses per neuron, so that's 7000 * 0.1 = 700 Trillion synapses. So thats *100 times larger than GPT-3. Also consider that a neuron does a fair amount of processing within the neuron, there is some very recent research on this, each neuron is a akin to a mini neural network. So I would not be surprised if the human brain is 10,000 times more powerful than GPT-3.
> It is not hard to find cases where it directly reproduces entire paragraphs from existing text. When given a prompt on a topic for which it finds multiple existing texts with similar degree of matching, such as different articles reporting on the same topic, it is able to blend the content of those articles smoothly.
This may be true, but it does not prove your hypothesis that all GPT-x models are simply "performing interpolation". Also the ability to perform recall better than a human may be to do with the way that we perform global optimisation over the network, rather than the local decentralised way that the brain presumably works. Point is accurate memorisation does not preclude general intelligence. Spend some time with the models, sit down for a few hours and investigate what they know and do not know, really look, see beyond what you expect to see. You may be surprised.
I'm sure there's a fallacy in the following, but here goes:, Who could have predicted the improvements in computation in the last century? Would someone a century have extrapolated sun-sized machines need to compute a nations taxes based on current SOA? We don't have it and then all of the sudden we will. Its worth recognizing the potential harnesses before the beast is born.
Regardless of the source I think the main point folks are missing is that a lot of DE jobs will require you to know Scala so it's a good tool to have if you want to be a DE somewhere.
SQL is also an amazing tool and you should definitely learn if but there are a lot of DE jobs out there with Scala in the "Requirements" section of the job listing. Parts of the industry might be moving away from it, but if you're looking to make a jump into DE I think you're hamstringing yourself by avoiding Scala.
I don't think that it's wise to sabotage your own future and productivity as a company just so you can pave the way for some language to become more popular.
I don't think that it's wise to try to optimize for some kind of speculative long-term success at the expense of higher early-stage costs that reduce your odds of getting there. This is similar to companies that choose their initial technology with scalability in mind before they're even remotely close to needing to scale. I've actually done this only to discover that scalability has a very definite cost and when you're small it has an outsized impact on your burn rate. If you have success, you're going to figure out a way to make the changes you need. Case in point: Facebook. They successfully grew a PHP codebase into one of the most popular apps in the world. It definitely cost a lot more money for them to make PHP work, but when they got to that point they were a lot less cost-sensitive.
Planning for that far down the road is the least of your worries. And any plans you make along those lines are not likely to be very accurate anyway. You're much better off optimizing for the near to mid term. Based on the hosting costs described by the OP they are already reaping a tangible value here.
I think at worst, Haskell is a a minor productivity cost for a company vs a mainstream language, and if it is, it's hard to pin it on the language.
So given the upside to paying people to using Haskell (they get to learn it for life, many join + grow the community, they enjoy working for your company more), I think it's worth that kind of harm to a corporation.
I'll keep trying to sap corporate resources into Haskell I take with me for life at least :)
I feel like this isn't discussed enough. I can't comment on the technical merits of Haskell but growing an organization and replacing engineers is so much more difficult when you're using tools that aren't mainstream.
The market works somewhat differently for small companies there. Yes, there are fewer people with relatively niche skills, on the other hand you have an easier time to attract the few you need. Not every company wants to become large.
I've never hired a Haskell developer, but anecdotally from my friends and associates who have, if you put out an advertisement for a Java/Python/PHP developer you get 500 applications from average candidates. If you put out an advertisement for a Haskell developer you get 5 applications from good candidates.
You probably got 500 applications for Java/Python/PHP, of those 450 average and let's say 50 good.
With Haskell you just get 5 good ones. You probably don't start with Haskell as your first language but rather move into it after you are a senior in another language. If you are lousy in Java, you probably won't go and learn Haskell or some other niche language.
For reference, that got called the "Python paradox" back then when Google was exploiting it. Of course, Python is now mainstream, so it doesn't have this effect anymore.
I hear far more complaints about how difficult it is to find good people from companies hiring for mainstream languages than from those using more niche stuff, I would assume mostly due to larger competition among employers for the former (and in parts better community access for small shops in niche languages and self-selection of who learns the niche languages)
> I hear far more complaints about how difficult it is to find good people from companies hiring for mainstream languages than from those using more niche stuff
I would assume that:
(1) Those using niche stuff are less likely to be hiring under the impression that the main measure of skill is years of experience with a language, and
(2) those using niche stuff are, on average, doing more interesting work that attracts more intellectually curious candidates.
As a result, the mainstream firms get worse candidates, and try to compensate by asking for even more years of experience, and asking for years of experience not just with language but specific libraries and other tools, hoping that will get them more skilled candidates, at least for their specific toolchains. But doubling down on that just gets them candidates that are less capable (because even to the extent years of experience are useful, there are diminishing returns, and people who have spent a huge amount of time with the same stack also are likely to be in the “1 year of experience, repeated N times” category, rather than N years of learning and compounding knowledge. (Also, because at a certain point you start making impossible demands, increasing the degree to which the hiring process filters for dishonesty.)
> I hear far more complaints about how difficult it is to find good people from companies hiring for mainstream languages than from those using more niche stuff
Of course, there is just more of them in the first place. The other effects that you describe might also be true but keep in mind what the base rates are.
Optimizing for worker fungibility, in a vacuum, seems like a -EV "playing not to lose" strategy. It's understandable that this line of thinking is common though.
I also believe that to be the case as an employee. Ie, being a generalist is probably -EV for your career but it feels safer so it's kind of a contrarian position to say "be a specialist".
> Optimizing for worker fungibility, in a vacuum, seems like a -EV "playing not to lose" strategy.
That's why you don't optimise for it in a vacuum. You weigh the potential benefits of switching to Haskell versus the additional cost of maintaining/growing a Haskell team.
It's possible depending on how much inconvenience you can accept in your life but that's also kind of irrelevant. If you don't like something online, don't take part in it. You don't have an obligation to consume and do everything online, just pick out the parts of it that work for you.
It's really a strength, not a quirk. Negative indexing and array slicing in general are great in Python. Really easy to pick up and way more convenient than any other language that I've come across.
Negative indexing in Python is a dangerous design that hides bugs and causes incorrect programs to return garbage instead of erroring. If you have an incorrect index computation, instead of getting a bounds error, you get different indexing behavior. This makes it dangerously easy to write code that appears to work but computes nonsense.
Without checking, I'm not sure whether a[-1:0:-1] reverses a list. I'm not sure if it includes the first element of the list or not [edit: It doesn't]. I'm not sure why in contrast a[::-1] does reverse a list. I found array slicing (and ranges) to be a source of confusion when programming the aforementioned linear algebra algorithms.
IMO, the Julia approach is better: 3:-1:1 produces [3,2,1]. Both the starting and ending points of the range are included.
The example a[-1:0:-1] should be pretty understandable once you've spent enough time in the language -- you're inclusive at -1 and exclusive at 0, so the list is reversed and is missing its formerly first element (should one exist).
That logic is pretty consistent. The start is always inclusive, so it needs to be `len(a)-`, `-`, or `None`. The end is always exclusive, so you need to choose `None`. As a result, counting all the syntactic sugar available to you, you have 8 slices that can reverse a list. To name a few you have a[::-1], a[None:None:-1], and a[-1::-1].
IMO a much more interesting example for the strengths of inclusive indexes is a[len(a)-1:-1:-1]. The result is always empty, but it wouldn't be too much of a stretch to think you included len(a)-1, you decremented to -1 exclusively (thus including 0), and hence reversed the entire list. The problem is that -1 is a valid index, and unlike the a[0:len(a)] case you don't have any values "before" the beginning of the list to include 0 in an _exclusive_ expression.
It's all a bit of a moot point though. I know Python especially chose its [inclusive,exclusive) convention largely because it wanted expressions like range(len(a)) to not require additional arithmetic for common use cases given that it had zero-based indexing. Julia has one-based indexing, so for common use cases an [inclusive,inclusive] pattern falls out as the most natural choice. I have no idea if Julia actually cared about that sort of thing or if such a convention came about by accident, but it seems like a clean choice for a one-based language.
Ohm, a lot of thought has gone into indexing in Julia. Julia allows indexing by position `A[1,1]`, slicing `A[1:5, :]`, linear indexing `A[1]`, and logical indexing `A[a.==1]`, relative indexing `A[end-1]` and cartesian indexing `A[CartesianIndex(1,1)]`.
The way Julia combines these in a mostly non-conflicting and non-confusing manner is a major engineering feat.
For example the following rule was found to be the just the right balance between permissive and strict behaviour:
* You are permitted to index into arrays with more indices than dimensions, but all trailing indices must be 1.
* You are permitted to index into arrays with fewer indices than dimensions, but the length of all the omitted dimensions must be 1.
I don't know; the grammar is already pretty complicated without introducing additional indexing schemes. Now that you mention it though, I don't know that I've ever written an algorithm mixing and matching negative/positive indices in a way that couldn't be trivially re-written with that kind of syntax. I'm sure such cases exist for somebody...
Looking for solutions, if you're stuck with Python and hate that behavior:
- If you don't need errors raised then a[~i] is already equal to a[-i-1].
- In your own code (or at its boundaries when wrapping external lists) it's nearly free and not much code to subclass list, override __getitem__, and raise errors for negative indices while responding to some syntax like a[0,] or a.R[0].
Bidding on your name might be an even bigger waste of money than bidding on a competitor's name, especially for established brands. eBay's experience with this came up on a recent episode of Freakonomics - https://freakonomics.com/podcast/advertising-part-2/
Whenever I search for a brand by name and an ad for them pops up (in the rare case that I’m browsing without an ad blocker), I try to click on the non ad version right below it just to save them a bit of money.
This is kind of embarassing. Do you guys think that the people at Facebook who make these decisions realise how pathetic it looks from the outside? Or are they somehow justifying it in their own heads?
They don’t “realize it’s pathetic” because it’s SOP for companies buying ads to bid on their competitor’s name. You can argue that that is pathetic, but this one instance isn’t special.
