> I don't think this is fair [at all], you use the types precisely to not need to be so overreliable on tests
At the extreme end of the spectrum that starts to become true. But the languages that fill that space are also unusable beyond very narrow tasks. This truth is not particularly relevant to what is seen in practice.
In the realm of languages people actually use on a normal basis, with their half-assed type systems, a few more advanced concepts sprinkled in here and there really don't do anything to reduce the need for testing as you still have to test around all the many other holes in the type system, which ends up incidentally covering those other cases as well.
In practice, the primary benefit of the type system in these real-world languages is as it relates to things like refactoring. That is incredibly powerful and not overlapped by tests. However, the returns are diminishing. As you get into increasingly advanced type concepts, there is less need/ability to refactor on those touch points.
Most seem to agree that a complete type system is way too much (especially for general purpose programming), and no type system is too little; that a half-assed type system is the right balance. However, exactly how much half-assery is the right amount of half-assery is where the debate begins. I posit that those who go in deep with thinking less half-assery is the way eventually come to appreciate more half-assery.
Hmmm, I think this is an interesting discussion. There's many sides I need to respond here, maybe I will not be able to cover everything but here I go.
See, I fundamentally disagree that those languages are "unusable beyond very narrow tasks", because I never stated that only a complete and absolutely proven type system can provide those proofs. In fact, even a relatively mid-tier (a little bit above average) type-system like C#'s can already provide enormous benefits in this regard. See, when you test for something like raw JavaScript, you end up testing things that are even about the shape of your objects, in C# you don't have to do this (because the type system dictates the shape). You also have to be very careful around possibly null objects and values, which in a language with "proper" nullable types (and support from it in the type system and static checkers) like C# can be lowered vastly (if you use the resource, naturally). C# is also a language that "brings the types into runtime" through reflection, so it will even bring you things that you don't need to test in your code (only when developing the library) like reflection for example (you will not see things that are meant to assert shapes, like 'zod' or 'pydantic' in C# or other mid-tier typed languages for example). C#'s type system also proves many things about the safety of your code, for example you basically never need to test your usage of Spans, the type system and static analysis will already rule out most problematic usages of those things. You also never need to test if your int is actually a float because some random place in your code it was set to be so (like in JS), you also never need to test against many other basic assumptions even an extremely basic type system would give you (even Go's one).
This is to say that, basically, this don't hold true for relatively simple type systems. I'm also yet to see this holding true for more advanced ones, for example: Rust is a relatively well used language for a lot of low-level projects. I never saw someone testing (well bounded safe) rust code for basic shapes of types, nor for the conclusions the type system provides when writing on it. For example, testing if the type system was really able to catch that ownership transference happening here, of it is really safe to assume that there's only one mutable reference to that object after you called that method, or if the destructor of the object is really running in the end of the scope of the function, or even if the overly complex associated type result was actually what you meant it to be (in fact, if you would ever use those complicated types, it would be precisely to have very strong compile-time guarantees that both a test would not be able to cover for -- entirely, and that you would not write unit tests specifically for in the first place). So I don't think it is true that you need a powerful type system to see the reduction in tests that you would need to write in a completely dynamically typed language, nor I think it is true when you start having really powerful type constructs, that you will come to this conclusion """start to notice that your tests end up covering all the same cases as your advanced types""". I also don't think that you need to go to the extreme of this spectrum to see those benefits, they appear gradually and increase gradually as you move towards the end (when you end up with more extremely uncommon things like dependent typing, refinement types or effect systems).
I also certainly don't agree that it does matter that "most people" think or don't think about powerful type systems and the languages using them, it matters more that the right people are using them, people that want to be benefitted from this, than the everyday masses (this is another overly complex disccussion tho).
And while I can understand the feelings you have towards the "low end of half-assery type systems", and even agree to a certain reasonable degree (naturally, with my own considerations), I don't think glorifying mediocre type systems is the way to go (like many people usually do, for some terrifying reason). It is enough to recognize that a half-assery type-system usually gets the job done and that's it, completely fine and okay, it may even be faster to write, instead of trying to justify that we should "pursue primitive type systems" because of the fact that we can do things well on them. Maybe I'm digressing to much, it's hard to respond to this comment in a satisfactory manner.
>> I don't think the "industry" is a person
> Nobody does.
Yeah, this was not a very productive point of mine, sorry.
> I fundamentally disagree that those languages are "unusable beyond very narrow tasks"
Then why do you think nobody uses them (outside of certain narrow tasks)? It is hard to deny the results.
The reality is that they are intractable. For the vast majority of programming problems, testing is good enough and far, far more practical. There is a very good reason why the languages people normally use (yes, including C# and Rust) prefer testing over types.
> See, when you test for something like raw JavaScript, you end up testing things that are even about the shape of your objects
Incidentally, but not explicitly. You also end up incidentally testing things like the shape even in languages that provide strict guarantees in the type system. That's the nature of testing.
I do agree that testing is not well understood by a lot of developers. There are for sure developers who think that explicitly testing for, say, the shape of data is a test that needs to be written. A lot of developers straight up don't know what makes for a useful test. We'd do well to help them better understand testing, but I'm not sure "don't even think about it, you've got a half-assed type system to lean on!" get us there. Quite the opposite.
> it matters more that the right people are using them
Well, they're not. And they are not going to without some fundamental breakthrough that changes the tractability of using languages with an advanced (on the full spectrum, not relative to Go) type system. The tradeoffs just aren't worth it in nearly every case. So we're stuck with half-assed type systems and relying on testing, for better or worse. Yes, that includes C# and Rust.
> I don't think glorifying mediocre type systems is the way to go (like many people usually do, for some terrifying reason).
Does it matter? Engineers don't make decisions based on some random emotional plea on HN. A keyboard cowboy might be swayed in the wrong direction by such, but then this boils down to being effectively equivalent to "If we don't talk about sex maybe teenage pregnancy will cease." Is that really the angle you want to go with?
> Then why do you think nobody uses them (outside of certain narrow tasks)? It is hard to deny the results.
> The reality is that they are intractable. For the vast majority of programming problems, testing is good enough and far, far more practical. There is a very good reason why the languages people normally use (yes, including C# and Rust) prefer testing over types.
Deny what results? Do you have some kind of formal demonstration that they are impossible to use outside of those "certain narrow tasks" (unknown)? Or do you have proof that NOBODY use them for more than those "certain narrow tasks"? Otherwise this is more "I feel like it" than something I would even need to justify deeply.
Also, with Rust this is certainly false, most people that I've saw using it (and myself) don't overly test everything in it besides more complex behavior (which types hardly can prove it is correct), but it eliminates the need for a whole suit of smaller tests that would be necessary in less powerful languages (it is literally regarded as one of the languages where "if it compiles, it works" -- or "generally works" for a reason).
> Incidentally, but not explicitly. You also end up incidentally testing things like the shape even in languages that provide strict guarantees in the type system. That's the nature of testing.
Now I want some proof to it. Make an example test that "incidentally tests things like the shape" in C#, please. I've seen a good bunch of codebases in C# and I'm pretty sure I never saw something even remotely like this.
> I do agree that testing is not well understood by a lot of developers. There are for sure developers who think that explicitly testing for, say, the shape of data is a test that needs to be written. A lot of developers straight up don't know what makes for a useful test. We'd do well to help them better understand testing, but I'm not sure "don't even think about it, you've got a half-assed type system to lean on!" get us there. Quite the opposite.
Now this point is getting lost, you changed from:
> At some point you start to notice that your tests end up covering all the same cases as your advanced types, and you begin question why you are putting in so much work repeating yourself, which ultimately sees you want to look for better.
To "I know better than a lot of developers how to test", which don't make any sense to me. You either has the same baseline testing knowledge of this "lot of developers", and hence reach similar conclusions in regards to testing (what I quoted), or you have a better understanding of it than them (and your conclusions are merely based on your own perception of testing). I don't think those points are free to take, you would need to justify this a little bit more, and I'm sure """don't even think about it, you've got a half-assed type system to lean on!""" was not the core of my point, nor a faithful representation of what I said.
> Well, they're not. And they are not going to without some fundamental breakthrough that changes the tractability of using languages with an advanced (on the full spectrum, not relative to Go) type system. The tradeoffs just aren't worth it in nearly every case. So we're stuck with half-assed type systems and relying on testing, for better or worse. Yes, that includes C# and Rust.
I would not call Rust's type system 'half-assed' tho, it is very compatible with a bunch of ML languages, it is really a very sophisticated type system with HM type inference, generic associated types, traits and more powerful things. Comparing it to C# would be unreasonable. I may also be a little bit mean to C#, it has a "mid-tier, but sufficiently good type system" for many purposes, my main problems with it are regarding to the type inference (and the lack of some basic features), but it has generics since early versions, it has interfaces, classes, subtyping, recursive type constraints, extension methods, deterministic destructors, scoped local definitions and a bunch of small useful resources. It is surely mid-tier in many aspects, but not something trivial and I don't think you can put them all in the same basket.
Either way, I also don't think that you got my point: I said it matters more that the right people are using them, precisely because those are the people that would make good use of those type systems. As a very simple example (and I consider Rust type system a very powerful one in this context) it was interesting to see that someone like Asahi Lina said the Rust language and it's features were useful for making a GPU driver, that she experienced less problems common in C (a language with a way smaller and simpler type system) and that it was having some positive effects on it. Surely, most software is not written in Rust, but the ones that are, and this is what matters, are being developed by the right people that would use them right. This is another point, as I stated earlier, but you responded to it so I'm giving a better exploration on the surroundings of what I meant here.
> Does it matter? Engineers don't make decisions based on some random emotional plea on HN. A keyboard cowboy might be swayed in the wrong direction by such, but then this boils down to being effectively equivalent to "If we don't talk about sex maybe teenage pregnancy will cease." Is that really the angle you want to go with?
It absolutely does matter. And I do believe we should talk less about mundane things and that glorifying bad ways of living can have a terrible influence in teenage brains (and even in adults in many cases), but this is also another discussion. My point is that people don't argue emotionally like they were arguing emotionally, they argue emotionally like they were right. They are (generally) not saying "well, see, I really love Go and its simplicity, so because of my personal preferences I'm saying that other languages are bad", they are saying "see, as we obviously values simplicity, and Go is simpler than that X language, Go is better than X language" (which is the shape of arguments I usually see, not ipsis literis, but in implications and the style of pointing things), and this is much more dangerous than any "purely emotional plea" (and also, most software engineers are not masters of argumentation that can dissect something like this and find all the intricate problems and possible fallacies behind it, they will believe what is most believable at the moment for them and that's generally it).
The results of software written in languages with robust type systems. Having mathematical guarantees that your program is correct is a good place to be, but climbing the mountain to get there is, well...
> Do you have some kind of formal demonstration that they are impossible to use outside of those "certain narrow tasks" (unknown)? Or do you have proof that NOBODY use them for more than those "certain narrow tasks"?
See, now you're starting to understand exactly why these languages are intractable. But perhaps we can dumb things down to my mere mortal level: Why don't you use those languages for regular programming tasks?
> To "I know better than a lot of developers how to test"
How, exactly, did you reach that conclusion? You don't have to be good at something to recognize when people are bad at something.
> but it eliminates the need for a whole suit of smaller tests that would be necessary in less powerful languages
What kind of small tests are you envisioning?
Furthermore, even if we grant that statement as being true for the sake of discussion, there is still the problem that the primary intent of tests is to offer documentation. That the documentation is self-validating is the reason you're not writing it in Microsoft Word instead, but that is really a secondary benefit.
If you defer to the type system then you're moving some, but not all, of the documentation into the type system, fragmenting the information. Is that fair to other developers? In reality, you're going to want to write the tests anyway for the sake of consistency and completeness. A programmer needs to deliver something that works, of course, but also something that does a good job of communicating to other developers what is going on. Programming is decidedly not a solo activity.
Sure, in an ideal world you could document the entire program in the type system, but the languages people normally use simply don't have what it takes to enable that. They lean on testing instead. Worse is better, I suppose.
> Make an example test that "incidentally tests things like the shape" in C#, please.
We should probably talk about my fee structure first. I don't want you coming back crying that it was too much when I send you the bill for the work performed.
That said, under my professional duty to act your interest, I expect you would far better served thinking about how you might go about avoiding testing the shape given a random useful test. You don't really need my services here.
> I would not call Rust's type system 'half-assed' tho
It doesn't even have proper support for something basic like value constraints, let alone more advanced concepts. It has more features than Go, but I fail to see how that offers transcendence beyond half-assery. I'll grant you that it is less half-assed, just as I did earlier.
> If you defer to the type system then you're moving some, but not all, of the documentation into the type system, fragmenting the information. Is that fair to other developers? In reality, you're going to want to write the tests anyway for the sake of consistency and completeness. A programmer needs to deliver something that works, of course, but also something that does a good job of communicating to other developers what is going on. Programming is decidedly not a solo activity.
As for this, as I said I really like having the type system as my documentation, and I don't know what exactly you are saying with "is that fair to other developers", this is not only fair but very useful, in fact this is WHY people really like good typed libraries in TypeScript world, they make using the thing MUCH easier and more guided, to the point that you don't even need to read the real documentation written in text that much as just exploring in the code editor. As a very good example, I literally make for my teammates many useful libraries and they love using them, the convenience they provide, and I find joy when they find "oh, that's amazing, your library can do this exact thing I was needing to do because you thought about it before", this makes their life easier, this makes my life easier, and even when they use the library a bit wrong it will still work because I made everything flow well and have easier paths for gradual usage when you need. It is not only fair, it is good if you know how to do it well, and I rarely need to write tests for my own code more than I need to write for my libraries (if they are right, generally speaking the code that uses it has way less easy to make flaws, and thus I reduce the number of tests needed in the end of the day). As if programming is or not a "solo activity", this depends on many things, I certainly have hundreds of solo projects of mine and I love working on them, and I also have my projects that are developed along with other people, and I love them as well. Programming for me is at the same time a form of expression, and art and a job at the same time.
> Sure, in an ideal world you could document the entire program in the type system, but the languages people normally use simply don't have what it takes to enable that. They lean on testing instead. Worse is better, I suppose.
As I said before, Rust does have that, and many modern languages have more and more on that. I understand that the world mostly uses less powerful languages, but this is not their fault, most languages have dozens of legacy projects behind them and it is really hard to let go (I mean, there are important programs written in COBOL nowadays, and I assume the language is even worse than your "worse is better", but people still need to use it). I'm not advocating for abandomning those projects, nor that languages with worse type systems are terrible, but that you should simply not say "it is better" just because people need to use it because of reasons, nor glorify their mediocrity. Mediocrity should be enhanced (and that's why even Go, a "simple language" is still gaining features from time to time, even it is not a crystalized stone of specific directives that will never change, and some day it will have more and more features as time moves on; all languages are slowly evolving, even COBOL itself, so if being "worse" is a goal, I think most of them are not following that goal).
> We should probably talk about my fee structure first. I don't want you coming back crying that it was too much when I send you the bill for the work performed.
> That said, under my professional duty to act your interest, I expect you would far better served thinking about how you might go about avoiding testing the shape given a random useful test. You don't really need my services here.
Oh, sorry, I thought this was a discussion where people was really trying to reach the truth, not some sort of "pay me if you want to see my point" kind of thing. I'm more of a "talk is cheap, show me the code guy", and I will certainly not pay someone to justify their own onus probandi. If this is how things will be, I think a discussion with you furthermore would be pointless.
> It doesn't even have proper support for something basic like value constraints, let alone more advanced concepts. It has more features than Go, but I fail to see how that offers transcendence beyond half-assery. I'll grant you that it is less half-assed, just as I did earlier.
And this is obviously an extreme form of exaggeration. I literally coded a basic working numeric system in rust type system with mathematical operations just for fun (and there are crates that does that), if this don't imply the language has a very powerful one I don't think anything would.
Obviously, I'm not saying that Rust has THE MOST powerful type system, I never once implied that, but it is not "half-assery" in any way, it is also many times above what Go is able to do. It's not only "more features", it is fundamentally more open to changes and advancements than it is.
> I really like having the type system as my documentation, and I don't know what exactly you are saying with "is that fair to other developers"
Because you didn't bother to read what I wrote, again, that's why. I suggested it is not fair to fragment the information. If you want to duplicate the information, go nuts. But that rounds us back to the very beginning where we opened with the topic of growing tired of repeating yourself...
> I thought this was a discussion where people was really trying to reach the truth, not some sort of "pay me if you want to see my point" kind of thing.
Yes, it is a discussion, not a make work project. If you want to deliver a point in that discussion, just do it. No need for stupid games.
> I generally don't care that much about them, because I don't find the need for them most of the time
Exactly. Same reason why nobody uses them. [I know, I know, you think this needs to be proven. But it really doesn't. That is silly.]
However, this means that you also recognize that there is a line where more typing is not worth it. So, where, exactly is that line? You say "here", but then I'll say "but no, you also need this". You'll say "that is really not that important", but I'll say "no, it is!" We could go on like that forever.
Eventually a sane person will arrive and simply say: "It depends." And maybe someday you too will understand that statement.
> And this is obviously an extreme form of exaggeration.
It is not. I will grant you that it is the lowest hanging fruit for testing, so in practice it is probably not worth the effort, but it is a great indicator of how the type system is half-assed. If it truly believed in not half-assing it, it would be there, and is in languages with robust type systems.
> For starters, things like "is the shape of this data correct"
When would a test like that ever be necessary? If the shape of your data is wrong somehow, the "documentation" tests won't be able to succeed either, so you implicitly find out that the shape is wrong anyway. There is no need to repeat yourself here. Not only is there no need for repetition, worse, tests like that usually end up making the test suite brittle and hard to manage.
> First, I don't think this is true [at all].
I gathered. This seems to be the source of contention around the testing topic when we cannot even agree what testing is. From your vantage point I can understand how you are unable to recognize the overlap. But it remains that if you write useful tests, you implicitly also end up testing what the type system covers.
The type system is still incredibly beneficial for other reasons, of course. To a point. But, again, the returns are diminishing.
> The results of software written in languages with robust type systems. Having mathematical guarantees that your program is correct is a good place to be, but climbing the mountain to get there is, well...
It is harder, obviously, but it still tends to be a matter of understanding most of the time. But I can understand what you mean now, you are not saying it is "impossible" to do so, but that it is very hard to do so, harder than using testing (even with a lower guarantee level). If this is the case I can buy it partially, but then your point would not be as strong, I mean, we need to make a language with a good type system that can prove things reasonably well and be not that hard to use. This is more of a call to action than an impossibility question.
> See, now you're starting to understand exactly why these languages are intractable. But perhaps we can dumb things down to my mere mortal level: Why don't you use those languages for regular programming tasks?
Following my previous response: I generally don't care that much about them, because I don't find the need for them most of the time (this is part of why I said the mostly the right people matter, instead of everyone), but also because I find many of them designed in a way that is not ergonomic enough for me. This is more a design problem (that most languages have) than something else. But also, I generally use for my everyday preferred programming tasks languages that have at least powerful enough type systems, like Rust or Swift, F#, C# (because it is on the higher level of mid-tier), or even Kotlin (that I like very much, it don't has everything I like but it is closer to Swift and has a better compiler tooling) than I use languages that have not that good type-systems and resources (like Go or C), in this sense I pretty much live to my own standards, I just write tests for the things that matter and I use the type systems of those languages to prove things about my code. It works very well, and I very rarely experience problems with this, it is really satisfying for me, but I get not everyone likes this way of coding.
> How, exactly, did you reach that conclusion? You don't have to be good at something to recognize when people are bad at something.
Wdym? You don't need to be good at something, but you surely need to [understand] [better] than most people to realize those same people are worse in [understanding] and [applicability] than you.
Even knowing that you don't know something is a sign that you have a better understanding of that thing than others, but you cannot overlook to other people and say they are dumb if you consider yourself to be at the same level than them, this would be irrational to believe (because you have virtually the same knowledge and limitations).
So, if you say "most developers don't understand testing" I must get from your words, that you at least know what is a [better understanding of testing], or that you are at least more [aware] than other people about the limitations of their own testing (which implies priviledged knowledge).
But if it is the case, then an affirmation like "We'd do well to help them better understand testing" would be just pure insanity. If you believe "we" can help people understand better something, you must understand this [better] than them, there is no teacher that, ceteris paribus, know less than his students in the specific matter that he's teaching and that the student has flaws on it.
> What kind of small tests are you envisioning?
For starters, things like "is the shape of this data correct" (in the base level of any strongly typed language), and things like "was this object unitialized in the end of the scope as intended" (on the destructors feature), and things like "did I forgot to call some method, make some state change or do something else" (on the typestate concept). And even things like "is this function being possibly misused" (with type-system guarantees about mutability, nullability, aliasing and owning references, you can remove a whole class of specific tests like "is this argument invalid", "is this object being mutated outside of this function and thus being possibly in an invalid state at some point in this function", "can I be sure I can optimize this code by doing in place mutations without breaking other parts of the whole software that would be depending on it", etc). Obviously this is kind of abstract, but this is because testing is usually a pursuit of turning those generally abstract concepts onto something practical like "when the user is loging in, is the returned object in a consistent state, are the services, managers and encryption tools being properly used" or "is the customer in a valid known state at this point in the code that is significantly more complex? Can I be sure that it is not null and thus I don't need to test against it in some point?" etc.
> Furthermore, even if we grant that statement as being true for the sake of discussion, there is still the problem that the primary intent of tests is to offer documentation. That the documentation is self-validating is the reason you're not writing it in Microsoft Word instead, but that is really a secondary benefit.
First, I don't think this is true [at all]. There are many kinds of tests, and the "intent" behind them is different. For example, I can say that the primary intent of tests is to ensure the given problem and the expressed code are aligned, to check if what I did is really doing what I intended and that I did not commit any logical errors when expressing the thing, or that I was unable to express correctly what I intended to express (like if, even if everything was correct from a purely computational standpoint, I was able to effectively reach the state I was trying to consciously reach). I think a [side effect] of testing is that it turns out to be pretty good documentation for many problems, but not that it is the [intended] goal of it. Maybe if you are developing it with this goal in mind, but not as an objective unique truth.
And also, even if I accept this: type systems are extremely good ways of documenting the things you are doing. I saw many times haskell programmers say the use the types of the functions they want to call, or the things they want to make, as the way to find appropriate usages of that thing (i.e. if they need to convert a string to an int, they can search in their editor for [String -> Maybe Int] and they will find many useful functions, and probably the one they want, and everything would be very clear for them in this sense). Good types lead the programmers using them to correct code, because they make it very hard (or, sometimes, even impossible) to express incorrect programs using those types. Part of the reason I really like good type systems is that I am a very forgetful person, if I write something the chances of me forgetting about it later down the line are very high, so I really like the sensation of coming back to a codebase and finding all the clues I left for myself (i.e. the types) and discovering that for use that I need that other thing (or else it won't compile), and that function I'm calling can error in some specific signalized ways in the types (and now I remember, I need to do this and this) and how everything fits well like a good and very comfortable puzzle. This is my ideal documentation, and tests are also important for me to remember more forms of how I used this code in practicality sometimes, but many times the types are really everything I need.
This, obviously, is more anedoctal, this is how I view things, but I think many people would agree with me on this, and that this is not absurd at all as a conclusion.
--------------
I was bitten by the char limits here, so I'll put in parts
(part 1)
At the extreme end of the spectrum that starts to become true. But the languages that fill that space are also unusable beyond very narrow tasks. This truth is not particularly relevant to what is seen in practice.
In the realm of languages people actually use on a normal basis, with their half-assed type systems, a few more advanced concepts sprinkled in here and there really don't do anything to reduce the need for testing as you still have to test around all the many other holes in the type system, which ends up incidentally covering those other cases as well.
In practice, the primary benefit of the type system in these real-world languages is as it relates to things like refactoring. That is incredibly powerful and not overlapped by tests. However, the returns are diminishing. As you get into increasingly advanced type concepts, there is less need/ability to refactor on those touch points.
Most seem to agree that a complete type system is way too much (especially for general purpose programming), and no type system is too little; that a half-assed type system is the right balance. However, exactly how much half-assery is the right amount of half-assery is where the debate begins. I posit that those who go in deep with thinking less half-assery is the way eventually come to appreciate more half-assery.
> I don't think the "industry" is a person
Nobody does.