Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: A SQL database implemented purely in TypeScript type annotations (github.com/codemix)
426 points by phpnode 11 months ago | hide | past | favorite | 191 comments



I’m impressed/fascinated/horrified by the way it parses a string into a type with a variant of template literal syntax—though it looks like the parsing is probably very fragile. I’m curious why that functionality was added to the language.

The end result is actually pretty approachable.


> I’m curious why that functionality was added to the language.

I think it's because there are a whole bunch of APIs in javascript where there is a function like `.get("foo")` that internally calls a function called `getFoo` or similar. And this functionality allows such functions to be accurately typed. It basically allows strong typing for stringly typed functions (which are pretty common in dynamic languages)


Stringly typed functions frustrate me to no end. Tooling for programming gets better every year, yet stringly typed code is so resistant to any of those improvements. Pycharm at some point had to add SQL tools in python strings for gods sake. That kind of weird deep special casing is everywhere with strings drive behavior. Why use a class (inspectable) when a dictionary, a function, and an if statement (opaque) will do?


I think they implemented a generic nested language framework (whose real name eludes me).


The rationale for template literal types is discussed in the recent blog post about the TS 4.1 Beta.

https://devblogs.microsoft.com/typescript/announcing-typescr...


it's a bit fragile only due to my laziness, it could be done properly if this were a serious project.


How would it be done if done properly?


same mechanism, just deal with more edge cases. e.g. whitespace isn't really trimmed correctly at the moment, it doesn't handle escaping quotes in strings and so on. I did think about trying to match postgres's grammar but it's a lot of work and probably implementing a PEG first would make it easier.


The implementation isn't even that horrible. Ignoring the integer list. That workaround is still hillarious to me.

Anyway, it seems both impressive that someone did this and that it actually works. Weird how typescript has the nicest typelevel programming I know, even nicer than some dependent ones. It probably is the only language one where autocompletion properly works with computed types?


> The implementation isn't even that horrible

You're making me blush!

The list of integers could be replaced by using tuples and reading their lengths to do basic arithmetic. Might run into recursion limits though.


> The list of integers could be replaced by using tuples and reading their lengths to do basic arithmetic.

Is that anything like church encoding?

https://www.wikiwand.com/en/Church_encoding#/Church_numerals


Not really. In the Typescript type system tuple types are array types where the length is encoded in the type system. Tuple types can be “added” by concatenation to produce a new tuple type:

    type two = [0, 0]
    type three = [0, 0, 0]
    type five = [...two, ...three]
    type TupleToNumber<T extends any[]> = T[“length”]
    const num: TupleToNumber<five> = 5


Am I understanding this correctly that typescript provides a type-level property on tuples called `"length"`?


To clarify, the "tuple" is just an array with fixed length. So ts isn't "providing" the length property, it's simply surfacing the underlying Array#length in JS, which it knows to be a constant.


yep!


Other uses for the template literal types:

JSON parser https://twitter.com/buildsghost/status/1301976526603206657

This one is more realistic: Typed route params https://twitter.com/danvdk/status/1301707026507198464


Great project, really expands what feels possible with the new improvements in TS 4.1.

I wonder if this strategy could also be used to have strongly-typed GraphQL queries without the need to have a type generation step that you run separately. Currently when using GraphQL with Apollo, it's great that you get types corresponding to all queries, but having to run the type generator and maintain types in __generated folders is definitely an extra hassle.


I had no idea you could compose string literal types AND infer variables using interpolation syntax in the context of conditional types.

I am salivating at the potential use cases for this...


It's a new feature from TypeScript 4.1 (this version is yet in beta)

https://devblogs.microsoft.com/typescript/announcing-typescr...


Lots of new toys to play with in this release! They've also explained each new feature really well...


Right! That is insane - I had no idea it was possible to infer part of a string literal.

My mind is exploding at the possibilities.


Typescript was created to bring static types in dynamic language, like JS. Now they made the static types more dynamic. What's next?)


JavaScript is such a cool language, and I really love it, especially the FP subcommunity, but the AI community for Python is way bigger, I wish there were more options for neural nets, differentiable programming, and data frame type things in JS. The interoperability with WASM is hugely valuable, and so is the ability to easily publish results on the web. It feels like it’s way easier to make an AI system in python, but then you have to figure out how to monetize it, while if you use JS, you can build a web platform or service and monetize much easier. Thoughts?


I think you're talking about two different systems and two different problems. First, there's the core service code, the AI system in Python you're talking about. Then, there's the web platform or service that you monetize. But there's no reason why you wouldn't just use both.

Write the core AI code in Python because that's what's practical to do. Write the app code in JS because that's what's practical to do. Sell the product to customers that want to use it and probably don't care what's underneath.


In my opinion it would be great to have a single language for developer productivity. Backend and frontend can be already written fully in TypeScript and you can share interfaces between frontend and backend so everything is connected and business logic is not repeated. Same thing could be done for machine learning.

If backend changes frontend has to change as well or compiler will yell. Same for machine learning input and output.


I think these are two different languages for two different jobs.

I can see javascript slowly expanding from the front-end into the back-end, mostly thanks to projects like TypeScript that appeal to the back-end crowds, but also because of the whole serverless hype which IMO is justified.

Now there still is a strong "cultural divide" between the software and the data-science communities. And when you see how strong the innertia for python's projects is.. Well I guess we won't see this changing anytime soon.


This shows how complex TypeSript has become. It may be fun to write complicated Type annotations such as this, but imagine having to read and maintain the code!


From years of real-world experience, it's rare to run into types that are too complex to easily understand. The syntax is pretty easy to read, and the compiler and language service make it easy to navigate types, even if you haven't looked at the source for them yet.


FWIW I find that TypeScript types can sometimes be overwhelming or hard to decipher, especially some third-party library type definitions. The connect function from react-redux is one example:

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/d07d...

Since types can get really complex, I feel a need to be conscious of avoiding complexity when writing types myself, unless the value is enough to justify a hard-to-read type.


This is the complexity introduced when applying static types to a vanilla JS codebase, imo. It's easy to get crazy with types in vanilla JS. Adding Typescript forces you to resolve the different types explicitly and it can get hairy.

What this really demonstrates to me is the difference between starting with TS vs migrating to it. Code that has to think about the types the first time it is being built is going to be better structured, on average, than purely dynamic code.


Many complex/difficult to understand TS types exist precisely because the type system was too limited to express certain real world use cases without workarounds. As the type system takes on that complexity, and removes more limitations, it actually makes writing simpler types for those cases much easier.


I find these types still exist in JS it's just impossible to decipher what you're writing and supposed to be doing without a solid TypeSystem. The types might look complicated but really this is much more simplified that getting runtime errors.


From ts conf talks, only a minority of people actually write most of the complex types. Those are mainly library authors and such, I presume.

https://youtu.be/jmPZztKIFf4 at 49:50.


> a database implemented purely IN TS

Ok, we might have went a bit too far


Nice! I built something similar - A set of scripts that generate SQL statements from typescript types to wrap a PostgresSql db.

It’s more a hybrid data structure where only relationships and certain fields become columns (which can be indexed and searched) and most of the other data becomes a Json column.

I wasn’t completely satisfied with the final results, but it does at least allow me to add additional data fields without having to migrate the db structure and to use Postgres sql to filter and page data.

I think the goal of being able to define a true database in typescript is great and I look forward to trying your code.

It’s also pretty cool if you are able to extract types from a string using the type system - that’s next level typescript wizardry.


Thinking about this more. I think this project has some promising practical uses when combined with code generation from the typescript ast (which is what my project does).

Since the type system can parse the SQL strings, then it becomes possible to include those strings as a source for the ast. Then the code generator can use the typescript parser to parse all the types and the sql. From that it can generate all the required code to create the runtime queries.

I don’t like ORMs because they make it very difficult to write complex queries (and my own attempt did not solve that very well - I still had to manually type any complex queries). However, if this could essentially allow writing a complex query in raw sql and have it type-checked at code-time, it becomes easier to use. Then the code generator would convert that into a type-safe runtime library which solves 2 problems: 1. The poor editor performance of the type system inferring so many nested types. 2. Actually having a runtime that executes the queries.

The outstanding problem would be migrations...

Anyway, this is certainly a thought provoking project.


I find the ORM in SQLAlchemy pretty cool — I’ve really written complex queries as if I had witten it by hand but then finally mapped them into the ORM at the end with like a line of code.

However it was a learning curve to figure out how to do it at first, but I didn’t have to drop down to raw SQL


my thoughts as well. compile-time checked arbitrary sql would defeat the purpose of so many abominable ORMs. make it work with DDL instead of sample data and you've really got something nice!


This project does exactly that (TypeScript/PostgreSQL):

https://github.com/MedFlyt/mfsqlchecker


That looks pretty nice. I’ll have to check that out.


> I built something similar - A set of scripts that generate SQL statements from typescript types to wrap a PostgresSql db.

If you think these are similar I don't think you understand what this is / what it's actually doing.


Yes, I only meant similar in the sense that it attempts to use typescript types as the primary source for the definition of the database schema.

I agree my wording could have been better.


Could someone ELI5?

Actually, I wrote a lot of code in TypeScript and I know my SQL but I have absolutely no idea what is going on here.


Since you already know TypeScript, I think it is important to note that this project uses a new TypeScript 4.1 feature called Template Literal Types to do pattern matching in order to parse the SQL string.

https://devblogs.microsoft.com/typescript/announcing-typescr...


This is fascinating! On one hand, as an old guard JS developer that upscaled to TS, I often find new TS syntax incomprehensible. On the other hand I am still slowly learning new concepts. Am I correct to assume these new concepts are imported from more advanced languages? Or are these invented in TS for TS?


This particular feature is invented in ts, other instances of ts inventing a completely new thing are conditional types and mapped types. F# has type provider which is similar in some of its usages but different enough i dont think they're related.


I think this is an exploration of the somewhat surprising fact that type systems can be, and often are, Turing Complete[1]. A very naive view of type systems is that they're merely the next level up from code comments, except with the annoying burden of having the power to prevent compilation.

As this project demonstrates type systems are far more than just glorified comments. Indeed, they are in fact a rigorous scaffolding that seemingly magically "intuit" the intention of code without having to run it. This project playfully explores the extent of Typescript's particular powers.

1. https://en.wikipedia.org/wiki/Turing_completeness


I'm not entirely sure, but I suspect it's one of these programming "joke" where you take a concept and bring it far enough that it becomes ridiculous.

So I'm guessing there's no real practical use case for this, and is a nice flex from the author.

It would be nice if that were made dead clear in the Readme, something like "Why? Because it's fun"


strong typing for arbitrary sql queries is actually really nice and this is much better than stuff like sequelize or typeorm which seem sadistic in comparison


generating types from a swagger spec? Graphql? Or any kind of schema? Look up fsharp type providers.


As far as I understand (and I might be wrong so please correct me), this can't be used to generate anything. Of course all the technologies you've listed are useful and brilliant but this isn't what I'm seeing.

It's more of an experiment of thoughts to generate types that are defined by the result of the SQL query, bending a compiler to a new strange behaviour that happens to be SQL like.


It definitely can parse anything, the code that is required to make such a parser might be a bit unreadable but look up somewhere in this thread there is a twitter link to a github project of a generic json parser. Typescript 4.1 isn't event released yet, once it is there will be a dozen of projects like this.

If you think this is useless because it doesn't generate any code then I'm assuming you're not familiar with typescript because type level libraries isn't supposed to do do that anyway. There are already libraries that generate open api client and graphql client but they generate type definition via an extra build step which libraries like this can eliminate.

And this typescript feature was made to parse strings so no this is not "bending the compiler"


Wow, I hope the typescript compiler is ready for this kind of dynamic complexity...

    type Protocol = 'http' | 'https';
    type DomainExtension = 'com' | 'org' | 'net';
    type DomainName = 'example' | 'google';
    type Domain = `${DomainName}.${DomainExtension}`;
    type Path = string;
    type ApiEndpoint = `${Protocol}://${Domain}/${Path}`;
This would seemingly get out of hand very quickly...


I work on a codebase with a lot of this type of code. So far the compiler is up for it. The only time it gets slow is using the Zod language for runtime validation.


Related, C++ template metaprogramming. Very useful, quite complicated to wrap your head around initially (at least it was for me), and a somewhat accidental language feature: https://softwareengineering.stackexchange.com/questions/1253...


This is terrible, I love it


TypeScript is becoming such a compelling language due to its insanely advanced type system (that allows for projects like this) that I now want to use it everywhere. I want it to become the next Python.

I know Deno is supposed to be first class TypeScript but under the hood it's still a JavaScript runtime with all the baggage that comes with that.

AssemblyScript is extremely interesting but last time I played with it I concluded it wasn't yet ready for anything serious, or is that no longer the case?

A first class TypeScript runtime with no JS overhead would be a dream come true. I just hope it happens someday, or that other type systems get these amazing features.


I’ve only been writing TypeScript for about a year, so I might be getting something wrong, but here’s my understanding...

I’m not sure your request fully makes sense. TypeScript is extremely expressive. You can specify basically any type you what... stuff like “The Number 4 Or A Function That Returns The Number 4 Or An Object With Any Attribute Which Is Equal To 4” and TS will just work with that.

This SQL compiler is a perfect example of how far that can be taken. But I think it’s important to realize how this differs from a strongly typed, compiled language.

A language like Go doesn’t just need to know you’re passing an “array of structs with attributes x, y, and z” for fun. It actually uses that information to generate a binary. The types are not just a check that happens before the real compiling begins, they are how the compiler thinks about structuring the executable.

I would love if a compiler expert could chime in here, but my understanding is you could never write a Typescript compiler that was “strongly” typed because there is no data structure that can efficiently represent arbitrary types like “4 Or An Object With An Attribute Set To 4” in any meaningful way.

These kinds of “wacky types” are things you can statically analyze but not really “compile” per se. That’s why it’s a good fit for being transpiled down to a language with no types at all.

TL;DR, TypeScript is not really a “typed programming language”, it’s a static analysis tool. It’s not really designed to “run”.


Types do not need to have a one-to-one correspondence with runtime data structures and indeed in many type systems often do not. In the limit, a type does not need to have any runtime representation at all (this is more than just generic erasure or phantom type parameters, you can have an entire type signature that has absolutely no runtime representation of any of its parts).

There is not really a fundamental difference between types and static analysis.


A predicate is a computer program that accepts an input and returns "yes/no." A type system could be an arbitrary predicate which returns "well typed/not well typed" when given a term. But obviously there must be some difference between types and predicates, such as extensionality -- the type of an expression can be decided by examining the code alone.


Are you responding to my assertion that there's no fundamental difference between type systems and static analysis? If so, if we just change "well typed/not well typed" to "acceptable/not acceptable" I think you've pretty much just described static analysis.


> You can specify basically any type you what... stuff like “The Number 4 Or A Function That Returns The Number 4 Or An Object With Any Attribute Which Is Equal To 4” and TS will just work with that.

Hi, an amateur here. When I started to learn JS and later TS I always disliked this expressiveness. Like when the first parameter you pass to JQuery can be anything in the world: a DOM element, or collection of elements, or an object, or a function - and only then you declare what you want to do with it. Yes, the syntax is short. And you pay for it with a lousy code readability and steep learning curve.

My question is, is there a name for this "expressiveness"? Is there a name for the opposite? I am looking at the definition of AssemblyScript: "...compiles a strict variant of TypeScript (basically JavaScript with types)". But the word "type" is again used recursively here. Is there a name for this "strict type"?


> Like when the first parameter you pass to JQuery can be anything in the world: a DOM element, or collection of elements, or an object, or a function

That specific part is called overloading in OOP (java). Don't know about the rest.


> I would love if a compiler expert could chime in here, but my understanding is you could never write a Typescript compiler that was “strongly” typed because there is no data structure that can efficiently represent arbitrary types like “4 Or An Object With An Attribute Set To 4” in any meaningful way.

(I don't consider myself a compiler expert by any means, but I've been doing it for nearly 20 years, so I feel relatively safe weighing in here.)

In short, if you were compiling a function like that for bare metal (or anything relatively like it), there would be no one function that takes in all of those types and acts on them. Instead, you would compile that function once for each combination of incoming types. The version that only takes the number 4 would not have any inputs at all and may end up entirely constant (depending on what it does, of course). It's then up to the caller to make sure the right variant is called. If you have strong types up the chain, this is easy and completely overhead-free -- they'll be compiled in their different versions, too. If you have a weaker type constraint (e.g. just any Number, rather than THE Number 4), then you'd do conditional dispatching at the call site.

For what it's worth, this is essentially how templates work in C++. It's also why compilation can take so damn long and can produce gigantic binaries, because when you have a function with 2 inputs of 4 different potential types each, you're up to 16 variants. Change that to 5 inputs of 4 different types and you're at 625 variants.

All that said, static compilation of TypeScript in its current form would be very difficult to make efficient, simply because JavaScript types in the abstract don't map nicely to hardware; Arrays can be extremely simple and linear or ungodly complex with holes and property overrides, and the 'right' thing to do with a Number is often different from the fast thing to do. That's why JITs are so valuable; they allow you to get around the ambiguity.

Edit to add: I should mention, it's possible you won't actually generate all combinations. You might know that certain combinations are impossible to hit due to other type constraints, or you might just know that they're unused (which leads to problems when you don't have the original function definitions to turn to; that's why C++ templates have to be in headers (I think that's true, at least? I might be wrong on that; I just use the magic, I don't understand it)).


I don't really know anything, but isn't this how Haskell and Elm work? With similarly expressive type systems?


There's a general mismatch between popular conceptions of type system expressiveness and actual type system capabilities. Elm has a very simple type system. In fact it is very close to having among the simplest type systems in any statically typed programming language (the only things that really set it apart are tagged union types and record types). For example Java has a much more expressive type system than Elm in many ways. What perhaps makes Elm's type system seem advanced is that the runtime essentially never violates the guarantees of its type system, unlike Java (depending on your opinion of unchecked exceptions).

Haskell has a much more expressive type system than Elm, but still lags behind Typescript unless you turn on a truly gargantuan number of extensions.

It's also a bit weird to compare the languages because Typescript has a very different typing regimen than Haskell or Elm. Typescript is entirely structural while Haskell is almost entirely nominal and Elm is mostly nominal (with the exception of records).

Typescript has an extremely expressive type system. It is in fact so expressive I'm amazed that it's gotten so much adoption when languages like Haskell are still considered "advanced." I suspect it has to do with the semantics of the languages rather than the type systems.


> Haskell has a much more expressive type system than Elm, but still lags behind Typescript unless you turn on a truly gargantuan number of extensions.

IIUC: it's not a total order, Haskell (even '98) has things TypeScript doesn't[0], and TypeScript has things that Haskell (even with extensions) doesn't[1].

[0]: eg. higher-kinded types [1]: eg. (convenient) row polymorphism


Yeah you're right it's not a total order. I was being fast and loose and conveying a subjective feeling.

RE row polymorphism, Typescript doesn't quite have what users of ML-like languages are asking for when they want row polymorphism (which is usually parametric polymorphism rather than subtyping). But it's close.

As for convenient... well I'd argue by the time you've got the whole cornucopia of GHC extensions at the top of your file nothing is quite convenient at that point.


> RE row polymorphism, Typescript doesn't quite have what users of ML-like languages are asking for when they want row polymorphism (which is usually parametric polymorphism rather than subtyping). But it's close.

How would you distinguish this from the following?

    function foo<T>(x: T & {field: number}): T {
        ...
    }
> As for convenient... well I'd argue by the time you've got the whole cornucopia of GHC extensions at the top of your file nothing is quite convenient at that point.

That was one part of my point. That said, I think the problems implied by language extensions are often exaggerated (probably not deliberately so, most of the time).

The other part of my point was that even with all the extensions you need, I've not found good row types in Haskell, although the particular failings vary by approach. Which isn't to say I can't get something that works well enough for my particular situation most of the time.


    function foo<T>(x: T & {field: number}): T {
        return x;
    }
    
    function bar<T>(x: T & {field: number}): T {
        const x0 = foo(x);
        // type error because x0 doesn't have field
        // would compile fine with row types
        const x1 = foo(x0);
        return x1;
    }
Yes it's true. I also sorely miss the lack of row types in Haskell (I miss them even in something like Idris where you can create them more easily, but it's still not great compared to first-class support).

EDIT: I may be being dumb. You could probably fix this by adding an intersection type again on the right-hand side. I think there was something else I was missing from TS, but I'll have to noodle on it a bit more.


Not dumb, you answered the question I asked :D

But yeah, I agree that duplicating the intersection produces another interesting question.

In any case, "convenient approximation of row types" probably still applies to TS more than Haskell. And possibly also "more convenient row types", but that remains TBD.


Most typed languages work like this way. For example, Java compiles down to the bytecode. The JVM bytecode doesn't have preserve generics information. But Java code has generics so when it compiles down to the bytecode, it removes that information. This is almost true for all high level languages. Kotline's type system is very different than Java but it also compile down to the bytecode so Java and Kotline shares their libraries without sharing the type system. Type Script is no different. Only problem here is that Type Script doesn't bring its own "runtime" or libraries.


> These kinds of “wacky types” are things you can statically analyze but not really “compile” per se. That’s why it’s a good fit for being transpiled down to a language with no types at all.

> TL;DR, TypeScript is not really a “typed programming language”, it’s a static analysis tool. It’s not really designed to “run”.

I think you have an interesting point but you've come to the wrong conclusion. Assembly language doesn't really have any notion of types (there is some notion of sizes as some instructions operate on, for example, single words and some instructions like SIMD ones operate on multiple words at a time) yet we still consider C, C++, Rust, Pascal, etc to be "typed programming languages". It's really only languages that compile to some kind of VM which have a notion of types at the runtime level and even then, many VMs erase some of those details like the JVM and generics.


This isn't strictly true, and @erikpukinskis has a point. Typed languages often do have type info that can make it to runtime. `typeof` -- virtually the only tool JS has for type reflection -- doesn't even compare to the richness of TypeScript. All of that richness is indeed lost in transpilation.

In C++, for example, we have `decltype`, `typeid`, type traits, RTTI, and so on -- all of which are available at runtime. Not to mention that in certain special cases, we also have the de facto storage of type information that makes it into binaries (e.g. discriminated union types).


You don't need any type information at runtime if your type-system is strong enough. You need only types at runtime to do reflective checks, and you need those only if your type-system can't give the needed guaranties statically at compile time.


If you're impressed with TypeScript, you should definitely check out ReScript (formerly BuckleScript). Its type system is clean and sound, and yet compiles to native JS without overhead. https://rescript-lang.org/docs/manual/latest/introduction


Ah yes. I looked at this a while ago. I liked it but then ran into a brick wall with a lack of support for an equivalent to async/await syntax. It has wrappers for JS promises but they are clunky and awkward, unless someone wants to enlighten me?


Try this for some syntactic sugar: https://github.com/digitake/bs-promise-monad


Just want to add that the genType project can even compile ReScript to Typescript!


don't get your hope too high it's just the intersection between the two, advanced typescript types are lost.


TS compiled to JS without any "overhead". This is not what the OP said.


I don't think even ReScript can do this (yet?)


given it's just ocaml I don't think it can do this ever, not that it's necessarily a bad thing.


Wow, when I read this earlier, I was dismissive (because I am really happy with typescript). But reading through the comparison - it seems to have many of the great parts I like in Typescript.

Then, I noticed it has the pipe operator! I'm going to be giving this a try.


I had been thinking that WebAssembly or CLR could be the target for TypeScript and that would allow to bring her own base class libraries etc.

But if you look closely again, the beaut & wide adoption of TS is because it has no "runtime" or no extra libraries or APIs to learn. Its the JS.

This also makes it easy to iterate quickly on Type System and bring new features. Having its own runtime means, that wouldn't be easy.

But I completely agree that TS right now has perhaps one of the best and most intuitive Type System than any other language.


Interesting referencing a programming language as a feminine article like that... Not saying it's wrong, just unusual to me.


People downvoting an observation on a curious use of language? If it's because of some perceived political motive, then that's really sad.

I really did just wonder why anyone would anthropomorphize a programming language, by ascribing a gendered pronoun to it.


They're probably a native speaker of gramatically gendered language (like German, most Slavic languages etc).


My native languages has gendered objects, and "language" is feminine. Might be the same case


I'm a bit torn on whether an advanced type system like TS produces better api designs for libraries or whether it promotes bad designs that need advanced typings to be safe and sound.

I feel typescript is in many ways closely tied to JS because many of the advanced typing features don't make sense outside of existing javascript patterns.

We all talk about simplicity so what makes type systems different that we get excited about their advanced and complicated features.


> I'm a bit torn on whether an advanced type system like TS produces better api designs for libraries or whether it promotes bad designs that need advanced typings to be safe and sound.

Well, I'd argue the type system surfaces the pain faster than the untyped variant. In the untyped version, you have to read the code to see that, "it's either the number 4, or a function which may produce the number 4."

Once you see you need a weird type signature to represent 4 OR a function which produces a number, then your spidey-sense kicks in and says, "this is getting gross, maybe I should either always return just the number, or always return a function which produces the number." This avoids a branch in the code that consumes the result where you check the type and do something based on that. At a meta-level, this is the type system telling you something about your program that you might otherwise miss if you were so focused on getting something working.


I agree on type systems being helpful, my point was about immensely flexible type systems like TS which don't force any usual constraints of designing with sound types. I do appreciate a good sound type system that constraints the behavior of my code in well expected ways.


My experience has been that complex APIs will be complex whether they're statically typed or not. It's always possible to add accidental complexity to a design, but an essentially complex problem domain necessitates an essentially complex solution. If that solution includes a static type system, then the type system needs to be at least as complex as the domain.

I'd be interested to hear more about the TS features that you find to be JS-bound. My experience with TypeScript gets deeper every day, but I haven't yet run into anything that was obviously only necessary because TypeScript is a layer over Javascript. I'd appreciate the benefit of some additional perspective there.


Sure, let me start with a disclaimer, I have used TS briefly and found it great but I personally prefer Elm's type system.

My understanding is TS was designed specifically to statically type existing JS patterns in popular libraries and frameworks and while that's great, newer libraries rarely need such flexibility if they aim for type safe designs from the start.

When I use typescript I often get the feeling there are multiple ways of typing the same function, which is reasonable given that one of TS goals was to embrace the JS ecosystem and that meant covering multiple overlapping use cases.

Consider Enums and Union types in TS, at a higer level both express the same thing (this value can be one of these sets of values) but in practice there is lots of discussion on when to use which. I have not seen sum types (as they are usually called) being implemented in two different ways in other languages.

I think one of the ways to think about TS is that it was designed to help programmers better understand their JavaScript code by adding types while reinforcing their designs in a safe manner so if TS didn't have to interface with existing JS code many of the cool tricks it's picked up would not be useful I think.

Finally on the point of expressivity, I'm interested to know how TS helps in modelling application logic to make invalid state impossible.


What do you mean by "no JS overhead"?


I'm glad you asked. :-) For a start, number types. I want i32, i64, u32 etc. JavaScript (and therefore TypeScript) only has "number". Yes we now finally have BigInt but it's not ideal for JIT optimisation.

Object prototypes is a weird part of JS that we really could do without. If you really want that, use class syntax.

The Date object. Need I say more...

Little things like Object.keys() should return a (keyof T)[] rather than a string[] but can't due to JS edge cases.

Want first class tuples/immutable arrays.

Many other new syntaxes that can't be implemented due to the need for JS compatibility.

I'm sure there are others that I can't think of right now...


Object.keys() is always frustrating.

Also, the array access not returning an optional type (I wish there was a strict option for that) often gets me.

We need a middle ground: “TypeStrict“ that can clean up some of the edge cases.

If you need strict integers, you can create your own fake type:

type Int32 = number & {__type:'Int32'};

I do this with strings sometime when I want a string subtype that can’t be accidentally assigned by a normal string.


In TS 4.1+ there _is_ an option for strict array access called `noUncheckedIndexedAccess` [1]

[1]: https://devblogs.microsoft.com/typescript/announcing-typescr...


Awesome!

I was just searching for that and all I could find were old github issues saying it was working as intended.

Thank you!


Good news, safe array access is coming in TS 4.1

https://devblogs.microsoft.com/typescript/announcing-typescr...


For anybody interested in this technique: the googleable term is "branding". It's bringing some nominal typing into TS's structural typing


I wrote an article about this a while ago: https://codemix.com/opaque-types-in-javascript


Cool, I never knew what that was called. Thanks!


For new code, I think most JS devs have already switched to using the class syntax. Also FP in JS is alive and well if you want to avoid dealing with prototype chains altogether.

I think we can all agree the Date API sucks, but life can still be good if you just give in and include a date library. Also, there's some light at the end of the tunnel with Temporal proposal coming along [1].

Object.keys returning string[] is purely a TypeScript design decision, coming from how TypeScript chooses to model object subtyping [2].

  type Foo = {
    a: string
  }
  
  const keysOfFoo = (obj: Foo) => Object.keys(obj)
  
  const foo = {a: '', b: 5}
  keysOfFoo(foo)
The fact that this passes type checks is a conscious TS design decision that comes with both advantages and disadvantages. It wouldn't have to be this way; Exact types [3] could potentially be used to describe that if the type is exactly Foo, it's safe to assume Object.keys(foo) is (keyof Foo)[].

For first class tuples (and records), there's a stage 2 EcmaScript proposal coming along [4]. There's of course also the readonly [] type in TypeScript if you only need the safety of not accidentally pushing to an "immutable" array. First class language support could have nice additional features though, like strict equality.

Anyways, if these are the things you don't like about JS/TS, I don't think AssemblyScript will be the answer for you, as the goals of that language seem to be entirely different from "fixing" old JS cruft. Apart from the i32, i64 stuff I guess.

[1] https://github.com/tc39/proposal-temporal

[2] https://github.com/Microsoft/TypeScript/pull/12253#issuecomm...

[3] https://github.com/microsoft/TypeScript/issues/12936

[4] https://github.com/tc39/proposal-record-tuple


> I don't think AssemblyScript will be the answer for you, as the goals of that language seem to be entirely different from "fixing" old JS cruft. Apart from the i32, i64 stuff I guess.

Basically AssemblyScript try to fix JS runtime as well. See this: https://twitter.com/AssemblyScript/status/114604718874680934... and https://twitter.com/AssemblyScript/status/114466636945316659...


That pull request from 4 years ago with regards to Object.keys is worth reading if you already or are considering writing a “type wrapper” for Object.keys, especially now that better tuple types have landed and string template types are coming.

Runtime types can still get you there. I’ve been happy with myzod for this purpose, which feels like writing TypeScript while getting you everything you need to validate at runtime in a very performant package.


Another potential benefit - runtime metaprogramming. All types are erased when converted to JS, but I can imagine a lot of utility for runtime types in Typescript.


Obligated mention of runtypes (https://github.com/pelotom/runtypes) that bridges the gap between type-land and runtime-land. It's not seamless at all but if you have a serious need for this sort of thing, it's great


Not quite the same, but reminds me a bit of yup (https://github.com/jquense/yup)!


I think there is some experimental decorator syntax which will allow for extracting type info. Does runtypes interop with that? Or is planning to?


there's the reflect-metadata package but the type information it exposes is very limited, "object" instead of {a: "foo"} and so on.


Why not use a language that has the benefits of TS + everything you say and is designed to be independent from JS.

I'm thinking of OCaml for example. Why not use OCaml rather than TS?

Edit: Actually the idea is so obvious that you have ReasonML that can compile both to JS and assembly and is basically OCaml with a JS-like syntax: https://reasonml.github.io/docs/en/what-and-why

Isn't that exactly what you're thinking of?


OCaml doesn't have all the benefits of TS, when it comes to the power and the flexibility of its type system.


Little things like Object.keys() should return a (keyof T)[] rather than a string[] but can't due to JS edge cases.

This was something that confused me when I first started learning Typescript but is actually not an edge case but a fundamental feature of Typescript's structural typing.

A type T is only guaranteed to be at least (a subtype of) T. Which in the case of objects means it has at least those fields, but potentially more. For instance

  interface FooBar {
    foo: string
    bar: string
  }

  interface Baz {
    baz: number
  }

  const fooBarBaz: FooBar & Baz = {
    foo: 'foo',
    bar: 'bar',
    baz: 3
  }

  
  const printFoobar = (fooBar: FooBar) => {
    for (const key of Object.keys(foobar)) { 
      if (key !== 'foo' && key !== 'bar') {
        // If Object.keys was (keyof T)[] Typescript would claim this branch is unreachable
      } 
    }
  }

  printFooBar(fooBarBaz)
It's really worth thinking about Typescript types not as nominal objects like in Java or Haskell, but contracts about minimum functionality. Embracing this in terms of function arguments and return types makes testing and composing typescript code much easier. For example, if you were writing a AWS Lambda function in Typescript that only uses the body of the incoming event.

  export const handlerOne = (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
    console.log(event.body)

    return {
      statusCode: 200,
      body: event.body,
    }
  }

  interface MinimalEvent extends Pick<APIGatewayProxyEvent, 'body'> {}

  export const handlerTwo = (event: MinimalEvent): Promise<APIGatewayProxyResult> => {
    console.log(event.body)

    return {
      statusCode: 200,
      body: event.body,
    }
  }
handlerTwo only requires a handlerTwo({ body: '....' }) call in a test, instead of having to fill in all the extra properties of the APIGatewayProxyEvent that aren't even required. You might be tempted to use a type coercion in the test instead e.g. handlerOne({ body: '....' } as APIGatewayProxyEvent) but the problem with the coercion is that is in not checked at all by the compiler, it's essentially like temporarily using an any type. So if handlerOne is updated in the future to depend on more of the structure of APIGatewayProxyEvent Typescript will be none the wiser that the test requires updating (although granted hopefully the test would fail).

Anyway, that's a long winded explanation of why Object.keys shouldn't return (keyof T)[]. If you want to program generically over the Type of something I'd suggest making use of a runtime type library like io-ts instead which gives you a data structure representing the type to iterate over, etc.


> Want first class tuples/immutable arrays.

On this subject specifically, see the records & tuples proposal (currently at stage 2): https://github.com/tc39/proposal-record-tuple


When you say doing without object prototypes -- are you advocating moving away from the entire prototypal inheritance model? Or just not exposing it?

Your requirements list makes me wonder why you want to stick with anything related to JS at all? With all the transpiling options, web assembly, etc.


What are the edge cases for Object.keys()? I've always wondered why that returned string[]


Example:

  const x = { a: 1, b: 2, c: 3 }
  const y: { a: number, b: number } = x
  console.log(Object.keys(y))


Why not make TS compile to LLVM?


Can't speak for the GP but my single biggest complaint about TS is how it's basically a massive set of assumptions layered on top of JS. If the JS types at runtime don't align with your assumptions in TS, the whole house of cards can come crashing down.


This is why you need input validation. Eg, zod or io-ts.


Did not know about these. I wrote my own a while back because I couldn't find an example of this being done (fully type safe validation).

https://github.com/cjdell/type-safe-validator


Not exactly the same thing, but protobuf.js also works really well for this: https://github.com/protobufjs/protobuf.js


Which is the nature of the compiler. It maintains the promise that what you're writing is still JavaScript—in that you could just strip all the type annotations and have valid JS.

But I would love runtime checks for types. Even if that was a compiler step I could flag on—but that makes it a great deal more complex both to implement and reason about, likely—without starting fresh anyway.


A while ago I built this exact mechanism for Flow, it could be done in TypeScript if there was enough demand for it (it's rather a large effort) https://gajus.github.io/flow-runtime/


Why do you need runtime type checks if the compiler already checked the type?

IMO, runtime type analysis is a code smell. There are few scenarios where the simpler and more efficient solution involves querying type metadata at runtime. Particularly in a language that directly supports dynamic dispatch, you should think hard before adding anything resembling an "if typeof(mything)" check.


The obvious use case is the interface between something that’s not type-checked by the compiler and something that is. It’d be useful to be able to do `runtime_cast<Foo>(JSON.parse(foo_json))` on, say, the response to an API call and know that the result was definitely a valid Foo without having to write a bunch of boilerplate to validate that by hand.

(Reading through the thread, there’s apparently some libraries that can do this; I’ll have to try them out next time I need this.)


Because there's no guarantee that compiler did that. Even if you do this in your code, the moment it gets called from pure JS, all bets are off. With proper runtime checks, you could get consistent behavior out of that.


I was thinking about this the other day i would love for TS flag that would turn an interface into an assert

const shout = (message: string) => console.log(message)

into something like

const assert = require('assert'); const shout = (message) => { assert(typeof message === "string"); console.log(message); }

of course this doesn't work for more complicated types


You should checkout myzod, which is very close to this experience.


Thanks for this


Or just plain zod


Genuinely curious, why would you pick zod over myzod?


At the end of the day, if you look at a low enough level in the stack, there are no types. In a very concrete way, types are a high-level idea, and if the high-level stops matching the low-level, then bugs crop up.

See for example Tetris implemented in Pokémon Yellow via runtime code injection:

https://www.youtube.com/watch?v=Vjm8P8utT5g

Their compiler missed that!

So I think there's good reason to switch from talking about types as "assumptions", and think of them as compile time "assertions". Something lower-level in the stack could break the high-level code, but that doesn't make the high-level code useless, just imperfect.


I get what you're saying however even at the machine code level there is a difference between an i32 and an i64. Also the "sizeof" of a struct is important too when calculating how far to stride with regard to memory addresses.

Types can and do affect the way programs run at the lowest level. Perhaps someone with a better understanding of compiler theory can elaborate further on this...


Probably running raw TS not TS -> JS -> Compiler flow. Running TS directly will have lots of performance benefits. Compiler will know the types and can optimize better.


Is this satire? What do you think is advanced about TS' type system and why?

Duck-typing that creates a minefield instead of providing correctness?

The unknown type that does the same?

TS is a step forward from JS, but JS's bar is so infamously low that making something better isn't a big achievement, especially compared to other languages with normal type systems.


I'm honestly curious. What other languages have a type system the would allow for this project? That is, (a subset of) SQL as a type. And assuming these languages exist, are they as ergonomic to the developer as TS while instrumenting the above?


Are you really calling this ergonomic? https://github.com/codemix/ts-sql/blob/master/src/Evaluator....

Same thing but actually readable and maintanable would've been better implemented as a compile-time (build-time) script, basically source code generation.


Nobody said it's ergonomic to implement SQL in a type system. It was simply an answer to your question "What do you think is advanced about TS' type system and why?". It's advanced because you're able to implement SQL in types alone when with most languages you couldn't. The question on ergonomics is for the other languages where you can do this is it more or less ergonomic than this. Again that doesn't mean this IS ergonomic it's about finding a language with a powerful type system that could do it MORE ergonomically.


This thread made me read about conditional types in TS and in general more about its type system. I was ignorant. I agree now that it's advanced. Previously I only heard bad things about it without bothering to study it myself. Turns out, it's not all bad, there is some good as well. Still wouldn't want to work with it, ever; but that's beside the point.


Well said. It's amusing imagining the soup that would result from attempting to do this sort of thing in various other languages!


Yes. What you don't see it that the TS language server provides things like autocomplete, definitions on hover, squiggly lines, sane error messages, etc. to the developer as they write the parsing code. What you don't see is that because that file compiles, it is guaranteed to parse (certain) "SQL" strings into types. This is because the code doesn't have to actually run to work, rather, TS's powerful type system does the parsing.

Source code generation wouldn't really provide the same end-user ergonomics and, frankly, would barely resemble the same project. How would that work? You generate source code from a SQL string and a data source? At that point you might as well just... write the actual code. The point of this package is that type information is dynamically assigned according to arbitrary "SQL" queries at compile-time[0].

Have you gone through short exercise I outlined below? Surely you can't think it would be better to manually generate type information than have it automagically available and enforced instantly as you work?

The closest thing I've seen to this is "type providers" in F#. Where, given a database connection, a "Provider" can offer compile-time contracts between your code and the schema of the database. What it does not do is provide contracts against arbitrary projections of the database (SQL queries). Of course one can write code that queries and transforms the data in a type-safe way, but these "transformations" must happen in F# for the compiler to infer any new types.

This project takes the above to a different level and infers the type of the result of an arbitrary (subset of) SQL operation before it is ever executed. It's fairly impressive.

[0] This has been pointed out more than once already, but this project does offer compile-time guarantees


Not sure if it makes it possible, but this is fun anyway: https://aphyr.com/posts/342-typing-the-technical-interview


What compile-time guarantees does that SQL library give? I don't see any docs or explanations at that Github repo and my TS knowledge is limited to be able to infer everything from a small code example.


Maybe clicking on the playground link would help your understanding of the project. You can literally hover over the types defined like:

      type EX = Query<"some query string", db>; 
and see the that the type as defined by the TS language server (i.e. in your editor) is the result of "running" the query without actually running the code. It's a little insane and somewhat akin to "type providers" in languages like F#.

For example, add this below `EX1` (line 66) in the TS Playground:

    let names = (persons: EX1) => persons.map(person => person.name);
Now change the column alias in `EX1` to something other than "name" and see what happens. You see that? There is now a compile-time error.


all of the functionality happens at compile time


TS is really getting Ninja with typing.

The ultra expressiveness of TS may not be a virtue. It may simply encourage us to be creatively academic with our designs, and do novel, non-obvious and hard to understand and maintain things in the code.

With some of the newer TS releases, I can hardly fathom through the release notes why such a complex degree of metaprogramming has utility for anyone, and what kind of corner cases people must be diving into. They are actually quite hard to follow, and I find myself trying to parse through the need for such a thing, literally unable to conceive of an example use case. Perhaps without a strict philosophy such as they have in Go, the designers just go on ahead and implement their favorite intellectual exercise, or something that helps them with a task in the TS compiler itself, which may not generalize very well.


This is a great demonstration of typescript's flexibility


Pretty interesting, but it doesn't seem to be possible to use the resulting values? Except as type declarations of course. Also, the playground link either fails (there are several errors) or doesn't produce any output.


hover over the type annotations to see results in the playground, that's the output :)

The errors aren't really errors, just warnings about unused types


yeah Typescript is compile-time-only type system, you can only use it as a type. still cool though


But if you declare a string literal as const, then the string runtime value will match the string type. So it would be possible to have a runtime result (that executes an actual db query for example) that is paired with the type result.


No, TypeScript is compile time only. There's no runtime type info.


The parent comment is saying that if you write the string twice, you can get the compiler to check #1 and the runtime to use #2.


Not exactly (there is no duplication), a string constant in typescript is it's own type:

  const sql = 'SELECT ...';
  type SQL = typeof sql; 
  // type SQL = "SELECT ..."


Very interesting project. If you want a deep dive into TypeScript generics, this project's code is truly a must-read! This project really shows how powerful TypeScript has become over the years.

Shameless plug: we're also working on project in the same domain. It actually uses the TypeScript compiler API to generate/update your SQL schema's and a typesafe (mongo like) API on top PostgreSQL. It's still very early stage, but if you're interested: https://samen.io


Maybe useful to build dynamic-types based on configuration files I think.


Is typescripts type system becoming like c++ template meta programming?


TypeScript is more powerful in some ways.


Only in some: the fact that the type system can know everything it needs to generate runtime code, but chooses not to do so, is kind of frustrating sometimes.


This whole magic can be explained in one sentence. `infer` is introduced to TypeScript's templating. The rest, I mean this sql parser is just the result of this new "tool".


This is insane!

Just curios, are there real-world use cases for stuff like this?


I could imagine working on very complex static JSON. You want to type-check selections and transformations on this JSON.

It's a very niche use-case, this is definitely more of an impressive showcase of the new Template Literal Types feature of TS :)


You can import a json document directly into typescript and the entire document is a readonly constant type of the entire json object literal.

I've used this with a template JSON document that has an instance of all possible types in the runtime json.

Sometimes you might have to loosen the type slightly, but it works great for a quick start.


You can embed any language interpreter into TypeScript and have that language syntaticaly checked by TS compiler.


That's pretty neat. What I really want is this:

  function fn(query:string){
    const stuff = // do some stuff with `query` variable
    return createType(stuff);
  }

  type MyType = FromJS<fn("SELECT id, name AS nom FROM things WHERE active = true")>;

This way we could compute types on the fly with JavaScript instead of creating monstrous types in TypeScript types system.


Interestingly this is something you can do in Zig with its `@Type`. Zig's compiler can eval (some) Zig code at compile time, this code can produce new types, and you can mention those types anywhere types are normally mentioned.


wouldn't that be neat...

However, there are Java frameworks that basically generate a type system from your database so whatever columns you query ends up being static types that can be determined at compile time. https://www.jooq.org/

Java is really quite cool. I wonder if we'll ever get a typescript version of jOOQ


I think we eventually will be able to evaluate JS in context of types and return whatever we want even asynchronously. Now we have some combinators given by TS authors which allow us to compose some limited set of computations. I don't know how Java does it, but I hope TS will do it better :-)


if we are talking about code gen then it exists in every language, nothing extraordinary here. In nodejs, there is prisma for example. This typescript thing generates static type on the fly, not by generating code.


Similar libraries are also available for Rust, see diesel [1] or sqlx [2].

[1]: https://diesel.rs/ [2]: https://crates.io/crates/sqlx


Is this just storing data in memory and querying it with SQL?

I use DynamoDB Data Mapper in production (https://github.com/awslabs/dynamodb-data-mapper-js) - would be cool to see something like this for SQL. Effectively define a class with TS decorators to automatically generate tables and columns.


https://typeorm.io/ support:

> MySQL / MariaDB / Postgres / CockroachDB / SQLite / Microsoft SQL Server / Oracle / SAP Hana / sql.js


I'm interested to see if I can use these new template string type features to add more/better type checking and inference to my TS Postgres library[1]. My suspicion is that the TS recursion depth limit of 50 might limit this rather seriously.

[1] https://jawj.github.io/zapatos/


Love Zapatos. Hope there’s a way to get this working!


I love Typescript -- I just wish there were an alternate runtime (.Net core CLR/DLR or other) that offered shared-memory green and/or native threading for true support for embarrassingly parallel but not vectorized workloads. FWICT, I don't think Deno changes things here much. Typescript on the BEAM would be interesting to me as well.


I would absolutely love to work with the TypeScript type system on BEAM, if it didn't leak as much of Erlang particulars as other BEAM languages like Elixir do.


I think I’d prefer compilation to Go.


DLang can sort an array at compile-time using a normal function: (CTFE)

  void main()
  {
      import std.algorithm, std.conv, std.stdio;

      "Starting program".writeln;

      // Sort a constant declaration at Compile-Time
      enum a = [ 3, 1, 2, 4, 0 ];
      static immutable b = sort(a);

      // Print the result _during_ compilation
      pragma(msg, text("Finished compilation: ", b));
  }
It can also reflect at compile time on user-defined annotations, which can store arbitrary metadata on any symbol, to generate arbitrary code at compile time. They serve a similar role to procedural macros from Rust, but they're easier to use than the earlier. Thanks to mixin templates, you can inject arbitrary symbols, making D's UDAs just as flexible as Python's decorators.


Genuinely curious: is this actually useful or just a cool thing that is useful simply for the fact that it can be done?


Personally I find RegExp more readable: SELECT(.)FROM(.)WHERE?(.)ORDER BY?(.)LIMIT?(.*)


I think that TypeScript is nice but I would never touch a database running on javascript.


Oh, this isn't running on JavaScript - this is running on TypeScript's _typer_!

... which is implemented in JavaScript ... er ... don't know if that changes your desires to use this in PRODUCTION.


This feels a bit like C#'s "Linq to Objects". Could actually be useful!


I'm viewing this on my phone, so maybe I'm missing something, but isn't this getting all of its typing information from you explicitly declaring a result set `as const`, and then subsequently doing `typeof` that const value? How does that help in real world queries, where the result sets are dynamic? Also, how does it help beyond just naturally declaring a result set as const without the use of this library?


oh let's be clear there's no sensible real world use cases of this thing. it's a demonstration of what TS's type system can do


Art is all about making interesting things that serve no functional purpose, though often they have useful byproducts.

This is a work of art.


One use case of something similar is generating a regular expression matcher at compile time for example. This is what the C++ library https://github.com/hanickadot/compile-time-regular-expressio... does.

There are a lot of uses of compile-time code. One I have been interested in is generating bindings to other languages or generating serialization code, or even generating bytecodes for VMs with specific bytecodes for accessing your native data structures (instead of adding a level of indirection).


This project allows you to query (transform) type definitions as data to create new type definitions using SQL.

Dynamically transforming types could be useful to generate complex types based on other complex types.

This project takes it to the extreme by adding an SQL interface to it


Ya I took a look again. It is very strange, but impressive.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: