
LInQer – C# Integrated Queries ported to Javascript - Siderite
https://github.com/Siderite/LInQer
======
skrowl
More info here - [https://siderite.dev/blog/linq-in-javascript-
linqer/](https://siderite.dev/blog/linq-in-javascript-linqer/)

Pretty neat project. LINQ is a great time saver in .NET development, I'm
looking forward to messing around with it in javascript.

~~~
gameswithgo
its a shame that linq had the overhead it does when used with collections of
objects. at work right now i am often having to de-linq things

~~~
fuzzy2
I’m interested in what performance problems you’re seeing. Maybe then I won’t
fall into the same trap in the future.

Off the top of my head, the only possibility I see is allocating objects with
`Select` or the like. If one can avoid that, the only overhead should be the
predicate/selector method calls...?

~~~
ygra
At the very least, each LINQ method needs an enumerator, which, since it's
typed as an interface, cannot be stack-allocated, even for value types (like
List<T>'s enumerator). For Roslyn they noticed that the additional GC pressure
was indeed a problem. But for most projects I wouldn't say it's much of a
difference (and definitely don't start banning LINQ or other things unless a
profiler tells you there's a problem).

~~~
fuzzy2
But that’s ~one small-ish object per operator. Unless you’re using
`SelectMany` or otherwise generating queries “in a loop” I don’t see how that
could be so bad.

~~~
gameswithgo
the gc pressure is normally not so bad, but there is also more indirection in
each iteration. So you iterate slower, and evict more data on your cpu caches.
Whether this "matters" depends of course. Are you doing a ton of work in each
step through the collection? If so, the overhead of linq will be small
relative to that. How much do you care about wasting microseconds and eating
more battery/power? At scale these little things add up, it can get important
quick. On small devices it can be important. For a personal script running on
a powerful desktop, it definitely doesn't matter, and everything in between.

My LinqFaster repo has some benchmarks showing time differences for some
common operations in a fairly worst case scenario (where the work done in each
iteration is small)
[https://github.com/jackmott/LinqFaster](https://github.com/jackmott/LinqFaster)

It would just be nice if C# reliably compiled this abstraction down to
efficient code, the way C++, Rust, and Java Streams do.

~~~
davidgl
There is also
[http://nessos.github.io/LinqOptimizer/](http://nessos.github.io/LinqOptimizer/)

------
macca321
These libraries miss the point of linq. Yes it provides a nifty syntax for
munging lists, but Real C# LINQ provides that munge as a abstract syntax tree
which can be interpreted before executing it.

This means it can rewrite the native language query, and say, turn it into a
SQL or graphql query, or a stream filter, or check against cached data or all
kinds of clever things.

~~~
flamtap
It does seem to miss the "Query" part of "Language INtegrated Query". Though
the methods are convenient in any case.

One of the most mind-blowing moments I can remember in my career is when I
realized just how powerful the tandem of LINQ and IQueryable was.

Now, writing your own LINQ provider is no walk in the park, but is a really
satisfying project and worth doing once, if only to get a deeper understanding
of what's going on with expression trees and the inner workings LINQ. It's
almost like a language within a language.

------
thrower123
I enjoy LINQ, but the real power comes from when the implementation is more
complex than just renaming your standard functional programming map, fold,
filter, etc functions to more SQL-y aliases.

With that power comes the responsibility to actually understand what it is
that you are doing, and where the abstraction leaks. If you are not careful,
and do not understand the ramifications of what you are writing, it is very
possible to write some horrifically inefficient code, or code that looks
correct, but will break at runtime.

I have seen many a horror with Entity Framework or Linq-to-SQL when I dug in
to fix badly performing code, and hooked up tracing to track the SQL code that
is actually generated and executed. Multiple iteration of an IEnumerable is
very easy to run into, although static analysis is getting good at flagging
that with a warning. Trying to use an IEnumerable that hasn't been
materialized from the expression tree into the actual data after the database
connection or other resource has been disposed is another common gotcha.
Understanding what operations can be translated by the expression tree engine
is very important. In LINQ database ORMs, the way that foreign keys behave can
vary wildly depending on how the query is written and on how the ORM happens
to handle those accesses.

~~~
nizmow
Honestly this just sounds more like a complaint about ORMs than LINQ in
general, though I do agree with it.

------
brazilianabroad
I don't understand why my comment keeps getting flagged. "You need to study
more" is not meant as an insult, simply as advice.

'You're too ignorant to be judging whole ecosystems as "shit".' is also not
meant offensively, simply as a statement of fact.

Someone who presumably writes code for a living, but doesn't understand binary
floating-point numbers, and finds themselves in a position to criticise
JavaScript as being "shit" while not even having a sufficient understanding of
their preferred technology to realize that it has the same (expected) behavior
is _objectively_ someone who "needs to study more" and who is "too ignorant to
judge whole ecosystems as 'shit'".

Edit: Before someone says it's because I wasn't being helpful, both my
comments also linked to
[http://0.30000000000000004.com/](http://0.30000000000000004.com/) which
contains an explanation.

~~~
codetrotter
Yes well, it comes off as flaming, and no matter who started it it is only
adding fuel to the fire. Flamewars are not productive, no matter which side of
it each of the people are at.

Statement of fact or not, and insult or not, the question is, does the comment
contribute to the conversation? And does it do so in a tone that is likely to
be received well by others?

Even if someone is inexperienced, is telling them to study more and telling
them that they are ignorant useful? Not really IMO. Aside from the thing about
tone, it is also a question of whether it is giving actual actionable advice.

Actionable advice includes pointing to specific sources of information, like
one of the sibling comments did.

Non-actionable advice includes telling someone to just study more in general.

~~~
brazilianabroad
> Yes well, it comes off as flaming,

Saying that someone who doesn't understand binary floating point numbers is
ignorant and needs to study, while pointing them to a study resource, isn't
flaming, it's an act of selflessness. It's a favor, a grant of knowledge and a
nudge in the right direction.

> Even if someone is inexperienced, is telling them to study more and telling
> them that they are ignorant useful?

Yes

> Aside from the thing about tone, it is also a question of whether it is
> giving actual actionable advice.

The sibling comments did not exist when I posted my first one. It had also a
link to actionable advice.

Mind you, no-one is forced to know how binary floating point operations work,
as long as they don't intend on passing incorrect judgement that's brash and
impolite.

------
hebrox
If you like this syntax and work with TypeScript and Knex, you can checkout my
typed-knex library: [https://github.com/wwwouter/typed-
knex](https://github.com/wwwouter/typed-knex)

All feedback is welcome :)

~~~
jeswin
Because there is no built in support to create expression trees from query
expressions, your library is forced to re-implement common query patterns with
a custom API. The result is not as clean as C#. And there is some loss of
type-safety (The 'SUM' in the groupBy() example).

An alternative approach could be to parse real JS expressions using babel (at
runtime) - essentially giving you the same flexibility as C#.

So you could allow users to write:

    
    
      typedKnex.query(User).where(u => u.id > 100 && u.city === "Calicut")

~~~
recursive
> Because there is no built in support to create expression trees from query
> expressions, your library is forced to re-implement common query patterns
> with a custom API.

Well, it _could_ convert the "delegates" back to strings, and re-parse an AST
from that. Not that I'm volunteering to write it.

~~~
jeswin
That works. An alternative could be to do it during build with a babel based
tool.

Earlier, I wrote a tool called chimpanzee[1] to parse such ASTs into SQL. Not
easy (but not too hard either); there are so many different ways to express
something in JS. Here's an example (using chimpanzee) which checks if an
expression is doing a sort (on an array/table) and to extract sort fields and
order. [https://github.com/jeswin-unmaintained/isotropy-ast-
analyzer...](https://github.com/jeswin-unmaintained/isotropy-ast-analyzer-
db/blob/master/src/schemas/sort.js)

[1] chimpanzee:
[https://github.com/jeswin/chimpanzee](https://github.com/jeswin/chimpanzee)

------
mythz
As LINQ has been positioned as one of C#/.NET's biggest strengths their 101
LINQ Examples are a nice set of simple code examples to compare how well
different languages fare against each other:

\- Swift [https://github.com/mythz/swift-linq-
examples](https://github.com/mythz/swift-linq-examples)

\- Kotlin [https://github.com/mythz/kotlin-linq-
examples](https://github.com/mythz/kotlin-linq-examples)

\- Java [https://github.com/mythz/java-linq-
examples](https://github.com/mythz/java-linq-examples) (Java 1.7)

\- Clojure [https://github.com/mythz/clojure-linq-
examples](https://github.com/mythz/clojure-linq-examples)

\- Dart [https://github.com/mythz/dart-linq-
examples](https://github.com/mythz/dart-linq-examples)

\- #Script LISP [https://sharpscript.net/linq/restriction-
operators?lang=lisp](https://sharpscript.net/linq/restriction-
operators?lang=lisp) (Interactive, .NET Lisp)

\- #Script Code [https://sharpscript.net/linq/restriction-
operators?lang=code](https://sharpscript.net/linq/restriction-
operators?lang=code) (Interactive, .NET JS-Like)

\- Elixir [https://github.com/omnibs/elixir-linq-
examples](https://github.com/omnibs/elixir-linq-examples)

\- Python [https://github.com/rogerwcpt/python-linq-
samples](https://github.com/rogerwcpt/python-linq-samples)

\- Groovy [https://gitlab.com/svkj/groovy-linq-
samples](https://gitlab.com/svkj/groovy-linq-samples)

Most languages fare well in both verbosity and readability so I don't view
LINQ as a major strength of C# anymore, it's just a well designed, typed query
language with the USP of being able to capture and traverse an expression's
AST which different LINQ providers can take advantage of by translating the
intent of the query into a different DSL, most commonly used by ORMs to
convert to SQL and execute the typed C# Expression logic on the RDBMS Server.

~~~
kccqzy
Isn't the point of LINQ to get monads into the language so that it can be used
by a lot of different things (seemingly having no relations to data
processing), rather than just data processing tasks?

~~~
goto11
Not really. You can have monads without Linq, but there is a particular use of
the 'from' syntax which can be used as a cleaner syntax for monadic
operations. But this seem more like a hack than an intended use case.

It is rumored that Linq was initially implemented using monads, but that
approach was abandoned due to performance reasons.

~~~
contravariant
That rumour seems like it's false, because not only is "implemented using
monads" a rather vague qualification, Linq has currently been implemented by
requiring objects to implement the SelectMany function, which has signature:

    
    
        MyObject<B> SelectMany(MyObject<A>, Func<A, MyObject<B>>)
    

or perhaps more familiar to the Haskell folks:

    
    
        SelectMany :: M a -> (a -> M b) -> M b
    

i.e. exactly the 'bind' operator for Monads.

I think you also need to implement a 'Select :: M a -> M b' operator, rather
than a 'return :: a -> M a' operator, which would be closer to a true Monad,
but the 'bind' operator is the one that really matters anyway.

~~~
goto11
The rumor was that the standard extension methods on IEnumerable<T> like Join,
Select, Where, Group were all implemented on top of SelectMany.

I call it a rumor because I cannot remember where I read it, so it is quite
possible I have misunderstood it.

------
remontoire
I would be remiss if I didn't mention something from clojurescript land that
accomplishes something similar:
[https://github.com/tonsky/datascript](https://github.com/tonsky/datascript)

Datascript allows you to write datalog queries in ClojureScript(and
JavaScript) that operate on a b+ tree maintained by the library. It includes
first class relationships and unique keys.

------
pmoleri
It looks pretty good. However, I find it similar to what you can do with wu.js
or immutable.js@4, although of course LInQer would be more appealing to
someone coming from C#. I wish you good luck with this project.

------
phillc73
Thought it also worth mentioning the Julia package, Query.jl, which supports
LINQ-like syntax:

[https://github.com/queryverse/Query.jl/blob/master/README.md](https://github.com/queryverse/Query.jl/blob/master/README.md)

[http://www.queryverse.org/Query.jl/stable/linqquerycommands/](http://www.queryverse.org/Query.jl/stable/linqquerycommands/)

------
jeswin
A big plus over previous attempts seems to be that this is written in
TypeScript.

Previous HN discussion on LazyLinq, which is another JS implementation -
[https://news.ycombinator.com/item?id=15913571](https://news.ycombinator.com/item?id=15913571)

------
haapanen
Here's another one that decided to use JS naming conventions while providing
Linq API: [https://github.com/tomi/fromfrom](https://github.com/tomi/fromfrom)

------
aabbcc1241
When would you really need this complex library over the native methods
supported by Array?

I know there are overhead to convert map and set into array but it should be
fine for non huge data set.

~~~
oaiey
Interesting question. I share this train of thought. When I restarted writing
JavaScript (thank god with TypeScript ;)) and I realized filter/map/reduce
array methods, my fear of lacking LINQ vanished away. Reduce is painful in the
beginning (because it is different to SQL's GroupBy) but it is a foundation
for so much more (once you REALLY understand it).

Some more helpers are useful (first, last, max, take, skip ... all with
variation). They should added to the core language.

~~~
jolux
Linq has Reduce as well, known as Aggregate (foldr/foldl in most functional
languages).

~~~
oaiey
I know, but thanks to convince function like groupby, max, etc... You never
need that.

~~~
jolux
Kind of, except folds can be used to replace a lot of complex arbitrary
iteration too. I hardly ever use loops anymore thanks to folds.

------
achou
See also IxJS, which is a pull-based version of RxJS:

[https://github.com/ReactiveX/IxJS](https://github.com/ReactiveX/IxJS)

------
greatjack613
Really cool, but the description of the project states,

"ported to javascript for performance"

In what alternative universe are things ported to javascript for performance?

~~~
HereBeBeasties
You have the wrong end of the stick. The author means that he's ported it to
achieve better performance than using Javascript's built-in map/filter./etc.
operators because those do not lazy-evaluate and do not chain together
efficiently, creating an array at each invocation.

------
scarface74
Public Service Announcement: LINQ is not just syntactic sugar for processing
collections.

The beauty and power of Linq is that it is translated to expression trees at
runtime, can be interpreted by a provider, transformed to basically anything -
sql, MongoQuery, intermediate language, etc. - and you can pass around types
of:

    
    
      Expression<Func<T,bool>> 
    

to different providers.

Like

    
    
       repo.Find<Customer>(c => c.Age > 50 && c.Sex == “M”);

~~~
mythz
It's a compile time, not a runtime feature.

It can't be implemented at runtime, if your method is passed an opaque
compiled delegate, it's not possible for the .NET Runtime to deconstruct its
pre-compiled AST.

If your method accepts a `Expression<Func<T,bool>>` the C# compiler passes it
the AST of the Expression instead of the compiled delegate.

For an example, paste the code below to
[https://sharplab.io](https://sharplab.io)

    
    
        using System;
        public class A {
            A() => M<int>(x => x == 1);
            public void M<T>(Func<T,bool> expr) {}
        }
        public class B {
            B() => M<int>(x => x == 1);
            public void M<T>(System.Linq.Expressions.Expression<Func<T,bool>> expr) {}
        }
    

and look at what the compiler generates for class A vs class B's constructors.

~~~
kjlkjljklkjl
but, Expression can be created from string, doesn't it?

~~~
mythz
Parsed by who, the C# compiler? I'm not aware of anything that generates C#'s
Expression AST from a string, other than a C# Compiler.

You can use Roslyn in code, but that's using the C# compiler in process.

~~~
gpderetta
The code->AST is done of course at compile time (there is no reason to delay),
but I believe the expression tree itself is processed by linq at runtime. You
can build expressions programmatically at runtime as well (and then ask the
runtime to optmize them to bytecode). I remember doing it years ago, it was
surprisingly simple (and very powerful).

~~~
mythz
Of course the Expression's AST generated by the C# compiler is analyzed at
runtime, how else could it be done?

------
Rapzid
Does it AST? Time for a TypeORM 2.0; Entity TypeWork?

------
duxup
Is there a good high level explanation for what this is for someone not
already familiar with LInQer or C#?

I'm googling around and still kinda not sure.

~~~
np_tedious
It's a bunch of functions that can be performed on an enumerable (array, set,
, etc) type. Commonly several are changed together.

Select (aka map)

Where (aka filter)

SelectMany (aka Flatmap)

Accumulate (aka fold)

First (with optional predicate function)

Max

Min

GroupBy

Count

And so on. The readme lists them all

~~~
bmurphy1976
It's more than that though. It's also a special monad like syntax that takes
advantage of those methods as well as transformation that takes the
declarative form of those methods and converts them into something else ala
SQL.

~~~
james_s_tayler
IIRC only SelectMany meets the criteria of a monad.

The powerful part about LINQ is it builds an AST and there are many different
LINQ providers which can translate that AST into some instructions to interact
with other systems. The typical one is converting it to SQL commands but you
could plug anything into it.

------
adgasf
The pipeline operator and some free functions is a superior alternative.

    
    
        const map = f => xs => ({
          [Symbol.iterator]: function * () {
            for (const x of xs) {
              yield f(x);
            }
          }
        });
    
        xs |> map(x => x * 2);

------
treve
Newb question: Can this translate to a MySQL query?

------
isuckatcoding
Did I miss the benchmarks somewhere?

------
yread
You can do a lot with pure js map, filter and apply:

[https://medium.com/@aikeru/a-c-linq-to-javascript-
translatio...](https://medium.com/@aikeru/a-c-linq-to-javascript-translation-
guide-6e1558fc1905)

