Sounds like exactly the problem of Julia language. On the surface, Julia seems slick in code and fast in running. But then you discover that
1. there are no objects, so when you switch from Python, instead of chains of methods, like `my_dataframe.groupby(...).aggregate(...).groupby(...).aggregate(...)` you need either (a) a lot of poorly readable nested calls, or (b) ugly "pipes", where the left side is implicit first param of the function on the right side: ` my_df |> groupby(...) |> aggregate(...) |> groupby(...) |> aggregate(...)`. (Writing them, you feel insecure of what is happening -- in this case, what gets passed to the function.)
2. You can't make it a CLI script, because it compiles the whole code WITH dependencies every time, and as soon as you import some serious libraries, compile times will skyrocket. I quickly hit 40 seconds of compilation with just geospatial lib and dataframes. These 40 seconds turned out to be A LOT when you develop interactively. And you can't build everything in a Jupyter Notebook, because eventually you'll have to switch to a CLI script.
I think I could put up with pipes, but the compilation times were the final argument against Julia.
> (b) ugly "pipes", where the left side is implicit first param of the function on the right side: ` my_df |> groupby(...) |> aggregate(...) |> groupby(...) |> aggregate(...)`. (Writing them, you feel insecure of what is happening -- in this case, what gets passed to the function.)
These are not ugly pipes. These are a very common feature in a lot of functional languages. Once you've used them, you want them everywhere :)
And of course you know exactly what is passed to the function. You wrote it yourself: left side is passed as the first param of the function on the right side.
I think it's not so clear a lot of times which function is called. Say you write a function that accepts your type as an argument. Cool, you compile it everything is good. Now you import a library, just so happens it also has a function that has the same name, and accepts an argument of the same parent type. Which function gets called? If you're reading the code and not intimately familiar with how imports work, all the imports in scope, and the function your coworker defined on like 5,000 in a random file you might not be sure unless you start debugging.
Pipes are fine, but methods and traits with a strong type system reduce any ambiguity there.
Doesn't mean you can't use pipes reasonably! But in Julia that seems like it could be harder to unpack.
> Say you write a function that accepts your type as an argument. Cool, you compile it everything is good. Now you import a library, just so happens it also has a function that has the same name,
This has nothing to do with pipes, this is a matter of namespacing.
This is why I generally dislike imports. It's common practice in Elixir (and I believe some other functional languages) to generally fully qualify function calls.
I'm sure some will think that is too verbose but I love it. Otherwise pipes are far more flexible than method chaining just because you aren't constrained to one "type". You don't always want to do this, but as per the example above, it's pretty handy. You also don't have to rely on a method you don't own returning an instance of itself.
well, it depends on how import works. in python you can't use a module unless you import it, (though you can import it in a way that the full path to the function is still needed)
but for example in pike import is only useful as a shortcut, and not needed if you use fully qualified function calls. and like you i never saw the point of import. using the fully qualified function calls makes the code more readable because i can quickly recognize library functions from custom code.
True enough about Python but I was responding more about pipes in general which Python doesn't have. I'm more confused about the original assertion that pipes give you lower confidence as to what is happening. But then I looked up how they work in Julia and saw there are two different ways they are implemented so I agree that that is maybe a little bit unfortunate, though I can't really say for sure as I've never written a line of Julia.
i saw pipes and import as two unrelated topics, and i was just responding to import topic.
with pipes i have no experience at all. but they certainly look useful. a()|>b()|>c() looks better than c(b(a())) but i can see the confusion because there are now two ways to pass an argument. but even then a(i,j)|>b(x,y)|>c(z) is still more readable than c(b(a(i,j),x,y),z). now i want this feature in all my favorite languages
Ya, pipes are really nice. Of course people do tend to get obsessed with them and try and make everything a pipeline. I find pipes fit the functional paradigm really well as they are clearest when they don't result in any kind of side-effects or mutation. Method chaining, on the other hand, is all about side-effects (mutating the object).
I really like pipelines but know people who try and make everything a pipeline.
Not sure why I got down oted so hard but I personally like, x.a().b().c(). Because the functions are bound to the type. When they aren't sure pipes, but that's rare when a type is well defined. Super zen for me
if course x.a().b().c() is just fine, but it only works in object oriented languages where these functions return the right object for the next step. the pipe method works with any other function as long as the output type matches the type of the first argument of the next function.
ML languages aren't really OOP similarly neither is Rust really. There's a beautiful middle ground between piles of functions and types with purpose that doesn't suffer from the swamp nor the over abstracted mess that a lot of paradigms offer. Okay I'll get off my high horse but I recommend trying one of these paradigms someday if you haven't already. Traits are wonderful.
I can't explain the downvotes other than that in OO method chaining does not bind you to a specific type. I don't really know Python but in Ruby, for example, you can do `"string".split.map { |s| s.upcase }`. IE, a method doesn't have to return a reference to `self`, so I'm not really seeing your argument other than choosing to use the style of only chaining when the object types match, which you can still do with pipes if you want.
The only ML language I have any experience with is OCaml which is statically typed and has pipes.
EDIT: re-reading your last comment it looks like I totally ignored the first part of what you said, but I guess I'm confused. This thread is also probably nested enough, lol.
Just that you can do `|> foo()` implicitly into the first argument or, via another package mentioned in the official docs, `|> foo(_)`. This isn't necessarily a bad thing, I was just trying to understand the comment way up this thread that said: "Writing them, you feel insecure of what is happening" as that makes no sense to me.
Ah, okay. The use of the second form requires a `@chain` macro annotation in the beginning of that expression, so I've never found this to be ambiguous.
Ah. I see what you mean. I think this is the issue with the language and/or tooling. Most languages will not let you import a function from two different libraries under the same name, and most languages will warn you if you use a function with the wrong arity.
> methods and traits with a strong type system reduce any ambiguity there.
Pipe is useful, of course. I personally just dislike the symbols selected for it, they're harder to type.
And no, in many cases, it's absolutely not clear what gets passed, because you may have a vector of scalars, and the function accepts... what? Is it restricted in input types at all? If it takes a scalar and then gets passed a vec of them (but you don't call it's vectorized equivalent), what happens?
Yes, I can go check the function, check if I have a vec and check if I call the function the proper (scalar or vectored) way,.. but this escalates the complexity and choices you should make, towards Rust language. And still it can run and try doing something implicitly. When I coded it, I wasn't sure I can't mess things up somewhere in the middle of the pipe.
Don't get me wrong, I liked many features of Julia. I made one project in several languages just to compare, and Julia was easy to learn and worked pretty well. The main issue that stopped me from using it was the above mentioned cold start time.
i haven't used a lanuage with this feature before so i don't know anything, but i don't understand the problem. how is the return value before the pipe being the first argument any different than normally putting it into the argument list in the first position.
or is there an implementation where a pipe takes a vector and applies it to the next function as multiple arguments?
Dataframes are vectors of vectors. And functions on them may return either vec<scalar>, or vec<vec<scalar>>, and in a chain of such operations it's easy to lose track of what you have.
1. there are no objects, so when you switch from Python, instead of chains of methods, like `my_dataframe.groupby(...).aggregate(...).groupby(...).aggregate(...)` you need either (a) a lot of poorly readable nested calls, or (b) ugly "pipes", where the left side is implicit first param of the function on the right side: ` my_df |> groupby(...) |> aggregate(...) |> groupby(...) |> aggregate(...)`. (Writing them, you feel insecure of what is happening -- in this case, what gets passed to the function.)
2. You can't make it a CLI script, because it compiles the whole code WITH dependencies every time, and as soon as you import some serious libraries, compile times will skyrocket. I quickly hit 40 seconds of compilation with just geospatial lib and dataframes. These 40 seconds turned out to be A LOT when you develop interactively. And you can't build everything in a Jupyter Notebook, because eventually you'll have to switch to a CLI script.
I think I could put up with pipes, but the compilation times were the final argument against Julia.