
Parallel-Stream - lukastyrychtr
https://blog.yoshuawuyts.com/parallel-stream/
======
yingw787
This is pretty cool :) Not sure if author is here, but I had some questions:

\- How do exceptions work in parallel streaming? How might async complicate
things?

\- What kind of logging / instrumentation is available? Can you use trace?

\- Is there a result set generated at the end like Trio nurseries, or are
results meant to be handled by another process?

I know nothing about Rust or low-level systems programming, but I should go
and learn it (and a bunch of other stuff too).

~~~
yoshuaw
Hi, author here. Glad you've enjoyed the post! To answer your questions:

As leshow pointed out, Rust uses the Result type to propagate errors. The way
we'll integrate this in parallel-stream is by adding `try_for_each`,
`try_fold`, and `try_map` methods. These methods will short-circuit when an
error is returned, and have counterparts in `std` and `async-std` already. One
tricky part is to get cancellation right, but we already have a design for
that, it's just a matter of implementing it now.

`async-std` comes with support for the `log` crate built-in, which provides
information when a new `Task` (lightweight thread) is spawned. `parallel-
stream` doesn't perform any logs on top of that, but it probably should! I'll
open an issue.

Result sets can be generated through the `collect` method which calls the
`FromParallelStream` trait. This can be implemented for various types, but
common examples include: `Vec<T>`(generic array), `HashMap`, and
`Result<Vec<T>>`. If you're interested in how this exactly works this post by
jcoglan goes into incredible detail:
[https://blog.jcoglan.com/2019/04/22/generic-returns-in-
rust](https://blog.jcoglan.com/2019/04/22/generic-returns-in-rust)

I hope this answers your questions!

~~~
yazaddaruvala
Given only the thread would die, what about panics?

For example, `.parallel_stream().map(n => 1 / n)`

What happens to the `n=0` case? Does parallel_stream propagate the panic, or
replace the value with a Result::Err?

~~~
yoshuaw
It relies on async-std's behavior, which will propagate the panic.

------
dnpp123
This looks cool. Something I really never get with async is the executor:

\- if I use this do I have do use tokio or async-std ? none ? is the code
assuming an executor ?

\- e.g can I use this library with, let's say, hyper ? tide ?

~~~
yoshuaw
This library assumes `async-std`; the way it parallelizes work is by spawning
tasks on an executor, so by design it needs to have access to one. Without
that we couldn't provide the API we do.

I haven't tested it with `hyper`, but it should soon be compatible with
`tide`(assuming `async-h1/async-std` is used as the backend).

