Hacker News new | past | comments | ask | show | jobs | submit login

Yeah... Reading this, it smacked of a possible combination of poor tool choice and over-engineering (which I've been guilty of plenty). I built a video processing/workflow application in Scala with Akka a few years ago and debugging that was hard enough, eventually it was refactored to a simpler Kotlin/Spring application... Actor systems are great for certain use cases but you can really hurt the transparency of your app if you aren't careful. I can't imagine maintaining the OP's application at scale for this use case, but maybe they have someone smarter than me!

Counterpoint: debugging erlang systems in production is a cakewalk. The tracing and introspection tools that come bundled in OTP make tracking problems down really easy. It's really hard to go back to systems that don't have erlang level visibility, so much so that it's kind of a crutch sometimes.

This is an ecosystem problem and not something inherent in a program using an actor abstraction.

Ah yes, that is a great point. Erlang/OTP were designed to be used as actor systems, whereas the actor implementations in Scala and other JVM languages/frameworks are at least one level of abstraction above that. Definitely agreed on Erlang/OTP having a wonderful set of tools for debugging/visibility, but I still stand by my assessment that OPs problems are from over-engineering (and secondarily from the ecosystem).

Definitely. In the handful of instances I've seen akka used, regular jvm abstractions would have been simpler and more coherent.

Systems in general tend to have a way that they like to be used. Erlang has all of the tools to support that model, and from my limited experience with it, works great when you respect it. I wouldn't reach for it due to my inexperience, but if I had someone like you on my team, I'd love to learn.

Java also has really great tooling.. when it is used like it wants to be. I don't have enough erlang experience to know if it is true there or not, but with Java, I've found that smart but green devs (like it sounds like Heap has) tend to reach for systems that solve a lot of their initial problems and cover up their initial ignorance, but then trade it for operational problems down the road. More experienced developers tend to build systems that are boring, explicit, take longer to get to Beta, but don't require a pager.

drob mentioned that they aren't writing flink jobs very often, which makes me think that they are probably using it for some sort of rollups / stream processing of their analytics data. If the business logic of those is complex, they'll probably have a bad time with outliers that fuck up their cardinality assumptions. If it isn't super complex, they probably didn't take the time to model their data correctly with boring java classes, and keep pushing complexity into the interactions between their actors.

Either way, they would have a lot faster and easier to maintain system if they forced themselves to pretend to be stuck with java 1.6 except where 1.8 stuff improved performance and readability (basically avoid abusing streams and reflection)

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact