Hacker News new | comments | ask | show | jobs | submit login
Show HN: Java Bullshifier (takipi.github.io)
199 points by tkfx on Nov 22, 2016 | hide | past | web | favorite | 80 comments

Hi, Alex from OverOps here. We've created the "Bullshifier" internally to stress test edge cases with our own monitoring tool (http://www.overops.com).

Whenever an exception or a logged error / warning happens in production, it shows you the complete source code leading to it, and the full variable state across the entire call stack.

Using Java Bullshifier, we're able to generate random applications with call stacks that can go thousands of frames deep and other quirky behaviors. This helps us make sure the overhead is unnoticed, even in the most extreme cases.

Would be happy to answer any questions.

I don't know why, but the fact that this is an actual tool that's been designed to solve a real problem that you guys faced (as opposed to being a random project that somebody dreamed up to make fun of Java or to see if they could do it) makes me really happy.

Java is alive and kicking. It's the most popular programming language in the world. As such, it also attracts most of the fire. We hope to see more colorful projects like these go open source. Thanks for the comment! :)

I've been wanting to build a generic version of this type of tool for quite some time.

When I'm waiting several minutes for our app server to startup, I often ponder what life would be like if we had chosen language X or another stack for our application.

I've primarily worked on JRE-based stacks for some time and once a project reaches a certain size developer happiness begins to decline due to large compilation times, app startup times, etc. Even incremental compiling or hot-reloading of classes seems unreliable at a certain app size (either due to memory or some other reason).

It would be neat if there were a tool to generate large and complex apps with various stacks to get a feeling what the developer experience would be like 1-2 years down the road once the app reaches a critical mass.

That is a good idea for testing out new stacks for that shiny new rewrite. One thing I've realized is most projects are not designed for 100+ developers working on it at +400kloc of code.

We adopted https://buckbuild.com/ at work for our android builds, and it has sped it up a lot compared to our previous gradle build process.

Why not OSGi?

Why not micro-services?

How were you able to go back in time to create Spring?

You misspelled "Hibernate". Hope this helps.

Personally I go for the Jboss Spring Hibernate stack with a touch of Oracle thrown in running on Windows Servers.

We have a self-hosted Jira/Confluence server running on Windows that's reached end of life....I can relate to this. Fortunately no Oracle in the mix other than Java.

joda-time + java.util.delorean

I did once work on a project that had a class FluxCapacitor. It was used in testing to set the time. It worked, but had about as many zany unexpected consequences as its fictional namesake.

You should be using the Java 8 time APIs if you want to do it right.

Personally, I find Java 11 makes this much easier.

How did you design this ? Do you have an interest in porting this to other languages ?

Hey, at the moment we're only interested in JVM languages, so Java was the best fit for this project. We'll be adding .NET support for OverOps in the future so that might be something we'll be looking at for Bullshifier as well.

There are a bit more details about the design on our blog: http://blog.takipi.com/java-bullshifier-generate-massive-ran...

And the full source code is available on Github: https://github.com/takipi/java-bullshifier

Basically, the tool is built with Groovy (many projects use it as a scripting language for Java), and there are 4 sections to each randomly generated class:

1. Variable definition, with random types, random names, and values.

2. A section that throws exceptions, or prints to the log, according to the config settings for the rate of events. This also includes a “suicide” function that stops the execution of the program.

3. Calls the next method.

4. Random code that never runs.

There are some advanced config settings described in the Github README.md

At first I thought this another one of the several Java shaming projects that have been posted in the past on HN (usually in regards to the ridiculous class names of some projects ... aka Spring).

But this project actually has some serious utility particularly for byte code manipulators are just alternative compilers like AspectJ. Particularly AspectJ as I have seen it fail on things that neither the Eclipse compiler (JDT) or Open JDK fail on.

I'm glad to learn that entire businesses exist thanks to the fact that Java has overly verbose exception outputs which are designed to hide the information you are seeking between 200 lines of call stack prints.

As an experienced Java developer I love long Java stacktraces and I miss them in most languages. In 90% cases stacktrace allows to find a bug without any debugging, because it's just very obvious where it is.

Of course if someone doesn't care about exceptions, it's just a bad style. Exception in the production log is like red light and alarm sound, BUG-BUG-BUG. I saw projects, which throw stacktrace after stacktrace, gigabytes of stacktraces (and then archived those teralogs and noone ever watched them). But those projects have much more problems, verbose logs is the least of them.

Yes, with heavy framework usage, especially when those frameworks generate proxy classes and wrappers and god knows what, it's often 2-3 lines of useful information between 90 lines of not-so-useful library or even autogenerated code. But only developer can judge it, environment should preserve anything.

The problem has historically been that most J2EE frameworks overly rely on delegation, because of GoF.

I don't like seeing the same 5 stack frames in the same class appear three separate times in a single stack trace. Because what does that stack trace say?

It says it took Foo five function calls -on itself- to figure out it's not responsible for an operation. So Foo asks Bar to do it, and after half a dozen more function calls someone asks Foo all over again. Who takes 5 function calls to figure out it's STILL not responsible for this action. Lather, Rinse, Repeat.

And how do you set a breakpoint in that call tree? Which time do you want to debug it? How many more times was it called before the exception was thrown?

Your concern is right, but how could we have solved it? I think it's a composition of the Reflection API, the Java style of not accepting optional function parameters, and the pluggable systems.

We call the first framework function, which calls the second function with the default values for the arguments that were optional. The second function looks up the application's method using Reflection. Because Reflection code is verbose and has a lot of tricky cases, we want to share code between all places where we call it, so it delegates calling the API in 4 different methods.

Then come up the ServletFilters in an HTTP application, which is an example of pluggable system. If we declare 10 filters, they all do something like check the authentication, gzip the response, check the XSRF token, transform a WebApplicationException into a 404/500 reaponse. Unfortunately they all appear as "ServletFilters" in the stacktrace, until they delegate to the final HTTPServlet itself. Another example of pluggable system is OSGi: Each method call is wrapped in 5 function calls if the method is in another classloader.

There's only one improvement I'm able to imagine: Java could define a pluggable system at the language level, so we don't have to implement them using function calls. This pluggable system stretches for all delegation needs, from OSGi to ServletFilters to apps where the list of delegates were defined by a REST api, etc. Sounds like a big JSR...

Barbara Liskov and Bertrand Meyer both have talked about separating decision making from execution. And though there work predates widespread adoption, I find this happens to work remarkably well for unit testing as well.

The thing to remember is that when Java was young, the pool of people who grasped OOAD was very small. Nearly all of us were making it up as we went along. So Design Patterns came out and everyone glommed onto that book, and then went out and wrote code so awful and convoluted that it's now a joke.

J2EE was built up in that space, on a foundation of bad assumptions and unfounded theories. I was simply amazed that the same company that brought us the Eight Fallacies of Distributed Computing brought us the J2EE 1.0 spec. I was so appalled at its quality that I switched to UI development until well after the 2.0 spec was out and implemented.

It says something profound that every vendor that had any succes with those versions of the spec did it by defying the spec and supporting local dispatch. A lot of people couldn't scale their app past two machines, but load balancer eventually came around and fixed that.

Anyway, my point is you have a bunch of APIs written by newly minted experts, many of the foundational ones by people with absolutely no awareness of the physics of distributed computing, it's going to take a lot to spackle to cover those sins. Some did, others doubled down, but overall every problem was solved by another layer of indirection.

When refactoring came to the fore I hoped the industry would hastily reverse course. Spring was supposed to fix a lot of this but from my view, Spring became the thing is sought to replace. Full of cryptic levers and dials and massive indirection.

I think it was Mike Feathers who attributed this to cowardice. People afraid to make decisions put all the decisions behind a layer or two of indirection so they can "cheaply" change their minds later. But again we are back to Meyer and Liskov here.

If A then B (where A is the decision and B the action) allows you to change A without adding much if anything to B, or add to B without changing A. If this shows up in an inheritance hierarchy then you need about half as many overridden methods for the same call tree. Which means a shallower call tree, and code you can speak out loud.

This, and also the fact that once you've been maintaining an application for a short period of time, you quickly learn to recognize those "framework classes that are always there" in the stack trace and skip over them quickly. They are not really a burden, and as long as you're viewing the stack trace in an editor that responds to your mouse's scroll wheel it's no big deal.

I will agree with vbezhenar that Java stack traces are extremely useful. When something has gone wrong with an application in production, it is useful to know the call stack that led to the problem. I know that I didn't just fail in the method `Foo.doFoo()`, I failed in that method which was called by `Baz.doBaz()`. I know these two pieces of code are interacting which is a big clue as to the problem.

vbezhenar also offers good advice that logs of production software should be empty of exceptions and stack traces -- that the presence of anything is the sign of a problem, and thus rare in well-maintained software. Honestly, well-operating production software shouldn't emit any logs at all. Metrics and activity records if you need them, sure, but there's no need for logs. Logs can consequently be used to quickly pinpoint unexpected things happening.

agree on mostly everything apart from last part - even well-working app should have some debug/info/warn statements. ie my current app - messaging integration tool build on apache camel, I log rather more than necessary, as messages go through various components, operations made on them etc. Each component isn't aware of others, so logging current state only if exception happens will lose some good info on overall situation happening.

a bit too verbose, but the amount of info I can get from UAT/PROD envs in case of an issue is mostly enough to fix the bug. verbosity can be solved easily by proper rolling & backup of logs, that's not rocket science.

and then archived those teralogs and noone ever watched them

I champion Sentry wherever I go, as it has an outstanding "fold these common explosions into one event" behavior, in addition to managing the _lifecycle_ of an exception/error, from first encounter through resolved in release, to the dreaded reoccurence.

I do keep a terminal open for my curiosity but I'm much happier with computers watching logs than me having to

Sentry is outstanding for this! There's an open source self hosted version available too.

That's the one I exclusively use because I'm a control freak :-)

I think the tool that we're building (http://www.overops.com) could be something that might help with the log data overload. It basically gives you everything you need to troubleshoot an error - without relying on log files, while deduping recurrences of the same event.

It gets all the required information (source, stack and variable state), through a native JVM agent that transmits everything directly to the tool.

You can accuse Java stack traces of a bunch of things but they do the exact opposite of hiding the information you are seeking.

When something blows, arent you just happy to get as much information as possible?

I remember when developing in VB classic, youd get something lile "error -68377388484". Perhaps you prefer this?

The level of verbosity is good, however when you replace almost all your catch exceptions with ex.printStacktraces() it gets enormously large and unmaintainable across a large project when too many things prints stack taces concurrently. Not the way I like to do it, however too many people do that unfortunetaly: http://plg.uwaterloo.ca/~migod/846/current/projects/09-Naksh...

That's a problem on the other side of the keyboard.

I prefer something in between. Something like: the method that called the current method, the line number and a sentence describing the exception. No need to know the function-that-called-the-function-that-called-the-fun[...]

In my experience the direct caller usually tells me very little about the root cause of the exception. The interesting context is often at least ten frames separate from the proximate cause in even moderately complex Java programs. Of course, often there's an additional couple dozen lines in the trace that aren't very useful...

That's sounds like you work on very small projects. That function-that-called-the-function-that-called-the-fun could behave very differently through that path due to calling parameters, environment or data issues.

> overly verbose exception outputs

I would love to have that problem. I work with old PHP where frequently it just barfs a fatal error with no stack-trace at all.

Java's exceptions are not overly verbose. Blame bad developers and worse architects for insane stack depths.

It's not only that. OverOps also shows the variable state and source code of every stack frame. It's like using a debugger in production.

You mean like Sentry? That's been around for ages.

Sentry doesn't give you the local variable state at the moment of error. Only environment related data. We use it as well, but only for JS frontend. It's more of an error tracker, rather than a root cause analysis tool. The benefit is that it's wider and doesn't only focus on Java, but you'll still need to spend time going through logs to actually troubleshoot those errors and understand what was wrong with the state that caused them.

With OverOps, you go deep on Java, with every variable value, overlaid on the code across the call stack. Without looking at / relying on logs.

There's a short video here (http://www.overops.com/) and you can see some more samples after you sign up.

Hey, Lewis from Sentry here. "Sentry doesn't give you local variable state" is not entirely accurate - it varies from platform to platform. We don't do it for Java (you're obviously ahead of us there), but we do with at least Python and PHP off the top of my head. I wish we could do it for JavaScript too, but it's only partially feasible and only in a very questionable way.

Note that in your browser JS use case, we capture breadcrumbs to show you console logs, user interaction events, XHRs, navigation events, prior errors, etc leading up to the error, so you should be able to easily just reproduce the error state rather than digging through logs to find out what it was. If you still find yourself needing to dig through logs to troubleshoot JS errors reported via Sentry, I'd really like to hear more about your experience to see if we can make it better.

> Sentry doesn't give you the local variable state at the moment of error.

I noticed the sentry json API at least supports them, so this may be a limitation of the specific language SDK as currently stands:

https://docs.sentry.io/clientdev/interfaces/stacktrace/ (see "vars")

(I'm fiddling around with writing a Windows C++ SDK for sentry with a C ABI...)

Java stack traces are verbose because Java devs write verbose code. Java stack traces are nice and readable. Just look at a Go stack trace, it'll make you want to vomit.

Two questions:

How do you debug your non-trivial applications: sprinkling print statements, some real time debugger, channeling your chosen god?

Why do you think that's better?

Well placed print statements. I print only a few things. Of course, in Java & eclipse, it is hard to see the result because so much stuff gets printed (The stack trace, Maven showing useless details, tomcat doing the same, etc.).

In C/C++/rust/JavaScript/Python, the last things you see on a failed build/run is usually the detail that will help you fix the problem. The information is organized to be read by someone.

You can easily control logging level with slf4j etc, you can run maven in quiet mode if you want to see only errors. I don't think these are java lang issues, its more of a people problem that they don't quite understand the language and the tools they use in depth.

I've done my fair share of work in those languages and it almost never has anything to do with the last line. Typically following back into some stupid === vs. == error or some dynamic typing issue many frames up the call stack. Even C++ is a nightmare to traceback when you use boost/STL heavily

Java's stacktraces aren't they much more verbose then Ruby, Python, or JS.

In fact, IMO they aren't verbose enough: they have only the file name of the source file, not the full path.

We have folks on-site doing just this, but thanks for the thought!

developers as a service is looking like a more realistic idea by the second!

Do you have a De-bullshifier for existing Java codebases?

Working on it as we speak ;)

If you use the right random word combinations for the class name generation, and the code actually does something, this may be near-indistinguishable from a real Java codebase.

In fact, it is a kind of common practice to generate tons of code. For C language, we have Csmith[1]. It is used to find bugs in llvm and GCC.

[1] https://embed.cs.utah.edu/csmith/

I wonder if you could use genetic algorithms on codebases like these to make them do something useful.

I wonder how much I'd have to spend on AWS if I just wrote the acceptance tests for our next project, then ran this in a loop/cluster until it wrote something that passed UAT... :)

Wonder if people will use this to post bullshified apps to Github to impress potential employers

That would actually work [1] to certain extend but once you have to prove your real skills during a technical interview it will be clear that you are not a real Java developer unless, of course, you are. If you are a Java developer this tool will (probably?) have the effect that you suggest, to increase the probability of an ATS [2] to hit your online developer persona.

I will actually try this because as people have mentioned before hiring is broken and this is either a way to exploit and/or bypass some of the inconveniences like coding challenges and the like used to filter non-programmers.

[1] https://github.com/avinassh/rockstar#testimonials

[2] https://en.wikipedia.org/wiki/Applicant_tracking_system

A world where Paula could have gotten away with it.

What does the generated code actually do when run, besides heavily exercising the JVM? Does it eventually print "Hello world" or something?

It was about time somebody creates something like this. I do have a complain, classes are named random. If you used a tool that generates human like names, it would still be random but looked much better.

I will see to create a similar rails tool. I intend to test just stress of people who will look at the code and try to understand what it does.

I was thinking exactly the same thing but to stress test "fancy IDEs".

What if I took emacs and cider and helm-cider and fed it 1e6 line long namespace browser?

(edited to note, I'm not sure why I'd want to try that, but I immediately wondered what would happen...)

for some reason when I first saw the OP I wondered if it was http://projects.haykranen.nl/java/

Heh, this might actually be useful for v2.0 :)


We've actually thought of using a random pool of Star Wars related names, will have to wait for the next version. Pull requests are welcome though :P

If you have the time (probably not :) ), there was a really cool project that took names from GitHub and generated random names using Markov Chains... maybe use the same for the class names :) .

Cool, sounds similar to the site on the other comment here: http://projects.haykranen.nl/java/

> I will see to create a similar rails tool. I intend to test just stress of people who will look at the code and try to understand what it does.

I think rails does that out of the box.

Nice, but this should be refactored to generate microservices.

Does this remind me legendary paper generator SCIgen... Good to know as a CS teacher ;-)

Neat idea! As a mostly-Java coder (by choice), I love both the approach to testing your own tool as well as the slight side blow into the Java community regarding long stack traces full of framework-contributed delegation classes and virtual proxies.

But...not more than 20k classes? That doesn't really count as "massive Java application" these days, more like an average-sized enterprise app when including all its dependencies. You would at least triple that to reach "massive" scale ;-)

Kind of off topic, but this reminds me of my internship this summer. What sucks about Spring is the horrible error messages. If there is something wrong in the XML configuration, or dependency configuration, it doesn't actually tell you that. Instead, it shows a deep stack trace with some strange exception that doesn't relate to the problem at all.

Then stop using spring. Don't get sucked into the vortex of using overhyped tools that require consultant engagements to fix production bugs.

You could also not use xml. Because spring supports java configuration and the stacktraces are easy to see.

Unfortunately, I wasn't in any position to make that call. Else, I would have ditched it.

Actually this could be a good platform for generating licensing algorithms that would be included in commercial software. This level of obfuscation would really be hard to crack by reverse engineers that produce cracks/keygens.

Features: Generates TONS OF CODE!!

I love it.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact