
Diagnosing a Linux-only unit test failure - curtis
https://codeblog.jonskeet.uk/2017/10/08/diagnosing-a-linux-only-unit-test-failure/
======
lloeki
> Tests that only fail in CI are really annoying. I’ve had this a couple of
> times, and it’s always taken hours to sort out, because the “try something,
> get results, think about what they mean” cycle is so slow.

To address this, CircleCI has an interesting feature where you can enable SSH
access to the build, so one can investigate locally (relative to the build)
instead of repeatedly throwing code over the wall and waiting for something
remotely meaningful to come back in logs.

~~~
wwwigham
Travis actually has this[1], too, if you ask their support to have the feature
beta enabled for your repo. A while back, I was having a CI only test failure
that was a massive pain to try and debug, and this helped enormously as I
could explore the build state and try alternatives without waiting on the
typical build time.

[1][https://docs.travis-ci.com/user/running-build-in-debug-
mode/](https://docs.travis-ci.com/user/running-build-in-debug-mode/)

------
dunham
I ran into a similar issue years ago. Our tests, written on macos, were
failing on linux, because pdftotext was producing different output.

I traced down the logic, added some debugging print statements, and it
magically fixed itself.

Eventually, I figured out the issue that they were comparing floats with ==
and intel processors have a few extra internal bits that disappear when
stored. Macos, which uses floats in its UI layer, defaults to adding -ffloat-
store to the compile flags to get around similar issues.

I ended up just adding -ffloat-store to our Linux build of pdftotext. (I
briefly thought about fixing the comparisons, but it turned out the code was
full of them.)

------
without_words
> My next step is definitely to try to find some history of this issue

Well, you know exactly how to reproduce the issue, so git bisect it to find
the exact commit that fixed it and how. Granted, it may take a very long time,
given the size of .Net Core.

------
alexfoo
Comparing doubles with == ?

~~~
xioxox
That's absolutely fine if they are set with explicit values or were assigned
with the same double. The main issues used to be with the old x86 80 bit wide
floating point registers which meant you could sometimes find x != x depending
on whether the number went to memory and back.

~~~
twic
There's also oddity around NaN - if x is NaN, then, on a compliant chip, x !=
x regardless of whether it's ever visited main memory.

If you find some reason to use a NaN inside one of these GeoPoints (at one of
the poles, perhaps), and the equality of them depends on the equality of
doubles, then you may give your users a bit of a surprise.

------
MPSimmons
Read this, thought it was well written and thought out, look up at the
headline... "Oh man, Jon Skeet!"

Fun problem.

------
magnat
Not directly related to the article, but string interpolation in code samples
in step 6 looks somewhat unreadable without proper syntax coloring:

    
    
      Console.WriteLine($"lhs=({lhs.x}, {lhs.y}); rhs=({rhs.x}, {rhs.y})");
      Console.WriteLine($"{lhs.x == rhs.x}, {lhs.y == rhs.y}");

~~~
masklinn
They look fine to me, but I guess it helps that both Python and Rust use the
same interpolation style.

~~~
gpm
Err... do either python or rust allow this style, in both languages I've
always used

    
    
        ("lhs=({}, {}); rhs=({}, {})", lhs.x, lhs.y, rhs.x, rhs.y)

~~~
cyphar
Python added "f strings" recently (read: in 2015)[1], which look like

    
    
        f"{some_var}"
    

which evaluates "some_var" in the local scope. Sort of like Ruby's "#{...}"
(if I'm remembering that syntax correctly). Rust supports named parameters in
format strings[2] but that's not quite the same.

[1]:
[https://www.python.org/dev/peps/pep-0498/](https://www.python.org/dev/peps/pep-0498/)
[2]: [https://doc.rust-lang.org/std/fmt/#named-parameters](https://doc.rust-
lang.org/std/fmt/#named-parameters)

~~~
vog
Note that you can do this in old version of Python, too, using the "format-
locals" idiom:

    
    
        some_var = ...
    
        "{some_var}".format(**locals())
    

Although this does have some limitations regarding variables to strictly being
in the local scope, this is a fine idiom if you need to be compatible with
older Python versions.

~~~
cyphar
That is a bit of a hack though ;). You could also do it with %-style
formatting as well:

    
    
      some_var = ...
      "%(some_var)s" % locals()
    

If you want to do both locals and globals you can do:

    
    
      some_var = ...
      "%(some_var)s" % dict(globals(), **locals())
    

New Python versions (>=3.5) have this syntax:

    
    
      "%(some_var)s" % {**globals(), **locals()}
    

But those versions also have f-strings so that's sort of moot.

