
Getting Past C - ingve
http://blog.ntpsec.org/2017/01/03/getting-past-c.html
======
e3b0c
Rust has some very desirable properties to me. Writing Rust programs _from
scratch_ is not as scary as I've heard of from the internet either. The
documentation is excellent, the compiler diagnostic messages are very helpful
and the notorious borrow checker didn't stand in my way that much. And I love
Cargo and Cargo.io. I have some projects where Rust is the saner choice than
Go or other GC based languages.

That said, there are actually drawbacks of Rust compared with Go, IMHO. When
facing a moderately large project __written by others __, the ergonomics for
diving into the project is not as smooth as Go. There is no good full-source
code indexer like cscope /GNU Global/Guru for symbol navigation across
multiple dependent projects. Full text searching with grep/ack does not fill
the gap well either since many symbols, with their different scopes/paths, are
allowed to have the same identifier without explicitly specifying the full
path. That makes troubleshooting/tracing a large, unfamiliar codebase quite
daunting compared with Go.

~~~
steveklabnik
RLS will hopefully full that gap.

~~~
rattray
RLS?

~~~
anuragsoni
it stands for Rust Language Server

Github project:
[https://github.com/jonathandturner/rls](https://github.com/jonathandturner/rls)

Announcement: [https://internals.rust-lang.org/t/introducing-rust-
language-...](https://internals.rust-lang.org/t/introducing-rust-language-
server-source-release/4209)

------
dreta
Can anybody make a strong case to me as to why are buffer overflows considered
an issue in C when it takes like 10 minutes to write and test an array
implementation that prevents that from ever happening? I do agree that C has
issues (though in my opinion neighter Rust nor Go address almost any of them)
i just don't understand why are buffer overflows such a huge problem in C when
the same thing is going to come up when trying to work with memory in Rust.

~~~
staticassertion
Because your 'safe' implementation will certainly have a performance cost, and
won't be the default. This is why, despite C++ providing std::array, you'll
still find buffer overflows in C++ code. C++'s std::array provides the safe
'at' function but you're opting into a performance penalty and it's not the
more familiar [] syntax.

Rust arrays/ vectors are safe-by-default. To use the unchecked, unsafe version
requires using the 'unsafe' keyword.

let v = vec![0, 1, 2]; unsafe { let x = v.get_unchecked(5); }

This means you can basically grep audit for vulnerabilities, and the above
code should be very rare.

~~~
jdmichal
In addition, the Rust compiler can also _remove_ the built-in indexing checks
if it can prove the code is safe. So, say, iterator loops over an array won't
have any index checking.

~~~
staticassertion
C/ C++ compilers are also capable of doing this, though rust's iterator syntax
tends to make it a pretty natural optimization.

~~~
Animats
Neither C nor C++ knows, at the language level, the size of an array, unless
that size is fixed. The subscript checking variants of C and C++ have to use
"fat pointers" which carry along size information. The overhead for this is
large and nobody uses that. Fat pointers used to be a feature you could turn
on in gcc, but it's somewhat abandoned now.

~~~
sirclueless
> Neither C nor C++ knows, at the language level, the size of an array, unless
> that size is fixed.

Neither does Rust.

> The subscript checking variants of C and C++ have to use "fat pointers"
> which carry along size information.

So do Rust's Slices.

> The overhead for this is large and nobody uses that.

People use std::vector all the time for this purpose in C++. It has about the
performance you'd expect, with very little overhead except where you want it
in bounds-checking.

I don't think there's actually a performance difference here. Rust's default
is safer because it requires dropping to unsafe code to do something
dangerous, but the same optimizations are available in both.

~~~
steveklabnik
One thing that I've heard might be a difference, but haven't confirmed yet:
Rust's lack of move constructors. So you have a vector, it's full, you push
one more. It has to reallocate. How do you copy all of the elements over to
the new allocation? In Rust, it's a straight memcpy of T * n bytes. But due to
move constructors in C++, IIRC they must be moved one at a time.

Again, I haven't actually dug into this; maybe someone more knowledgeable
about this can point me in the right direction here?

~~~
rustmemcpy
How would a self-referential object work in Rust in that case? The move or
copy constructor could not be a simple memcpy.

The self-reference would point to the old object. See this for an example:
[http://ideone.com/sEFtbN](http://ideone.com/sEFtbN)

~~~
Manishearth
[https://github.com/Kimundi/owning-ref-rs](https://github.com/Kimundi/owning-
ref-rs) exists, but basically you just don't do intrusive datastructures on
the stack in rust.

(Intrusive datastructures on the heap are doable with some tricks.)

They're not very essential so it works out.

------
jstewartmobile
A 62 KLOC secure NTP server seems like an ideal project for this kind of
experiment. I imagine it would be self-contained enough to actually _use_ Rust
or Golang instead of just treating them like FFI scripters.

------
awinter-py
> One such cleanup: we’ve made a strong start on banishing unions and type
> punning from the code. These are not going to translate into any language
> with the correctness properties we want.

Really? This sounds like idiomatic rust to me (heavy with enums).

~~~
Ericson2314
C unions have no discriminant. But yeah it's a pity—I'd try to hack up
discriminated unions then. Or convert to Rust unions and then enums and remove
unsafety.

~~~
awinter-py
C union declarations should correspond pretty closely to rust enum
declarations.

It's a surprisingly important feature. Every large codebase I've worked on has
clunky workarounds for storing heterogeneous types in collections.

Haven't seen a silver bullet; dynamic languages are great at this until you
have to scale (either in lines of code, number of types or dataset size) and
then they become unmanageable. Static languages force you to type a lot and
build in-memory ETL-style transformations but scale better. Using SQL is
another solution, but that separates you from the language's type-checker and
can be another source of error.

Type safety at the serialization boundary was the promise of CORBA and
protobuf but we need better tools. I hope we see more focus on ser/des types
in the next generation of industry languages.

~~~
steveklabnik
Close, but not quite: unless the type is NonZero, you need space for the tag.

c-style unions are in nightly behind a flag; they're not stable yet.

~~~
Manishearth
I think they meant that the use case corresponds closely, not the
representation. Generally C unions get used the same place Rust unions would
in a rust program (not vice versa though), unless they're being used for type
punning, which is pretty rare anyway.

~~~
awinter-py
I meant if you have a large program where many types are C unions like:

    
    
        struct TaggedUnion {
            enum {T1, T2} type_id;
            union {type1 t1; type2 t2;} u;
        }
    

If the goal is to translate automatically to a rust enum declaration, that
will be (a) possible to do with an automatic tool and (b) will give enhanced
safety because it will force the type to be checked everywhere these values
are used.

------
aaron-lebo
Outside of the language war bubble it's really great to see a post like this.
Practical concerns, reasonable advantages/disadvantages of each language, a
real project dealing with real timelines. Thanks!

~~~
jstimpfle
Was going to say the same. In the past ESR has come across as a patently
arrogant gun maniac, but the first part of this post is great for all the
reasons you mentioned.

(What irritated me though was the switch to first-person narrative at the
end).

------
tptacek
This is literally the only thing I can think of that "NTPsec" can do that
would result in the project having any relevance. I understand why some very
specific sites are chained to the ntpd codebase, but the vast, overwhelming
majority of the ntpd deployed base not only isn't tied to ntpd, but also
doesn't need 99% of what ntpd does. Trying to "secure" that codebase always
seemed to me like a very silly windmill to tilt at.

------
giancarlostoro
I wish more mention of D would happen. It is compatible with C and C++
libraries and features GC without sacrificing the good things of C and C++. I
always loved the idea of Rust and Go but they are nowhere near C or C++ where
it matters to me. D fits the bill, otherwise I just use Python. I like being
able to design software in my own way as opposed to being told how to do it.

~~~
moosingin3space
The truth is the Rust leadership did the hard work of building a community
around the language, making high-quality tutorials and introductions to the
language all before it stabilized for 1.0. Many developers claim to dislike
"marketing", but Rust did its marketing/evangelism and D didn't.

From a purely technical standpoint, Rust's borrow checker is able to catch
data races, while D has no such functionality (unless I'm not caught up),
which is a huge advantage in today's world of multithreaded applications.

~~~
Manishearth
Even in single-threaded mode D isn't too safe. Safer than C, but unless you're
using the GC it is still unsafe.

------
falcolas
How much concern would non-standard architecture support matter for ntp? Given
how many architectures Linux supports, I would think that C would still be the
best choice, until these other languages gain support for those missing
architectures.

Or perhaps it's a good opportunity for a language which offers transpilation
with ANSI C as the target?

~~~
jerf
In addition to squiguy7's point, I'd add that A: it may not be clear if this
is your only contact with ntpsec.org [1], but this is actually a fork of the
classic ntp, so they may be more willing to abandon some older architectures
to produce a more secure product going forward than the core NTP project would
and B: the older versions will still be around even so.

Also, it has been suggested by some experiences that older architectures often
end up getting supported past when they really should be stopped out of sheer
inertia, though I'm not having luck digging up the articles that prompt me to
say this. Are there that many systems out there running ntp that can't run Go
and/or Rust, and if there are, are there enough to be worth bending the
project around? If the people running those things care, perhaps they should
fork the project and maintain it themselves. Which is less harsh than it may
sound, because they can still pull from upstream, and they probably just need
to tread water rather than stay up with the latest & greatest.

(I should emphasize that the operative question is _are there enough to be
worth bending the project around_ , rather than whether there are any. Because
there certainly are non-zero numbers of systems running ntpd that can't run
Rust or Go. But who's going to pay to maintain them? Especially if classic ntp
is still available?)

[1]: [https://www.ntpsec.org/](https://www.ntpsec.org/)

~~~
falcolas
Well, it's not just older architectures which are not currently supported,
it's architectures used in IOT devices, mainframes, and other non-commodity
hardware. Specific to LLVM based languages, if it's not a priority to Apple
(or the other big LLVM players), it infrequently gets done.

Specific to IOT devices, I would personally love to see secure _everyting_.
NTP, TLS, SSH, etc.

~~~
eridius
There is a lot of community interest in Rust with regards to running on
embedded devices, so I wouldn't be so quick to point to IOT devices as a
reason for not using Rust. That said, I don't know what the current state of
support for these less popular architectures is. The official page for
platform support is [https://forge.rust-lang.org/platform-
support.html](https://forge.rust-lang.org/platform-support.html).

~~~
mitchty
LLVM's biggest weakness right now is micro controller support like PIC/AVR.
But its getting better: [http://lists.llvm.org/pipermail/llvm-
dev/2016-November/10717...](http://lists.llvm.org/pipermail/llvm-
dev/2016-November/107177.html)

That should hit most of the chips most people would care about. I presume rust
would gain from the upstream LLVM support?

~~~
eridius
Yeah it should. AIUI Rust uses a fork of LLVM, but they should be regularly
pulling in upstream changes (but I don't know how often this happens) as well
as submitting their own changes back upstream.

~~~
steveklabnik
We do, in both directions.

~~~
moosingin3space
Is there a technical or political issue preventing Rust from using upstream
LLVM? Either way, is there a possible resolution on the horizon?

~~~
steveklabnik
Nope. You can use a stock LLVM if you want. Some tests might fail, since
patches might not be in upstream yet.

It's not really something to resolve; it's just the usual situation with a big
dependency.

------
w8rbt
I would start by getting the code to compile with g++, then begin migrating
the dangerous C constructs to safe C++ constructs. IMO, that would be a safe,
reasonable thing to do.

~~~
pjmlp
How would you prevent other developers to write C style code?

~~~
blub
Project leadership, coding standards, _automated analysis and code reviews_.

But realistically speaking it might be easier to change tools for a
significant number of teams, because the software development community is in
general poor at leadership and process. A rewrite seems more approachable for
the average (not necessarily average skills-wise) dev compared to a change in
attitude, self-reflection and incremental improvement at a department or
company level.

Then the rust compiler will pummel everyone into submission. This can be a
succesful strategy. :)

~~~
pjmlp
Which I seldom see in enterprise projects, specially in companies whose
business is unrelated to software development and IT is nothing else than a
cost center.

Anything that is optional gets pushed aside.

------
zzzcpan
After reading this post the idea of a C-to-C translator that injects bound
checking, etc. comes to mind. Such translator could be used by OS
distributions to provide safety in the least intrusive way and possibly
completely automatically for many C codebases they have in their repositories.
Translating into Go or Rust, on the other hand, cannot scale beyond some
individual projects, that decide to undertake such efforts. Mainstream C
compilers could implement safety features too, but realistically it cannot
happen, as it's not something most people care about. So, C-to-C translator
might be a best bet with the most impact.

~~~
nickpsecurity
CompSci keeps making them, even open-sourcing some, but little uptake or
improvements from the FOSS crowd. Here's two of the top ones:

[http://sva.cs.illinois.edu/](http://sva.cs.illinois.edu/)

[https://github.com/jtcriswell/safecode-
llvm37](https://github.com/jtcriswell/safecode-llvm37)

[https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/](https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/)

~~~
duneroadrunner
As we've discussed, one issue is the big performance cost of these solutions.
Anyone interested in working on an automatic C to SaferCPlusPlus[1]
translator? It should address the performance issue[2]. And should be much
more straightforward than these C to Rust/Go translators.

[1] shameless plug:
[https://github.com/duneroadrunner/SaferCPlusPlus](https://github.com/duneroadrunner/SaferCPlusPlus)

[2] [https://github.com/duneroadrunner/SaferCPlusPlus-
BenchmarksG...](https://github.com/duneroadrunner/SaferCPlusPlus-
BenchmarksGame)

~~~
nickpsecurity
I thought you have an interesting solution to improving safety of C++ code.
So, your techniques work with C code done in the old and new C styles w/ no
work? Just fire and forget translation of C code written in arbitrary styles
to to be completely memory-safe?

~~~
duneroadrunner
Yeah, the idea is that C/C++ has a finite set of "dangerous" elements.
SaferCPlusPlus attempts to provide safe compatible substitutes for those
elements. In C, basically the only dangerous elements are pointers and arrays
(if you consider them separate things), right? SaferCPlusPlus provides safe
compatible replacements for pointers (and new/malloc and delete/free). There
is a "general" safe pointer type that can be used as a direct substitute for
native pointers in most situations, but can sometimes have a noticeable
performance cost, depending on usage. Faster safe pointers are also provided,
but cannot be used in all situations.

Replacing all the pointers and arrays in your C code with the safer
substitutes will eliminate the possibility of invalid memory access, in your
code. Of course this doesn't prevent them from occurring in any unsafe
libraries you use, including the standard library.

Also, there are certain behaviors in C that may not translate well to
SaferCPlusPlus. Like exotic pointer arithmetic. Or, for example, you could
imagine some C code that compares two pointers that point to items that have
already been deallocated to see if they were previously pointing to the same
item. Is that valid in C? Anyway, that kind of thing is not supported by the
safe pointers.

There is not yet an automatic translator from C to SaferCPlusPlus, but the
translation for most code is straightforward and direct. No new paradigms or
"Rust borrow checker" type restrictions. You can check out the benchmark code
I linked to in my comment to see examples of C++ code before and after
conversion to SaferCPlusPlus. A little code reorganization sometimes helps to
achieve optimal performance. But isn't that always the case? :) And of course,
SaferCPlusPlus requires a modern C++ compiler and has dependencies on the
standard library.

------
acjohnson55
It's not yet ready for primetime, but Scala Native ([http://scala-
native.readthedocs.io/en/latest/](http://scala-
native.readthedocs.io/en/latest/)) might just make a splash in the systems
space. I don't think it has anything like ownership yet, but I wouldn't be
surprised if it eventually develops that capability. I think you can get it to
run without GC, too, but using C Stdlib memory management. Although, that
largely defeats the memory-safety.

Just throwing it out there as something to keep an eye on!

------
jaco8
Looking at the current new and coming languages I would take a hard look at
NIM. It may be is not there yet but it looks highly appealing, is as fast as
the often mentioned Rust and compiles significantly faster. [http://nim-
lang.org/](http://nim-lang.org/)

~~~
nimmer
Nim compile times are faster and often the execution time is faster, too. Also
it's very expressive, almost like Python.

------
rafinha
I didn't understand why rust and go are natural alternatives to C. Wouldn't
C++ be a more natural option? (Despite the fact that both go and rust are
developed by third party companies)

~~~
jimbokun
"out of C into a language with no buffer overruns, and in general much
stronger security and correctness guarantees."

Doesn't sound like C++.

~~~
rafinha
I still don't get it. Can't it be done in c++ via appropriate data structures?
I mean, it's not like Go is magic, at the end I suppose Go would be doing just
that. As far as I know the main reason people opt for Go over C++ are the
compiling times...

~~~
steveklabnik
You can write safe C++, if you're careful, and everyone you work with is
careful, all the time. Judicious use of features can make your code more safe,
but they can never make it actually safe.

Rust and Go, in various different ways (and to various degrees), make it
actually safe.

------
tannhaeuser
My experience is that using C for main(argc,argv)-style programs is rarely a
problem. Trouble comes when using long running single-address space containers
for service-like abstractions with pthreads etc.; in that kind of environment,
malloc() and co. don't cut it because even _if_ you get memory allocation
right, unless using pooled memory allocators, memory fragmentation is becoming
a serious (ie. unsurmountable) problem.

It's been said over and over since at least the Java times that creating OS
processes for individual service invocations is bad for performance, but I've
never seen proof for this statement in the form of a benchmark.

Even the OpenBSD developers (who know a thing or two wrt. security of memory
allocation schemes) diss process-per-service-invocation architectures in their
httpd implementation (eg. calling their CGI bridge "slowcgi" and favouring
fcgi over it).

Isn't that inconsequential? I mean if there's a performance problem with CGI-
like process-per-service invocations, why not target these problems at the OS
level (or via pooling of network connections or whatever the bottleneck is)?

------
amadvance
Rust is surely fine, and an improvement over C, but its main advantage is that
all the rust code is written now, when everyone takes more care about
security.

It doesn't have to deal with 40 years of bad legacy code written by sloppy
developers.

You can obtain similar quality in a C modern code-base, using tools like
static and dynamic analyzers. In fact, today the hardest issues came from
multi-threading. I won't even dare to write multi-threading apps without
helgrind/TSAN.

And Rust doesn't help in this regard. From: [https://doc.rust-
lang.org/nomicon/races.html](https://doc.rust-lang.org/nomicon/races.html) 'So
it's perfectly "fine" for a Safe Rust program to get deadlocked or do
something incredibly stupid with incorrect synchronization.'

~~~
_yosefk
I find it rather daring to use raw POSIX threads or similar, hoping that
automated debugging tools operating at this abstraction level, such as
Helgrind or TSAN, will pinpoint your bugs. The problem is that the state space
of a thread-based program synchronizing with mutexes is too large to explore
at a reasonable time. More about this: [http://yosefk.com/blog/checkedthreads-
bug-free-shared-memory...](http://yosefk.com/blog/checkedthreads-bug-free-
shared-memory-parallelism.html) and [http://yosefk.com/blog/making-data-races-
manifest-themselves...](http://yosefk.com/blog/making-data-races-manifest-
themselves.html)

Which higher-level abstraction to use is a question and depends on the problem
at hand; for throughput computing in C, Cilk is perhaps the best today (I
guess you could call my own project checkedthreads a bit of a Cilk knock-off.)
AFAIK, Rust developers are more interested in higher-level abstractions than
Go developers (Go's built-in goroutines being on par with threads in terms of
ability of automated debugging tools to flag bugs; perhaps there's work in Go
on higher-level abstractions but the vibe I felt, perhaps mistakenly, is that
these are non-problems, which I disagree with:
[http://yosefk.com/blog/parallelism-and-concurrency-need-
diff...](http://yosefk.com/blog/parallelism-and-concurrency-need-different-
tools.html))

------
ericfrederich
I like go, I'd love to write libraries in it but as far as I can tell you
can't really create a C compatible shared library from it. That it still the
common denominator if you want to call into it from other languages. I'd love
to write Python programs with performance critical stuff in Go

~~~
LukeShu

        as far as I can tell you can't really create a C
        compatible shared library from [Go].
    

Sure you can! (as of Go 1.5, iirc)

Change your build command to use the c-shared build mode:

    
    
        go build -buildmode=c-shared
    

Then, to change the linkage of the functions you want to export, use the
magical comment "//export" directive:

    
    
        //export MyFunction
        func MyFunction(arg1, arg2 int, arg3 string) int64 {
            ...

~~~
steveklabnik
This still doesn't get over the inherent overhead in calling between the two
languages though, right?

(And you're bringing that whole runtime with you. Which might be okay! As
always, it depends.)

------
asdaksdhksajd
why does it take 62KLOC of C to distribute time? thats more code than the
whole plan9 kernel.

~~~
asdaksdhksajd
doom was 52KLOC of C.

~~~
chungy
Less than that even:

    
    
      all            37641 (100.00%) in 135 files
      c              37311 (99.12%) in 66 files
      asm              232 (0.62%) in 1 files
      makefile          98 (0.26%) in 2 files

------
arunmu
I do not contest on the opinion that Rust is a good language, but it slightly
hurts me when people club C and C++ together. One can easily write correct by
construction code using modern C++. Use of meta-programs allows you to create
typesafe constructs. It provides you with zero cost abstractions to specify
ownership of resources and ..... <I can go on> One has to just strive to not
use the C baggage that comes with it.

~~~
Manishearth
C++ is much better, but still not safe. Null still exists. Moves are runtime
moves, so you can still attempt to access the value at compile time. Iterator
invalidation is still an issue.

Of course, C++ might be "safe enough" for your use cases.

~~~
arunmu
> Moves are runtime moves, so you can still attempt to access the value at
> compile time.

What do you mean by that ? Are you saying the standard decided to make it non-
destructive moves ? Then yes, it is a possible scenario for error but also a
performance advantage in some cases.

~~~
Manishearth
> What do you mean by that ? Are you saying the standard decided to make it
> non-destructive moves ?

You can still access a value after it has been moved out. use-after-move is
allowed by the compiler. It places (stdlib) types in an unspecified but valid
state (I've seen C++ code reusing these types and assuming that the state
after move is something in particular -- it's not).

Most optimizations you can do by reusing a moved value could be done
automatically by the compiler (by using the same stack space since it
statically knows that it's been moved out).

Linear types are a pretty well known pattern and could have been used as a
part of the design of moves. This would have the additional benefit of
removing explicit move constructors from 99% of all types out there. This has
not been done (and can't be now).

Edit: The pretty rare optimization potential of runtime moves is pretty much
negated by the fact that common things like reallocing a vector can't be made
into a memcpy for most types, just because they have nontrivial move ctors
which wouldn't exist in the compile-time move scenario.

~~~
silent_hunter
Use-after-move is allowed because of existence of value categories, if you use
a return value from a function its creation is usually elided or it is moved,
but you can't access that temporary without directly referencing it. When you
move an existing object (by specifically saying std::move), what should happen
with it? Destroying the object is not a solution because its variable might be
still accessible in existing scope (something like dangling a reference in the
middle of a scope), which imply use-after-move should be prohibited be
language and all variables should have a "not-a-value" state and throw
exceptions on use.

~~~
Manishearth
> When you move an existing object (by specifically saying std::move), what
> should happen with it?

You don't destroy it, you treat it as an actual "move", where the compiler
will not consider the original variable as accessible after this point, and
_not allow accesses after the move_. The memory it took up is free for reuse,
and no destruction code is run on the side of the code doing the moving (in
the case of a conditional move this gets a tiny bit more complex, but not too
much). "It's variable will be still accessible in local scope" is exactly what
I'm getting at; you can enforce at compile time that this isn't the case by
simply disallowing access.

You shouldn't have local references active, just like how you shouldn't have
local references to the contents of a vector when you push to it. You can
already invalidate local references to a part of a struct when something gets
moved in modern C++. Being wary of invalidating local references is an
established concept in C++; this doesn't exacerbate that problem.

It's even better if you track scopes in references which gets rid of the
dangling reference problem entirely but at this point you've reinvented Rust
:p

To be clear, I'm talking of a completely different model that could have been
used in place of the rvalue reference and move model. Making C++ have linear
types _now_ would be tough, but it could have been done before. There are
different tradeoffs there, but I suspect it would have been safer.

> all variables should have a "not-a-value" state and throw exceptions on use.

You can make this a compile time error. Like I said, linear typing is a pretty
well established pattern.

------
kcudrevelc
> Under Linux, some SECCOMP initialization and capability dances having to do
> with dropping root and closing off privilege-escalation attacks as soon as
> possible after startup.

I was under the impression that these specific things were actually quite hard
to do in Go. I believe that both setuid/setgid and seccomp_load change the
current OS thread (only), and since Go multiplexes across multiple threads and
gives programmers very little control over which ones are used for what
goroutines, I'm not sure how you would, for example, apply a seccomp context
across all threads in a Go program. setuid/setgid are currently unsupported
for this reason, with the best method being "start a subprocess and pass it
file descriptors"
([https://github.com/golang/go/issues/1435](https://github.com/golang/go/issues/1435)).

I'd be interested to hear if others have found ways to actually do this
reliably for all OS threads underlying a running Go process.

~~~
plietar
I did this once, by writing a small C program which sets up the seccomp
context before exec'ing the Go binary. Unfortunately Go's runtime makes a huge
number of system calls in the background, and the whitelist kept growing.

Switched to Rust and there was only had one hidden system call left, getrandom
used to initialize the hashmap

------
eggy
I didn't see Ada or ATS mentioned upon a quick read here. I don't program in
either beyond exploratory play, but they seem to fit the need here no?

------
elchief
what's wrong with OpenNTPD exactly?

~~~
NelsonMinar
There's a summary of some of its limitations here:
[https://en.wikipedia.org/wiki/OpenNTPD#Criticism](https://en.wikipedia.org/wiki/OpenNTPD#Criticism)

One of them (lack of leap second support) is fresh on our minds since we just
had a leap second. Some thoughts from the OpenNTPD folks are at
[http://undeadly.org/cgi?action=article&sid=20150628132834](http://undeadly.org/cgi?action=article&sid=20150628132834)

Chrony is another alternate NTP implementation, here's their comparison vs
ntpd and openntpd.
[https://chrony.tuxfamily.org/comparison.html](https://chrony.tuxfamily.org/comparison.html)

~~~
elchief
It sounds like the leap-second thing isn't that big of an issue though...

~~~
NelsonMinar
The leap second thing is a big issue for a server. OpenNTPD serves the wrong
time on the day of the leap second. They don't propagate the leap second flag
properly either. It's not correct and doesn't belong in a public server pool
IMHO. This issue isn't just theoretical: it was responsible for a lot of the
bad time being served by the NTP Pool last week.

You're right that the article I linked argues the leap second isn't a big
problem to client machines. I guess I sort of believe it, if you don't care if
your client is a second off from true time once every couple of years.

------
z3t4
Why do they have to use a low level language ? Why not use NodeJS ? One
argument would be time management, but even with C you can not be sure that no
interrupts happens in between ...

------
pklausler
Honestly, why not do it once and for all in a strongly typed pure functional
language, validate it, and then tweak the GC parameters to get the
performance? Use the safest and most powerful language that you can, if you
can.

~~~
jstimpfle
Some people don't believe in tweaking GC's. I've heard horror stories from
people who tried it on the JVM.

(And I have made terrible experiences with JVM's GC, but never tried tuning).

Some people don't believe that extremist functional languages are needed
(depending on the domain).

Validation. Oh, well.

I think you propose Haskell, and it might disqualify for an NTP daemon, for
example in terms of debuggability (stack traces?).

~~~
nathyong
A more relevant point that would disqualify Haskell is the fact that its
runtime (or rather, the GHC's runtime) is nowhere near predictable enough for
the realtime guarantees that time synchronisation software would require.

A Haskell DSL that outputs some safer low-level code would be a more likely
choice (for example
[http://hackage.haskell.org/package/atom](http://hackage.haskell.org/package/atom)),
but Rust is both more popular and has more commercial support.

~~~
jstimpfle
When opting for a DSL that compiles to low-level the "Haskell" part is less
important.

I've heard the idea of DSLs over and over again, but who actually does that? I
know of course of sed, awk, regex, etc but what part of NTPsec is narrow
enough in scope and large enough in volume to justify creating a DSL? (just
asking -- I'm not familiar with NTPsec).

~~~
nickpsecurity
Companies that like consistently getting results. Example:

[http://ivorylang.org/ivory-introduction.html](http://ivorylang.org/ivory-
introduction.html)

------
dhsjxhx
Rather than looking for a new language, why not ban programs from writing to
executable memory?

~~~
bvinc
I'm not an expert in this field, but it's my understanding that this is often
the case for C programs executing nowadays. The AMD64 architecture even has
the NX bit. There are some other ways that this can be enabled too.

But surprisingly, this doesn't solve all code execution security problems from
buffer overflows. Sometimes exploiters can find non-executable memory to
change that allows them to change the program behavior to do what they want,
such as changing the command string that gets passed to a normal execve call
later in the program.

But even if you solve the security issues, buffer overflows are still a huge
headache. A one byte buffer overflow can cause your program to crash an hour
later, with almost no hope of figuring out why it happened. A developer can
easily spend weeks tracking down a single buffer overflow crash.

~~~
nullc
> Sometimes exploiters can find non-executable memory to change that allows
> them to change the program behavior to do what they want, such as changing
> the command string that gets passed to a normal execve call later in the
> program.

Changing function pointers or the return value on the stack are the normal
things to do. In particular if you corrupt the stack you can simply return
into system() with whatever arguments you want.

------
nurettin
Why does 90% of this _huge_ comments section revolve around Rust? Haven't we
had enough of the same already? Yes, we know it provides memory safety
guarantees. Yes we know you hate everything written in C with great passion.
This has been made apparent and banged on our heads for the past several dog-
years. Did anyone bother to look at the sheer amount of refactoring the author
& team did? Did anyone realize how difficult this is and what they might learn
from this tarrasquesque ordeal they went through? Oh no, apparently it is C
and logic mandates that Rust supercedes it so why.the.hell.bother.

~~~
eridius
The huge discussion about Rust is because that's literally what this post is
about. It's titled "Getting Past C" and most of the post is about considering
Go or Rust as the future language for the project. And, not surprisingly, a
lot of people think Rust is a great choice for this (and I agree).

~~~
nurettin
The title is Rust-bait so let's all chant rust! Rust! RUST! Oh but there are
only three tentative mentions of Rust in it and this entire comment section is
corroded. Just compare the two pages in a browser with ctrl-f rust

~~~
kbenson
I'm not sure where you're under the impression that the number of times
something is _explicitly_ mentioned corresponds directly to the number if
times it's being discussed, which is what is implied by your suggestions to
search for occurrences of the name. The whole article is about future plans
for NTPsec, and the work going into making it so it can be converted to a new
language later. The article is about the relative merits of different
languages for this task, and Rust is one of the languages. Of course Rust is
going to be talked about in the comments. The only odd thing here is that Go
isn't talked about more in the comments, not that Rust is talked about a lot.

~~~
nurettin
For me, this is the gist of the article:

>> But NTPsec is a lot smaller and cleaner now at 62KLOC of C (that’s just 27%
of the original size). It’s been brought up to pretty tight C99/ANSI standards
conformance, and the few remaining platform dependencies are either already
well isolated or can easily be made so.

Then they have a section about future plans and a short comparison of two
possible languages. I'm surprised that you're surprised Go is virtually non-
existent in this thread.

Hackernews comment sections are strictly for endlessly typing about Rust's
memory safety features to seemingly no end. You will find this behavior under
any article that mentions C or C++.

At this point I don't care about downvotes because the whole finding-valuable-
comments-to-learn-from experience I've had in the past is getting harder and
harder to attain to. So instead of bitching about that, I will just go upvote
someone who is downvoted.

~~~
eridius
Maybe you should learn from the fact that so many people are talking about
Rust.

~~~
nurettin
I did. See above comments.

------
anthk
>Under Linux, some SECCOMP initialization and capability dances having to do
with dropping root and closing off privilege-escalation attacks as soon as
possible after startup.

Thanks, Eric. Now, FFS, learn how the Cathedral model WORKS, get OpenBSD,
learn pledge(2) and stop spreading old FUD against C.

------
lacampbell
> One of the medium-term possibilities we’re seriously considering for NTPsec
> is moving the entire codebase out of C into a language with no buffer
> overruns

For a small one time fee of 1000 USD I can copy paste you 50-100 lines of C
that provides an implementation of an array without buffer oveflows. Cheaper
than switching to rust or you know, learning C properly.

~~~
noselasd
Can you pass this buffer to e.g. read() and it will still guarantee no buffer
overflow even if the programmer mis-calculate the size argument to read() ?

~~~
civility
I'll bet he can give you an array_read() function that does what you ask. I
know I could.

~~~
lacampbell
The fact that it doesn't occur to people to build safe abstractions in C to
deal with things like buffer overflows still shocks me. C is fairly low level
by todays standards. You 100% have to build abstractions using the standard
library and then use those abstractions, rather than just using standard
library functions everywhere.

This isn't an argument against safer languages, or higher level languages, or
languages with built in bounds checking or anything else. This is more an
argument against programming in a "close to the metal" language like C so
thoughtlessly that buffer overflows are actually a serious issue.

Maybe you just need a pedantic mind. When first learning C, as soon as I
figured out that writing "char buffer[BUFFER_LEN];" could cause your program
to crash, I immediately set out to write a safe dynamic array implementation I
could push chars onto it and I didn't have to worry. And I am by no means an
expert C programmer.

~~~
munin
Is this abstraction zero-cost? What's the overhead?

Can you give me a similar set of primitives to manipulate memory in a
temporally safe manner as well? What is the cost of that abstraction? How does
it compare to a runtime's GC?

~~~
lacampbell
I never made the argument that you shouldn't use a safe GC'd language. I
merely made the argument (now flagged for whatever reason) that buffer
overflows are an easily solvable issue in C, and if you are having issues with
them you really need up your game and learn to create abstractions.

As for the _cost_ of the abstraction of bound checked arrays of arbitrary
length, I can't imagine it being any slower than rust. I could be wrong. If
you'd care to provide an example program in whatever language you are
advocating, I can give you an implementation in C using a bounds checked array
ADT, and we can compare notes.

~~~
jdmichal
Obviously, if you have to check an index length you're going to be doing a
branch. However, because the index checks are intrinsic to the Rust compiler,
it can _remove_ them when it proves the code is safe. So, for instance, an
iteration loop over an array won't have any index checks in the generated
machine code.

~~~
dbaupp
There's two problems with this statement:

\- index checks aren't intrinsic for all cases where it is done, e.g.
[https://github.com/rust-
lang/rust/blob/8f62c2920077eb5cb8132...](https://github.com/rust-
lang/rust/blob/8f62c2920077eb5cb81323142fc5dbe6ae8813c0/src/libcore/slice.rs#L610-L618)

\- being intrinsic is absolutely not required for the checks to be removed.
Bounds checks are just a normal branch with a fairly straight-forward
condition, and the always-true nature of those conditions is inferred in the
same way for both the built-in indexing and non-built-in indexing. This is
true in languages other than Rust.

An iterator is designed to just not do any indexing at all (neither using the
built-in [] operator or one of the functions that implements manual bounds
checks), because it instead just manually (unsafely) walks a pointer along the
array.

~~~
jdmichal
> An iterator is designed to just not do any indexing at all (neither using
> the built-in [] operator or one of the functions that implements manual
> bounds checks), because it instead just manually (unsafely) walks a pointer
> along the array.

It obviously still has to bound-check that it's not about to walk right out of
the array. If you want to call this something other than a "bound-check", I
think that's being overly pedantic.

~~~
dbaupp
Yes, it checks that it's reached the end of the array as part of the loop, in
the same place that `for (int i = 0; i < n; i++)` checks whether it's reached
the end of the loop. I think this is different to the indexing bounds checks
we've been discussing since the whole process is more controlled (compare and
increment a pointer), rather than taking an arbitrary integer.

But yes, strictly speaking you're right, an iterator does do check when it's
reached the end of the array, it just doesn't do any indexing nor does it use
the indexing checks built into the compiler (which is what you implied the
iterators benefit from).

