Show HN: Jackal – XMPP server in Go

[ortuman]

https://github.com/ortuman/jackal/releases/tag/0.2.0

[orivej]

The principal feature that I consider when evaluating XMPP servers and clients is their support for reliable message delivery (no message loss on connection interruptions), and the main method of achieving it is with XEP-0198 ("Stream Management") [1]. Unfortunately Jackal does not support it, and AFAIK it is much harder to implement it for an existing server or client than to design the application with XEP-0198 in mind.

To support reliable message delivery as defined in XEP-0198, the application needs to:

(1) count and acknowledge stanzas;

(2) buffer unacknowledged stanzas for redelivery in case of stream resumption;

(3) resend delayed stanzas in case of stream not being resumed.

Note that some clients that claim support for XEP-0198 implement `urn:xmpp:sm:3` (Psi+, Conversations, Gajim with python-nbxmpp>=0.5.6), while others implement `urn:xmpp:sm:2` (Swift, Gajim with python-nbxmpp<0.5.6). The revisions differ in how they count stanzas. Ejabberd supports them both simultaneously.

The level of support also varies. An application may:

(1) just count and acknowledge stanzas;

(2) resend stanzas on stream resumption;

(3) resend stanzas on reconnect without stream resumption;

(4) resend stanzas after the application is restared.

Psi+ supports 1 and 2, does not support 4, and probably does not support 3. Swift supports only 1. [2]

[1] It is also technically possible for the client to use XEP-0313 ("Message Archive Management") to retrieve undelivered messages and to discover and resend unsent messages, but the latter is not described in the XEP, and I'm not aware of the clients that do so. I have found this suggested by Holger Weiß at https://github.com/redsolution/xabber-android/issues/128#iss... , together with a clever (currently abandoned) extension of XEP-0313 with `urn:xmpp:mam:sub` that subsumes XEP-0280 ("Message Carbons").

[2] To study interactions between Jabber clients and servers, I wrote https://github.com/orivej/xmpp-logging-proxy that serves as a Jabber server by relaying communications to the actual Jabber server while saving the decrypted streams to files.

[defanor → orivej]

I agree: not to discourage OP, but there is plenty of XMPP implementations (as well as those for other protocols), yet very few complete (say, in the XEP-0387 sense) ones, and even fewer reusable (libraries providing C interface, or something else more or less common and standard) ones. It seems that it would be more useful to put time/effort into improving those, unless there are really good reasons for making new ones.

[sofaofthedamned]

I've just checked the Docker image and it's 417mb. How on earth is it this large?

https://hub.docker.com/r/ortuman/jackal/tags/

[eximius → sofaofthedamned]

Why is it even using Docker at all? One of the features of Go is that it is statically linked with it's dependencies!?

[kiliankoe → eximius]

Docker has a few more upsides than just bundling an app with its dependencies, orchestration being one of them. Also, a go app wouldn't have to be cross-compiled for whatever architecture as long as Docker runs there. Go makes that easy and effortless, sure, and there's nothing stopping you from doing so.

[chatmasta → kiliankoe]

Docker containers inherit the architecture of the host system. There is no CPU virtualization involved. So you still need to cross compile if you want to run the binary on another architecture, regardless of whether or not the binary is running inside a docker container.

[mike-cardwell → sofaofthedamned]

I just downloaded the image it's based from "golang:1.10" according to the Dockerfile - https://github.com/ortuman/jackal/blob/master/Dockerfile

  mike@ung:~$ sudo docker image ls|grep golang
  golang              1.10                6b369f7eed80        3 days ago          794MB
  mike@ung:~$ 

794MB...

Should probably base it off an Alpine image instead. Would make it many times smaller.

[edit] In fact, there is a "golang:1.10-alpine" Docker image:

  mike@ung:~$ sudo docker image ls|grep alpine
  golang              1.10-alpine         05fe62871090        4 days ago          376MB
  mike@ung:~$ 

Hmm. even that one is bigger than I expected.

[unfunco → mike-cardwell]

  FROM scratch   
  ADD main /  
  CMD ["/main"]

[r_venkat → sofaofthedamned]

The image seems to have the code and dependencies copied into it+ `go build` run on it, instead of just copying the built artifact.

[ihsw2 → r_venkat]

This is an opportunity for enterprising OSS contributors -- by virtue of being written in Go, converting their current Dockerfile to one that uses multi-stage builds should be simple and the resulting image would be as minimal as they come.

[SEJeff → ihsw2]

Example golang multi-stage build:

https://github.com/jaegertracing/jaeger/blob/master/cmd/quer...

[Boulth → SEJeff]

With the caveat that it doesn't build the app inside container, so the multi stage build is kind of underutilized here.

[e12e → Boulth]

You almost certainly want to build the app inside a container, but probably not the same container that result from the build.

[ed: well maybe not for a simple go project, as go has great cross compilation support. But if there's a bit of Javascript/(s)css/static resource munging - a container with a predictable build environment might become appealing. ]

[Boulth → e12e]

Isn't this the purpose of multi stage builds? To have one stage for building it and another, from scratch, to run it?

[SEJeff → Boulth]

It is one of the purposes, yes. The idea being that you don't need build artifacts in containers which run the application.

[true_religion → sofaofthedamned]

It probably bakes in the dependencies.

I do that on CI as well, since building dependencies often takes longer than just downloading a full VM.

[Boulth]

This is really interesting. As far as I know there are only two servers that power the majority of public instances and both of them have weak type systems.

[ktosobcy → Boulth]

There is Tigase (http://tigase.net) written in Java with public instances tigase.im/sure.im

[kiliankoe → Boulth]

I assume you're referring to ejabberd (erlang) and prosody (lua)?

[Boulth → kiliankoe]

Yes.

[asabil → Boulth]

Since when Erlang is weakly typed? It is dynamic strongly typed.

[jerf → asabil]

The type of strong typing that Erlang has is not a useful definition of the term, as the only weakly-typed language in current use would be assembler. By any other useful definition of the term, Erlang is weakly typed. Erlang + Dialyser can be read as incrementally typed, if you follow my idiosyncratic view of considering the entire programming environment in such decisions rather than just the language itself, but I don't think that's a majority viewpoint, and still doesn't yield a strongly typed language... just an incrementally typed one.

[hauleth → jerf]

As far as I know there is no way for Erlang to automatically cast between types (except numeric, but these are treated as one type with variants), so by any definition that I know it is strongly typed language. Dynamically typed, yes, but strongly.

About other languages that are weakly typed in "current use":

- JavaScript is still weakly typed as far as I am concerned, nothing prevents you from 0 == "" and get true value. - PHP is still weakly typed, with optional type hinting but still sometimes can bite you in the ass

Erlang + Dialyser can be read as quasi-static typing, but it is still strong one.

[funkaster → Boulth]

What about openfire[1]?, written in java.

[1]: https://www.igniterealtime.org/projects/openfire/index.jsp

[Boulth → funkaster]

I know that it exists but have never seen it in the wild. It's probably used by closed enterprises. If I was in charge there I'd use ejabberd, and Prosody if having less than 50 users.

[sam_goody → Boulth]

We use Openfire, after running eJabberd for a while and having too many issues (I actually learnt some Erlang just to fix one of the bugs).

We have two uses. 1. Web based per-page chat for all our logged in users, then using Candy+ejabberd/openfire, but now using a custom node socket based chat server. 2. Internal chat still on Openfire. Have thought of checking out Zulip/Mattermost et al.

Never tried Prosody, and feel that xmpp never got the client thing quite right.

[lucian1900 → Boulth]

Go's is not much stronger either. It hardly makes any difference.

[Thaxll → lucian1900]

You don't known what you're talking about, did you even bother search interface {} in the code?

[erik_seaberg → Thaxll]

Here are methods that store and retrieve … something, probably. The author had to make up method names because map[string]interface{} doesn't have an interface we could satisfy.

  func (ctx *Context) SetObject(object interface{}, key string) { … }
  func (ctx *Context) Object(key string) interface{} { … }

Here a caller has to provide a method that takes an arbitrary number of … whatever.

  type rowScanner interface {
  	Scan(...interface{}) error
  }

[icholy → lucian1900]

Keep telling yourself that

[JepZ]

Nice to see someone actually starting a Go powered XMPP server. I hope their breath is long enough to bring it to a competitive state.

[0xdeadbeefbabe]

It's good to see something written in go.