
OpenPush: A Free, Decentralized Push Messaging Framework for Android - blendergeek
https://f-droid.org/en/2020/02/03/openpush-talk.html
======
Andrew_nenakhov
I don't think it'll fly, here is why:

Our product is a XMPP client, which can't work if it is not actively running
in the background, so we were working with making apps run in background on
Android for over 10 years, since version 2.0.

The trend is clear: with every update, we get more restrictions. Not only bg
services are killed off from memory by aggressive memory managers of crappy
phone vendors, Google themselves push developers to rely on proprietary FCM
services. The writing is on the wall, and one day background services in
Android will simply cease to work, all in the name of better battery life and
customer care. So this OpenPush service will be short lived, it'll likely be
disallowed to work in this form in Android 12 or 13.

The only proper long-term solution here is an anti-monopoly legislation, that
would make Google allow Android users to subscribe to a third-party push
notification service. Such service is, actually, a relatively simple: current
FCM is just a modified XMPP connection that a phone keeps, and it will be easy
to replicate and deploy.

Apple is guilty of this, too. A proper anti-monopoly legislations should
target them as well, and not only make them allow third-party app stores, but
third party push notifications as well.

This control of push notification services and restriction creep that occurs
on both platforms makes Google's and Apple's grip on their respective OSs much
tighter than Microsoft's grip on Windows ever was. This tyranny must be
stopped, and I don't believe it can be done without legal action, by technical
means only.

~~~
snarf21
The problem is that _every_ app thinks that they must run in the background.
Every app must have millisecond push updates for their social media network
that another Like was just received. Worse yet, most of them don't know how to
do so in a symbiotic way. It makes complete sense that OS manufacturers have
taken a stand to protect users from bad developers. They don't want the blame
and they want to keep selling devices.

It is a tough spot. How do you have open app stores and not have them filled
with malware? Android hasn't figured this out yet. Users are dumb and will
just click on anything. They need to be protected from themselves sometimes.

~~~
Andrew_nenakhov
Well, it's understandable that many apps want to run as a regular process.
Good old UNUX was and it's GNU/Linux descendants are quite OK with it. The
users have control and can choose which apps to run and which not. If some app
misbehaved, it was up to a user to identify it and remove a problem.

But Google decided that it will be our nanny and will guard all users against
misbehaving apps, whether the users asked about it, or not, and you can't
refuse this care. That is tyranny and should be fought against.

Now, I don't think that push notifications are bad. They are rather good idea
and let them be. It is the monopoly on push notifications which is bad.

> How do you have open app stores and not have them filled with malware?

Long before the 'app store' term was coined, Linux had software repositories,
which worked like magic, and without issues:

    
    
        sudo aptitude install libreoffice 
    

Boom, you now have libreoffice on your PC! If repositories had massive
problems with malware, I'm not aware of them. So it was done decades ago and
can be done again.

~~~
jaywalk
Comparing Unix/Linux running on a PC and a iOS/Android running on a smartphone
is insanity.

And comparing software repos with app stores is just as bad.

I'm not even quite sure how to address what you're saying, because it's just
so completely out of touch with how smartphones are used.

~~~
cutemonster
I like the idea of being able to connect my phone to different software repos.

Play Store would be one among many.

I think then there would soon be curated repos for well behaved apps that
don't abuse push notifications and background abilities.

~~~
driminicus
That is basically what f-droid allows you to do, add different repos. Though
google play doesn't provide a repo that fdroid, obviously.

------
me551ah
Running background services in Android is tricky. Most device manufacturers
put their own battery management software on devices, which makes it almost
impossible to run long-running services in the background. And since this
framework would either perpetually run in the background or run frequently to
check for new messages, your app is guaranteed to be throttled(force-killed)
by most devices. And being force killed means that your app cannot receive
push notifications even from FCM.

I'm curious to know if anyone is using this framework in the wild.

~~~
BubuIIC
Author here (of that post and the framework).

The solution to the background limitations for now is running as a foreground
service. This shows a persistent notification (which can be hidden by the
user) but can then run indefinitely. The other option is to integrate this at
the system level (via a custom rom, root, magisk, etc.).

> I'm curious to know if anyone is using this framework in the wild.

The client side implementation of this is still WiP, so it's not used yet in
the wild. I hope to be at a point where I can work with some apps on
integrating it there in the next few months.

~~~
smichel17
Maintainer of Red Moon, a screen filter app, here. I've struggled with similar
problems as others describe for a long time.

As far as I know, the only truly 100% reliable way to never be killed is to
run as an accessibility service. I haven't implemented this in Red Moon but
will likely add it as an option. I believe it has a large impact on bettery
life.

You can also set alarms using AlarmManager to restart the service if it has
been killed, and optionally wake up the device. This is what DNS66 does.

~~~
UncleSam
This might have changed since, but in 2017 LastPass had to work with Google to
update how they run their app. Google introduced "a new policy restricting the
use of Android Accessibility Services" [1]

I'd look into that before starting that implementation.

[1] [https://blog.lastpass.com/2017/11/lastpass-android-
accessibi...](https://blog.lastpass.com/2017/11/lastpass-android-
accessibility-services.html/)

~~~
smichel17
I have a legitimate accessibility reason for it, to help epileptic users.

[https://github.com/LibreShift/red-
moon/issues/253#issuecomme...](https://github.com/LibreShift/red-
moon/issues/253#issuecomment-569219668)

------
est31
This is a great and important project for F-Droid.

The problem is harder than it seems on the surface as idle connections can be
terminated by the network operator. There's a paper about this:
[https://ieeexplore.ieee.org/document/8214213](https://ieeexplore.ieee.org/document/8214213)

I guess ultimately one should build a database about phone networks and store
measurements of their maximum heartbeat intervals in there so that new
connections can use those heartbeat intervals to save battery.

Also ideally the app should send its data via WiFi if available. So it has to
detect when a WiFi connection is established and open a connection wia WiFi.
Ideally the mobile connection is kept alive though for the case the WiFi
proves unreliable. E.g. when I leave my home on foot I get further and further
away from the WiFi and there's a short time span where my phone keeps trying
to use the WiFi connection even though nothing gets through any more.

Detecting all these edge cases can be tricky. Fortunately the design makes the
protocol between the push server and the on-phone app open to iteration and
innovation. It's a good design!

~~~
awelkie
Excellent point. So how does Google solve the problem? Do they just coordinate
the heartbeat intervals with the carriers, or do they bypass the Internet
altogether and use some cell-network-specific push mechanism? If the latter,
is this mechanism available to everyone or just partners of the carriers?

~~~
est31
> Do they just coordinate the heartbeat intervals with the carriers, or do
> they bypass the Internet altogether and use some cell-network-specific push
> mechanism?

IIRC they use the xmpp protocol, so standard TCP over IP. In threads about
this topic you often hear claims that Google has special deals with carriers.
Whether this is true or not, no idea. I haven't seen any confirmation of this.
It would be interesting to conduct measurements whether FCM sends less
frequent heartbeat messages than the timeout is for "normal" long lived tcp
connections.

In any case, deals or not, I guess Google has built a database as I've
mentioned. But I'm only guessing here.

------
osamagirl69
I am really excited about this project! Having a source of push notifications
without routing them through the google servers will really advance the state
of open source and/or google free android. It is great to see it being backed
by f-droid and apparently funded by the German government (through prototype
fund and Federal Ministry of Education)

~~~
TrickyRick
Considering the BND's (German equivalent of NSA) love of intercepting traffic
and gathering metadata about citizens, I wouldn't put "funded by the German
government" in the Pro column.

~~~
TuringTest
It may be a case of German government's right hand not knowing what German
government's left hand is doing. An open source distributed framework can't be
coerced into routing its messages through a central server for surveillance.

~~~
ge0rg
Open Source and centralized are orthogonal to each other. Signal is both, just
to name an example.

An open push framework only makes sense (for battery optimization at least) if
it's centralized, and all applications make use of it.

~~~
ryukafalz
>An open push framework only makes sense (for battery optimization at least)
if it's centralized, and all applications make use of it.

Sure, from the perspective of a single device. But that doesn't mean all
devices have to use the same push server.

~~~
ge0rg
So as an app developer, I need to integrate each of those into my app, or
provide alternative builds for all?

~~~
lern_too_spel
No, you need to use a library that allows the user to configure the push
server, which should be configured device-wide and communicated to the app
server.

~~~
ryukafalz
Exactly. I believe this is how Web Push works; the major browser vendors each
run their own push servers, and the protocol is standardized.

------
TheAceOfHearts
I have a tangentially related tip or suggestion of which many appear to be
unaware: most (all?) cellphone service providers (at least in the US) let you
send and receive emails over text message. For example, with Project Fi you'd
use $NUMBER@msg.fi.google.com [0]. You can use this as a super simple alert or
notification system for scripting and small self-hosted apps without having to
setup any additional services or tools. However, it's important to keep in
mind that both email and text messaging are unencrypted by default, so you
should be careful with what you send.

Let's consider a situation in which this might be useful: You have a 15 to 30
minute task running, such as downloading a large file (e.g. Linux ISOs) from a
remote server or compiling a small C++ project. Instead of sitting around
waiting you could go outside for a short walk in order to get a bit of
exercise and sunlight, which helps you get to the recommended minimum of 10k
daily steps. Once the task is finished you receive a text message which
prompts you to head back inside.

If you just want to send yourself notifications then you can actually get
pretty far only using email and text messaging. However, I'm aware that
OpenPush solves a few different problems, and I love that we're finally
getting an open-source self-hosted alternative to GCM.

[0]
[https://support.google.com/fi/answer/6356597?hl=en&ref_topic...](https://support.google.com/fi/answer/6356597?hl=en&ref_topic=6216184)

~~~
est31
The protocol between the push server and the push app on the device is
entirely up to those two. You can use e-mail over SMS if you want.

------
gdeglin
This is an interesting idea. Sadly, applications that use this are probably
not eligible to be distributed through Google Play. One of the main reasons,
as I understand, is that Google wants to discourage lots of apps from each
having their own network connections open in the background.

[https://news.ycombinator.com/item?id=20145206](https://news.ycombinator.com/item?id=20145206)

~~~
JoshTriplett
There's really no good reason that a network connection open in the background
_needs_ to use extra battery, if configured carefully. If configured
incorrectly, it absolutely would, but a background connection doesn't need to
regularly wake the phone up, just as Google's own push messaging service
doesn't.

~~~
_-___________-_
More connections probably does mean more wakeups, in general, though. FCM will
coalesce notifications that happen close together in time, for example. Even
if OpenPush does that (I haven't read all the details yet), it still means two
coalesced batches of notifications rather than one if you have some apps using
FCM and some using OpenPush. The situation obviously gets even worse if every
app implements push itself.

I still think it's worth it, to be clear, and for people who want a fully de-
Googled device, hopefully eventually all their apps will use OpenPush and the
battery life situation is the same as a device where all the apps use FCM.
(Assuming OpenPush implements similar coalescing.)

------
commoner
OpenPush could be very valuable for open source Android apps once it is fully
developed. Currently, open source Android apps that do not depend on Google
Play Services are using WebSocket or Server-Sent Events (SSE) for push
notifications. For example, Signal falls back to WebSocket when Firebase Cloud
Messaging (FCM) is not available.

Tutanota published a blog post explaining how they deliver push notifications
to their Android app with SSE instead of FCM:

[https://f-droid.org/en/2018/09/03/replacing-gcm-in-
tutanota....](https://f-droid.org/en/2018/09/03/replacing-gcm-in-
tutanota.html)

OpenPush would reduce the amount of work needed to implement push
notifications compared to the made-from-scratch implementations developed by
Signal and Tutanota.

------
hjkgfdfgh
I wish something like this existed that could additionally proxy FCM
notifications. Allowing push notifications without any Google involvement is
the ideal long term goal, but one transitional path I dream about would be
forcing Google to proxy all their notifications through a host of your choice
(ideally that you control).

I think I have about 30 false start projects with names like PushMux and rough
outlines on potential ways to accomplish it.

Tangentially, I also dream of having true one-way push notifications. Send a
minimal notification to the target device via FLEX/POCSAG or even one-way
Iridium, and allow the device to connect to the internet as needed to fetch
the rest of the notification.

------
shujito
Can websockets be used as a self-hosted push notification alternative?

------
mrtksn
Wow, I didn't know that the Apps on Android do implement their own push
services in the background.

For all the edge cases out there, OpenPush is great news.

However, this is a huge no-no for me as it means that I would need to actively
manage the device to make sure that things do work as intended.

Not happy with Google "owning" the push notifications? Then maybe the solution
should be a non-profit independent corporation running it.

Having a service-per-app app cannot be as efficient as having a single service
serving them all. Both in terms of device resources and user time &
experience.

Surely having an alternative is a good thing but the issue seems to be
institutional(You don't want to have Google in charge), not technical(Google's
service works great), therefore the solution out to be institutional too. Or
something that acts like an institution, something like bl_______n.

