Google Play makes you jump through a bunch of hoops to ship an oversized APK, and the tools they provide are totally inadequate. (For example, see this years-old bug affecting expansion files: https://code.google.com/p/android/issues/detail?id=61344)
Raising the basic size limit to 100MB doesn't fix any of the underlying problems. In fact it may make things slightly worse for users, because some developers who would otherwise have been forced to ship variant APKs or an expansion file (more work for the developers, but easier to download for users) will now just ship a single bloated APK instead.
When you get an app update, why do you still have to download the whole thing? Why not a binary delta? For that matter, why can't the developer upload a binary delta to the store? The Play Store holds golden copies of each version of the APK, after all. Instead, we have the clunky expansion file system, and each developer has to do the work.
When shipping variant APKs to different phone sizes / architectures / etc, why does the developer have to build and upload each APK separately? Why can't the Play Store just do the "app slicing" automatically? In most cases it already has all the information it needs.
I realise that all this stuff is hard work, and my complaints really boil down to "Google should do all that hard work for me". But they have thousands of engineers working on Android and Google Play, so I don't think it's too unreasonable.
> But they have thousands of engineers working on Android and Google Play, so I don't think it's too unreasonable.
Yes, but given the quality they deliver, I seriously doubt their skills.
Before the downvote rampage starts, just check:
- Bug tracker for each Studio release
- Broken NDK toolchains
- No proper support for shader and Renderscript debugging
- How they managed to have one of the worst JIT and GC implementation than many embedded Java vendors for so many time (don't have an ART enabled device to check how it improves over Dalvik).
- How they aren't able to deliver a bug free Support Library, even though it is visible their own teams (e.g. YouTube) have workarounds for the bugs
- No roadmap on improving the Java situation
- No roadmap for NDK improvements
But hey, just watch all the goodies they deliver with Play Services.
Is there an up-to-date official doc or blog post about that? I haven't been able to find one.
It seems like in practice expansion files never use deltas, and (as far as I can tell) APKs generally don't either. I could be wrong about the APKs, or maybe my updates are always big because I'm using native code.
[Edit to add: my paranoid theory is that they pre-announced this, but then never properly shipped it. But I'd love to be proven wrong!]
[Edit edit: OK, it sounds like APKs already get deltas and it all just works invisibly. But I don't think expansion files get deltas.]
Expansion files do support a so called patch version[0] - which may sound like a delta but it isn't. The job of creating a patch and merging it with the main expansion file is placed on the app developer. Additionally there can only be one patch to each main extension file. There is not a new delta generated for each new version of the expansion file.
I'm not at all surprised if most developers just ignores the patch option all together and just re-uploads a new main expansion file every time any content has changed.
Besides, while seemingly well documented[0], IME expansion files requires quite a lot of undocumented magic. Including patching Google's own source code. Code that they recommend that you vendor into the app rather than include as a library.
Further the user experience is less than stellar. On many devices the expansion is only downloaded after the app is started the first time. So the app needs to work even while the file is missing, and at that time manually request that the system retrieves the file.
BTW, did anyone notice the fine print of this announcement? Only devices with Android 4.0 or newer supports 100 MB apk:s. If your app supports 2.3 - i.e. some 70 million Android users - you're stuck at 50 MB and will have to keep using expansion files to serve those users.
Genuine question, what better solution is there on the market at the moment?
I like the idea of binary deltas and expansions if handled well, and I imagine a service that makes it easy for developers to distribute in this way would be a goldmine.
I wish I knew! I've used Google Play, iTunes Connect (i.e. the App Store), the Amazon Appstore, and the Humble Mobile Bundle.
iTunes Connect is overall a bit nicer to work with (but often fails with completely incomprehensible errors, usually involving certificates and code signing). The new "app slicing" thing Apple are rolling out sounds very good. On the downside, it's a very closed system. Apple don't want you to script or automate anything, they want you to use their tools, of which some are slick and some are very rough.
Amazon's store is similar to Google Play but much worse overall. Their web app is very clunky. Good: no APK size limit, so no expansion files needed! Bad: big APKs need to be uploaded via SFTP(!??). No clever binary deltas or anything as far as I can tell, just massive uploads and downloads for everyone.
In the Humble Bundle, a lot of the work is done manually. That sounds bad but they've actually been a real pleasure to work with. It's great when you're doing a Humble release of an existing game; do a few code tweaks if necessary, upload to Dropbox and mail them a link + hash. I don't think they do anything fancy in their back-end, though (and they probably don't have enough engineering capacity to do so).
The MAS (Mac App Store) uses deltas for upgrades, and has for quite awhile. Sometimes a delta update will fail and you have to re-download the full app but thats rare.
What does "too little, too late" mean? Is Android now doomed?
Android updates have been incremental updates for three years now (introduced at the 2012 I/O, and immediately enabled). Your point on that is confused in any case -- why can't it do delta updates (it does...), and why can't the developer do it, but also why does the developer have to do it? You're incredibly complaining about a situation you invented.
And larger APKs are a problem because now the ridiculously easy tools that Google already provides (generating thin binaries for varied targets is extraordinarily simple. A bug regarding expansion files...what is the point of even linking that?) will somehow be unused?
Your post is a very poorly informed rant. I have no idea how it sits on the top.
What does "too little, too late" mean? Is Android now doomed?
I'm just frustrated that they're just doing tweaks, rather than any deep fixes. Obviously the platform isn't doomed, because the competitors aren't much better!
Android updates have been incremental updates for three years now.
Hmm, OK! I have seen some news articles around that, but I can't find an authoritative bit of documentation or official blog post. If you have a link handy, that'd be great.
I've observed in practice that all my test devices seem to do full downloads rather than deltas, though many of them are running older OS versions. And I'm mostly concerned about expansion files. It's possible that the APK parts use deltas at least some of the time, though again, in my spot checks that doesn't seem to be the case.
why can't the developer do it, but also why does the developer have to do it?
What I'm envisaging is, the developer builds APK version 2 locally. They send a v1 -> v2 patch to the Play Store (which already has the master copy of APK v1). Use checksums to ensure it's all working as expected, and in the worst case fall back to a full upload of v2.
Maybe from Google's point of view it isn't worth spending any engineering effort to help developers who don't have access to really fast uploads. But it's not rocket science and it sure would be nice.
A bug regarding expansion files...what is the point of even linking that?
I think it's relevant because their recommended method of shipping a really big app is to put resource data into an OBB (a virtual filesystem), but the tools for building an OBB don't actually work reliably. Worse still, Android itself can't mount an OBB reliably! So if you care about stability and/or supporting a decent range of OS versions, you can't use OBBs at all. You have to use a zipfile or something as your expansion file, and manage it yourself. Not a huge deal, but it's more work, and it all contributes that little bit more to the bloat many people complain about.
Google announced smart updates and then enabled it (they had already been using this for Nexus system updates). There is absolutely nothing any developer or user has to do, so there is no documentation on it because none is needed. If you actually monitor the connection during a smart/delta update, while the UI shows the full size of the APK it actually downloads 1/4 or less of that.
It is absolutely enabled. It is absolutely working. There is no conspiracy about this. It is one of the reasons most updates take hours to propagate.
You complained about managing thin binaries, so I don't see how the developer managing that would improve anything, not to mention that it adds difficulties to cryptographic signing and verification.
Google announced smart updates and then enabled it. There is absolutely nothing any developer or user has to do, so there is no documentation on it because none is needed.
OK, that's good news! I stand corrected.
It would still be great for expansion files to get the same treatment, or better yet to abolish expansion files and just allow big APKs.
I'm a little sceptical that absolutely no documentation or feedback is needed. Surely there are things that I as an app developer could be doing to make the deltas smaller.
You complained about managing thin binaries
To clarify, as an Android developer you can manage thin binaries yourself, as a way to reduce the download size; it's just a hassle. My complaint is really that the Play Store should do that app thinning automatically.
Updates to Xcode added what Android's tools have had for years, which is the notion of generating multiple binaries for different devices/profiles. Xcode then signs those independent sections and packages them together in an archive, while Google has you send them up separately, each signed separately.
Apple sort of presented it like the App Store is picking and choosing, but it's the enhancements to Xcode that actually enable the functionality. Google could add some tooling improvements to make it slightly easier, but really they've already done all of the hard parts.
Expansion files were a hack, and remain a hack. They are necessary for large games obviously, but really 100MB encompasses the enormous bulk of apps with ease. It would be ideal if Google simply abolished expansion files and folded the sizing into normal APKs.
> Even though you can make your app bigger, it doesn’t always mean you should.
I'm glad they included and expanded on this point. I used to be stuck on a gingerbread device that wouldn't let me install anything larger than a megabyte due to storage issues. I was always grateful to devs who kept their APK sizes in the kilobytes. Now I have a Nexus 5 and install 30MB+ APK's without hesitation because that seems to be the status quo. Does anyone know why most apps these days are so bloated in size? Is it all the frameworks and libraries we use?
It's one part. The other is graphical assets: You have to provide them in half a dozen DPI versions (vector graphics are still hit or miss), with nowadays ridiculous densities (QHD on 5"? Oh come on). That's a lot of essentially wasted data.
Agreed, it's terrible. It's amazing there is no plugin for Gradle that does it all for you, including splitting the app for different dpi and arch. Is there a way and it just isn't publicized? AFAIK Google Play does have an API..
Well, they should just be doing it for you on the back-end. In fact, all you should have to do is upload an APK with all high-res assets and google should downsample them as appropriate for the downloading device. But they don't so...
Well, your signing of the package would just be a signature to Google that it was indeed you who uploaded it. Google would then sign the new packages as themselves and the Play store could only download and install Google signed packages.
Android package security (who can send intents to who) is based off of signatures and it's not really possible or desirable to change that.
However, a build script to handle it all for you during upload would be a great way to solve it.
The API for this seems to lack the features that you would need though [1]. It doesn't appear to let you upload apps for only certain devices - unless you would specify it in the manifest of the app somehow?
Also amusing - the API says "Do not supply a request body with this method", yet obviously you'd have to supply a request body - the app itself! I have a feeling this is not a well-used API.
> Also amusing - the API says "Do not supply a request body with this method", yet obviously you'd have to supply a request body - the app itself! I have a feeling this is not a well-used API.
Or Google just doesn't give a fuck. There's various bugs with uploading expansion files that are documented from day one and still unfixed.
For apps that use native code, you typically ship a "fat binary" with ARMv5, ARMv7 and x86 binaries. Maybe even ARM64 and x64 for the adventurous. Also the set of stable native APIs on Android is pretty small, so you typically include lots of frameworks and libraries directly into your APK, yeah.
It seems like apps written in Java have less of an excuse -- no fat binary needed and the system APIs are much richer. But maybe people tend to use a lot of frameworks there too.
Also, graphics can take up a surprising amount of space in normal apps, not just games, if you ship variants for all the different screen sizes. On iOS, for example, you include a startup image for each screen size / orientation. That could be a dozen or so full-screen images! And screens are massive these days.
Modern "flat" UI images ought to compress much better, though...
Overall, I don't think it's just bloat, there is some justification. But I bet most apps could fairly easily be made significantly smaller.
For apps that use native code, you typically ship a "fat binary" with ARMv5, ARMv7 and x86 binaries. Maybe even ARM64 and x64 for the adventurous.
It's not as if one of the major use-cases is to copy APKs between devices with different architectures and have them work easily, and I'm pretty sure it should be possible to implement something that detects which one the device is before downloading, so how did this situation occur?
On iOS, for example, you include a startup image for each screen size / orientation. That could be a dozen or so full-screen images!
Unless the images are truly very different (and I don't see why they would be), isn't it possible to just include the highest resolution images and resize if necessary?
Modern "flat" UI images ought to compress much better, though...
If anything, "flat" UI means there shouldn't be a need to store much in the way of imagery - they can be procedurally generated very compactly in the code itself, or stored in some sort of vector format (even better) - but then again, I've seen people use huge solid colour bitmap images for UI elements in other (desktop) apps.
Libraries/frameworks are slightly more tricky but once again, there already is a solution: identify them as dependencies by something like a hash, and don't download if they already exist. You may end up with multiple versions of the same "module", but at least you won't get multiple copies of the exact same one.
I'm really astounded that such wasteful practices are prevalant in the mobile device world, where bandwidth and storage are far more constrained than for desktops. It's almost as if they deliberately chose the most bloated way to do everything...
> I'm pretty sure it should be possible to implement something that detects which one the device is before downloading, so how did this situation occur?
If you only have 2 architectures, and .so files in tens of kilobytes, it's just more convenient than building and uploading multiple APKs. When you have 3+ architectures, and 7MB javascript runtimes for each, it indeed feels ridiculous. Looking at you Titanium, NativeScript and now React Native.
> Unless the images are truly very different (and I don't see why they would be), isn't it possible to just include the highest resolution images and resize if necessary?
Good icons can be quite different in different sizes: in smaller sizes you remove details, adjust line thickness etc.
Another, now historic reason is that early Android devices were underpowered and loading correct size graphic vs loading-and-resizing was important for performance.
Localized strings are not compressed in a standard apk; if you have a decent number of strings and support many languages, that takes up a lot of space.
Doing the work to split packages up into different architectures helps a lot (if you use a bunch of native code), but then you have to deal with the fallout of people downloading from google play for their device, and then posting the slim binary on the web that other people with different devices are going to download and install to incompatible phones.
My company's app is a free install, and we have a fat binary to download on our website... but people still post and use the slim binaries elsewhere. Would be amusing if it were pirates though :)
For games its mostly binaries (textures, models, sounds, etc). Dunno about anything else, but 100MB is not a lot of asset space these days, and 30MB is approaching game jam sizes.
Whatever it is, iOS is also affected. I think the main cause is the addition of various screen resolutions which results in large graphics being needed in multiple variants. Then there's 32bit and 64bit code. But that is an issue that will hopefully be resolved soon.
Just curious how does large apps like Microsoft Word for Android gets packaged? The size on Play Store is 105 MB, and in App Info, it shows: Total - 152 MB, Application - 66.92 MB. Is there a way to look at APK sizes on these apps?
Most "AAA" wanna be 3D games these days they download a bunch of files.
But word isn't actually big enough on small screen mobile devices to need that.
Office 2016 is 2GB on the PC for it's full installation, if you don't install the fonts, templates, themes help and quite a bit of other nonsense you can cut it by almost half.
On small screen devices Microsoft is shipping the office suit in individual APK's so they are probably still fine within the 50/100MB limit per size considering it's a much cut down version of office.
So, how many apps can now fit on a (rumored) 16GB Google Nexus 5X taking into account that:
1. APKs will expand when installed (dalvik-cache directory)
2. APKs need to store user data and cache data (think Google Play Services, Chrome)
3. Google's “ecosystem” takes up more than 1.5GB from user-space (services, chrome, maps, google app, gmail, docs, slides, sheets, photos, drive, hangouts, street view, tts-engine, youtube, music, calendar, games, webview, play store etc)
4. User will want to store a few photos and videos on his phone and maybe have some offline music.
Several apps download a large amount of additional data to the phone after installation, typically large blobs like audio, video and textures. The limit on APKs only affected the part of the application that was shipped through the Play store.
Google Play makes you jump through a bunch of hoops to ship an oversized APK, and the tools they provide are totally inadequate. (For example, see this years-old bug affecting expansion files: https://code.google.com/p/android/issues/detail?id=61344)
Raising the basic size limit to 100MB doesn't fix any of the underlying problems. In fact it may make things slightly worse for users, because some developers who would otherwise have been forced to ship variant APKs or an expansion file (more work for the developers, but easier to download for users) will now just ship a single bloated APK instead.
When you get an app update, why do you still have to download the whole thing? Why not a binary delta? For that matter, why can't the developer upload a binary delta to the store? The Play Store holds golden copies of each version of the APK, after all. Instead, we have the clunky expansion file system, and each developer has to do the work.
When shipping variant APKs to different phone sizes / architectures / etc, why does the developer have to build and upload each APK separately? Why can't the Play Store just do the "app slicing" automatically? In most cases it already has all the information it needs.
I realise that all this stuff is hard work, and my complaints really boil down to "Google should do all that hard work for me". But they have thousands of engineers working on Android and Google Play, so I don't think it's too unreasonable.