I also implemented a batched client-side web tool to resize photos to a certain file size (in a few hundred lines of JS). It only supports JPEG, but it resizes photos almost instantly https://randomforest.net/resizePhoto.html
Here are a few ideas to make yours faster:
1. Use the browser's canvas element to compress JPEG or PNG images instead of WASM. The libraries which come with browsers can use processor-specific instructions, while WASM only targets the smallest common denominator of the most common architectures, so it will be slower.
2. Which resizing method are you using to find a fitting image size? For me, it worked well enough to downscale by some factor a few times until it fits, but if you want to get super close, you could use binary search.
My experience has been that using canvas to resize images produces inconsistent results from one browser to another, and in most cases is higher file size and lower quality than other tools. Enabling imageSmoothingEnabled helps, but is only supported in chromium-based browsers right now:
I'm completely new to WASM but excited to learn — how did you use the Squoosh repo to turn it into WASM, and how did you embed that into your app? Is that just a static file you can "import" and use in the client?
And can anything be "turned into" WASM (e.g. a python library), or is that kind of a new paradigm of writing code from scratch?
> Any pure Python package with a wheel available on PyPi is supported. Many packages with C extensions have also been ported for use with Pyodide. These include many general-purpose packages such as regex, PyYAML, lxml and scientific Python packages including NumPy, pandas, SciPy, Matplotlib, and scikit-learn.
Squoosh already had wasm files built, all i had to just import and create a common API.
For now wasm can be compiled from C,C++, Rust and Go, so mostly compiled languages. https://emscripten.org/ can help compile C,C++, Rust and Go has native support to compile. There is also a Typescript like language which compile to wasm(assembly script)
Python or most dynamic languages are too difficult to compile to wasm.
Oh heavens, this is really neat! Thank you! I love Squoosh but I often have a multiple photos to compress to WEBP.
On a similar note, does anyone know similar lightweight web/desktop apps for basic photo editing, specifically, resize, conversion, crop, color overlay, etc. I use Photopea[0] on Linux but I still feel I could go lighter.
You still have to trust that the server is not going to target you and send you a trojan version. Content hashes (like in IPFS or git) are needed to minimize the trust on servers.
Webassembly is going to be fantastic for building a lot of similar apps like this that avoid the server. It also keeps costs of the developer to a minimum because huge data files are not being transferred to/fro.
I’ve actually done this 10 years ago with HTML canvas - resizing images on client side before upload, to save bandwidth and speed everything up: https://github.com/josefrichter/resize
yes of course, wasm is far superior. we are 10 years farther, after all. it just reminded me that the problem of resizing before upload exists for at least a decade, even solutions exist for at least a decade, yet it’s still rarely being solved. maybe it’s not the biggest problem, so devs focus on other stuff.
I want to make an app that allows folks to upload an excel file and I want to extract just the top 10k rows of the first two columns to plot them. I’m afraid of allowing an upload option for a free service, can I parse an xls file using wasm and avoid uploading anything?
For client-side stuff like that, there is sheetjs which has support for XLSX but there are file size limitations. XLSX is a glorified zip file and to parse it you 1st have to unzip it and then build the XLSX file.
I found that it starts to hang on XLSX files ~50MB because the spreadsheet model it produces get's a bit too big and sheetjs dies. Doing this in a web worker is a good idea because otherwise it will clog the main thread.
For CSV the story is different, because you can read a plain text file in chunks (even in JS) with FileReader and readAsArrayBuffer(file.slice(byteOffset, byteOffset + chunkSize, file.type)).
Sadly, because of the nature of the web, one can never be sure that the image actually stays within the browser. There's no way to prevent it from secretly sending everything you convert with this to some shady website because it runs in the browser. I trust the people behind this just fine, but sadly the promise of offline-level privacy and running code inside the browser just don't go well together.
> Sadly, because of the nature of the web, one can never be sure that the image actually stays within the browser. There's no way to prevent it from secretly sending everything you convert with this to some shady website because it runs in the browser.
The thing is, that logic applies just as well to desktop apps. There's never any guarantee on either desktop or web unless you have very specific counter-measures in place (firewalls, airgapping, ...).
It's just "closer to the web" when it runs in the browser, but if malice is the intent, it's as likely on either. Desktop apps are just "harder" to install.
> A website might serve a different code every time you use it. It might even serve a malicious code specifically for some users.
Desktop apps can do this little trick, too. Also, many desktop apps can enlist the help of other desktop apps using the three decades or so of various ways our OSes have made to lett apps talk to each other. Bad actors were using macro languages in desktop apps long before the invention of the web browser. Now, it's just as easy for a desktop app to hit an endpoint, grab a script (shell or macro language) and then execute it. The browser is just another desktop app - one with much more scrutiny than that MP3 tagging thing with a built in Lua interpreter that Bob in sales just gave local admin privileges to...
Oh, and most desktop apps have some level of access to your identity - or outright have you confirm your identity when you activate your license.
> There's never any guarantee on either desktop or web unless you have very specific counter-measures in place (firewalls, airgapping, ...).
Windows Firewall commonly serves this purpose even among users who are not familiar with terms like “countermeasures.” The only problem with their system is, the warnings are so common that clicking through them becomes a reflex action.
Also, how many times have you launched an app for the first time on windows, use it flawlessly online for hours, and then close it only to find out that there was a firewall access request in the background the whole time. Failing to allow the app didn't make any difference at all.
You get a Windows Firewall prompt when an app wants to allow inbound connections, not outbound. This can include p2p technologies, most commonly things trying to register their own local discovery mechanism. If you don't use such inbound-initiated functionality it won't affect your outbound-initiated download requests.
It's a rare breed that runs Windows Firewall (or any firewall) in default deny outbound mode, I can't see more than 0.01% of users (i.e. no more than 1 in 10,000) being protected from this scenario with their current Windows Firewall setup.
This is why one reason why sensible Linux distros go though a multi-year process of release stabilization, assessment and testing.
While not impossible, the likelihood of, say, imagemagick doing this, while going through this process, is extremely improbable.
I agree that just downloading apps off random websites is every bit as risky as a browser app.
But who does that?!
(The last statement was sarcastic, and aimed at the curl | bash crowd, the nodejs crowd, the composer crowd... these things are the polar opposite of trusted, secured, audited, safe, sensible.)
Seems like you’ve conflated a lot of things here. Binaries in the Ubuntu Snap store are not reviewed for years, it’s days at best (and that’s only for classic mode, the confined apps are released without human review). Software you get from NPM isn’t a random website nor is it a desktop binary, it’s source code you can read. I trust random web pages much more than I trust Linux Desktop apps, because the browsers are far more audited and safe than applications, and I have complete control over what data I put in the browser.
> Software you get from NPM isn’t a random website nor is it a desktop binary, it’s source code you can read.
Most NPM libraries (even open source) are compiled and / or minified before they are uploaded to npm so it’s actually pretty common to get unreadable source code from npm.
Nothing in NPM ensures that the package code is the same thing you see on GitHub. This would require NPM to checkout and build all the packages by themselves.
So no, with NPM, you are totally running random code on your company’s computer. But surprisingly the world seems to be ok with that.
That’s a fair point, though are you sure about most? And are you saying most code only comes in minified form, or are you referring to libraries that come with a pre-minified option in addition to readable source? I’ve used NPM a bunch and rarely seen minified code only, and usually minification happens on my end as part of my site’s build, not to obscure JS library source code.
It’s true that use of NPM requires trust. At least NPM projects have a name attached to them and some accountability. It is by no means foolproof, but malicious projects lose their accounts along with the reputation that takes time to earn. The world does seem to be okay with a certain amount of trust.
This is really true of any software distribution system, even and including all distros of Linux. The point at the top of the thread is that we don’t have a way to know what happens when we download any software, or when we give our data to an application. We can protect our computers by auditing software and sandboxing systems, but the system still requires trust. Open source and auditing is still just a narrative that I have to trust from my point of view, unless I’m the person doing the auditing myself.
It might help to clarify the threat model. The poster I replied to confused a security model with a privacy model. The thread above it was about privacy, not about security. There is no software distribution model anywhere that addresses what happens with my data on somebody’s backend once sent, nor whether they can share data that I enter into an application over the internet, right?
So you pick one linux variant which doesn't follow my statements, then presume I'm conflating?
Just because ubuntu is terrible, and uses snaps, does not mean all distros do. You should have taken the high road, and debated what I was discussing here.
In Debian proper, you can read the source code of everything too. After all, old school distros like debian are they very reason you have such freedoms. They've been fighting the good fight for decades, working to keep OSS running pure, untarnished from closed licenses!
The very idea that you think Linix distros are less audited than npm, and have less accessible source code, is, to me, incomprehensible. It also shows how little you know about OSS, while you use the very tools giants crafted for you.
This doesn't make me mad, it makes me very, very sad.
Snaps seemed like a fair example since you didn’t specify, and Ubuntu is one of the largest distros by market share. I don’t know what licensing has to do with your original point. I don’t know what vanishingly small linux distros used by a tiny minority of technical people has to do with @scrollaway’s point. I don’t know how making assumptions about what I know to slide in an ad-hom zinger has to do with taking the high road.
The point that you bypassed with your straw man in the first place is that once a software distribution ecosystem gets large enough, it’s guaranteed to run on some amount of trust and gets too large to be auditable. This is why we have sandboxing, but ultimately even sandboxed apps when they get popular are a privacy threat, this is a problem Debian hasn’t even remotely solved.
I tried making a chrome dev tools network throttling profile set to "0 kbps", and it seems to work...you get "no internet" if you try and reload the page, xhr in the console fails, etc. So cumbersome, but it does work.
Android used to have a permission for internet access and when I was building an app that I wanted to prove to people that was local-only, fully offline, it helped to show that the app didn't even have permission to access the internet.
I wish browsers, Android, and iOS had this option so that users could be confident that the app/site doesn't have access to the internet. Maybe a little tricky with a browser but perhaps it could be possible somehow.
Offline-level privacy is not guaranteed with native apps either, so please don't live under that incorrect notion!
If you do not trust the developer of a web app, you can always disconnect from your Wi-Fi, use the app, close the tab and then reconnect to your Wi-Fi.
Firewalls have been built into Windows for around 19 years now. There are tons of other options on all other platforms (thought Apple's platforms have more restrictions than others) to prevent outgoing connections. Mobile firewalls seem to use built-in VPN functionality and can often block per domain and per app as well.
Web browsers don't come with such a firewall, and they need to be permitted by the system firewall to operate normally. It's technically possible to block outgoing requests with addons and configuration, but that's a pain to get right, especially compared to the simple toggle you can find in most OS firewalls.
You can, and you can also unplug the ethernet cable, but without any user-friendly, ways that are _actually intended_ to do, so it's easy to get wrong.
For example: you enable throttling, upload a picture, that picture gets buffered by a serviceworker, you close the tab, therefore closing the dev tools and disabling the throttling you set up, and the serviceworker uploads whatever you resized to the server anyway.
Or: the application just tries to upload stuff whenever you open the page, turning the throttling solution into a race between you and your browser.
I don't think using dev tools intended to replicate offline-only behaviour are a good way to actually enforce offline-only behaviour.
I actually use wasm codecs built by squoosh team. Squoosh is freaking awesome.
Resize to particular size idea came, when online service asked me to upload image upto 25KB. I had to just keep on changing size till image became 25KB.
But most people are not comfortable using CLI or someone may just not want to install it for one off conversion. Also webapp works on smartphones, tablets or anything with browser, while Imagemagick does not.
Imagemagick is ofcourse lot more feature rich and fast. If anyone is already comfortable using Imagemagick, webapp is not for them.
I sometimes think we're approaching a future where people will learn to bake bread by scraping the topping off frozen pizza unless we make an effort to educate users on how to use the tools they have at their disposal.
This, exactly. We're raising a generation where coding is synonymous with a browser, and the browser is effectively an operating system. It's getting a bit silly.
I don't see the value of commendeering a browser to run applications that would run more efficiently on the OS natively.
These points are only weakly in favor of such a web app.
Don't like CLI? Use gimp or similar.
Don't want to install anything? On some OS it might make sense. On that OS consider Paint. It probably does not support webp, but if you're serious about webp you have tools.
Smartphone/tablet? Is there even a need to do conversion on those?
Privacy? Use an operating system that has some reputation, like Debian stable, not a web app.
IMHO this web app is a nice exercice for its author.
Please don't respond to a bad comment with swipes and putdowns. HN is a statistical distribution across a wide spectrum of comments. The only helpful thing to do is to make sure your own contributions raise the median quality, rather than dragging it down further. Your comment would have been fine without the first sentence. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
> HN feedback has become cancer
It's all too easy to succumb to false feelings of generality about this kind of thing [1], because (a) there's a spectrum, as I mentioned, and (b) we're all inclined to notice the things we dislike, and weight them more heavily [2]. But these are all comments that other users posted to the same thread:
Those commenters all managed to offer positive feedback without combining it with negative generalizations about others. If you could do the same in the future, that would be very helpful.
Negative feedback (including yours, and mine, and that of those who may reply to this disagreeing with me) has its place.
I’ve seen forums (fora?) where almost everyone is exclusively positive (like some reddit look-what-I-made topics), and that well-intentioned behavior can be pretty unhelpful if you’re trying to get different perspectives on your own project.
Yet there are always people there who bend over backwards to make someone feel bad if the person was (sarcasm: rude enough) to suggest there may be a way to improve something. Enforcing the positivity-only social norms.
I’m glad that HN, by contrast, stands out as having more free range thinkers who tell it like it is.
This is wholly true, but the comment above makes absurd claims ("Smartphone/tablet? No one converts there! Want privacy? Use Debian!"), bending backwards to say something negative.
Images are not send to server, all processing is on client side. Server is required to download the webpage/wasm, but not to process files.
In future planning to make the app available offline using service worker. So first time internet would be required, next time onwards without internet also it will work.
You can certainly open local html files that run JS using a web browser while your network stack is offline though, and doing so can't be described as being online. I guess you could argue whether doing this actually constitutes opening "web pages" but I think most people would say it does.
Would it be possible to eliminate the need to be online, so then you could make the much more enticing claim (for privacy-conscious users) that it works offline? Maybe a PWA, or just by clicking "File > Save."
This isn't any different from any other image processing app that you download and use, except you have to download it every time you need to use it again. So the advantage is that you don't have to make multiple versions of native apps, and you didn't make it an electron app. Other than that, it's YA image tool. I hope you learned whatever you set out to learn with this.
Here are a few ideas to make yours faster:
1. Use the browser's canvas element to compress JPEG or PNG images instead of WASM. The libraries which come with browsers can use processor-specific instructions, while WASM only targets the smallest common denominator of the most common architectures, so it will be slower.
2. Which resizing method are you using to find a fitting image size? For me, it worked well enough to downscale by some factor a few times until it fits, but if you want to get super close, you could use binary search.