Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: TxtNet Browser – Browse the Web over SMS, No Wi-Fi/Mobile Data Needed (github.com/lukeaschenbrenner)
234 points by lukeasch21 on April 21, 2023 | hide | past | favorite | 73 comments
Hello all,

This is my second year of working on a project[1] with the goal of browsing the web, on an Android smartphone, without reliance on Wi-Fi or mobile data. While this concept might seem aimless, my goal was to provide a way for people in areas with limited, expensive, or censored cellular internet access a way to view the web in a basic format. I finished work on a basic client-server model last year[2], and this year, I implemented a new pseudo-distributed peer-to-peer model that allows any TxtNet Browser user to use their own smartphone to run a background server service that communicates via the user's own primary mobile number. The main advantage to this model over last year's use of the Twilio API is the fact that with an unlimited SMS plan from a consumer carrier, you will likely end up paying significantly less than the amount you would pay for Twilio credits (averaging about ~$0.50 per website). There's a lot going on with the stateless nature of SMS, GSM-7 encoding, and Brotli compression, so please ask any questions you might have!

I've also started up a test server instance running on a +1 country code phone number, so feel free to test out the app with your own smartphone. Like mentioned in the GitHub repo, please be aware that I (necessarily) have access to every phone number and associated request that is sent. Of course, anyone can host their own server instance, and if you would like to share it, feel free to get in touch so I can add the number to the repo! Also, there are likely many bugs still lurking, so feel free to report those.

[1] https://github.com/lukeaschenbrenner/TxtNet-Browser/

[2] https://news.ycombinator.com/item?id=32905496




I used to run a small relatively unknown service back in the day when internet on mobile was still unaffordable in India.

People could sms to a number that I had procured with any keyword. The service send the user a list of 5 top results' domain names from Google results for that keyword. User would respond with a number (1-5). The service would send the 20 characters of the first h1 tag on the page, and first 140 characters of the first body tag on the page. (tyring it's best to not send any markup but just the text). Last 20 characters would be an ad.

It was buggy but it had decent traction initially. I didn't get any advertisers so for the most part of it the ad text was something like 'advertise here - with short link to contact from'

Reminds me of those days where every kb accessible on mobile was so valuable.


that reminds me of how my first blackberry used to function. it was mindblowing at the time.


This is pretty cool, but I wonder if you came across SMS Without Borders[1] and, if so, how your solution is different/better?

On this note, just as an FYI, I'm leading the Awala project[2], which is a new computer network where compatible apps use the Internet when it's available, but can also switch to a fallback medium when it's unavailable. The only fallback medium we have today is a sneakernet, and SMS support[3] might be added in the future but only for high-priority messages (given how expensive and slow it'd be).

[1] https://smswithoutborders.com/ [2] https://awala.network/ [3] https://github.com/AwalaNetwork/specs/issues/81


Thanks for your comment. In all my research, somehow I never came across either of those two projects! SMS without borders looks the most similar to what I'm trying to accomplish but their goal is much more focused. I would say the biggest difference is the fact that they are focused on supporting personal 2-way chat services like email and telegram, and because you have to have Internet access when you sign up with their service, they are able to generate encryption keys to obfuscate the communication through relays. Whereas with my project you are never assumed to have Internet access, even if I did implement encryption it would only be transport level, so the gateway could still read all your data. But the concept of being only able to access globally available websites makes it so that unless a government starts decoding messages in transit, you're not going to have any major upsides to encryption if you trust the server host. Another major difference is the fact that setting up your own SMSWB instance requires a lot of technical skill and infrastructure, which isn't the case with TxtNet. Both TxtNet and SMSWB are different enough that I would say they actually complement each other really well. Honestly I'm pretty impressed!

The Awala network is similar to what I was imagining in place of IP over SMS. I think the main problem with that approach is that for the average user, adapting most programs directly to the protocol would be confusing to use. The ping example provided in your demo video is of course only a PoC, but translating that to, say, a web browser could be confusing to users who don't understand why when they request the website, they have to keep the tab open and then come back to it once they've connected a hotspot who happened to request the right websites (possibly much) earlier. But still, I think it's totally possible to adapt programs to make them fit this paradigm better and I'm excited to see what comes next out of the project! It seems like a very promising solution more resilient to intermittent connection failure and just from the looks of it, you've got really brilliant minds working on the project (I'm not an expert, but I did find that some of the encryption concepts mentioned in the SMS spec discussion went over my head!) You've also managed to secure funding from higher profile names, which is awesome.


Thank you so much for the feedback and the kind words!

Based on my limited/high-level understanding of the SMSWB and TxtNet projects, your assessment sounds about right to me.

Re: UX for Awala-compatible apps, I totally agree it's going to be difficult for people who are used to standard Internet apps, like web browsers. In fact, I'd say it's going to be even more challenging for people in those regions where they do have smartphones/PCs but have never had any connection to the Internet.

In both cases, I believe the solution is to totally rethink the common UX patterns we employ in networked apps, where the lack of connectivity would be treated as an exceptional event that the user has to sort out.[1]

This is top of mind for me and I'm actively working on a partnership with other organisations in this field to produce a guide for UX designers. I'm hoping we get to announce it and get to work on it in the coming weeks!

[1] https://awala.network/service-providers/ux


I feel like this would be a perfect use case for Chat GPT (with something like the now removed web browsing plugin).

Instead of sending the whole HTML page, the user could request just the information they want. You could even instruct Chat GPT to use abbreviations and shorter words as much as possible.

This could even go through one of these extremely long-range but low-bitrate radio standards to provide web access to literally the whole world.


this! I was thinking the same


Maybe it is a shame that WAP usage died off as you could probably get the essential information from a site with very few bytes. Combined with something like this it would be great for getting stuff done.


I feel a WAP resurgence will be inevitable once we are all in space ;-)


In space you may have minutes of latency. Need some good edge caches.


or if Apple can reconfigure its emergency use satellite connection setup for actual data but at very slow rates.

I wish they could set it up so it could work in download-only and you could get some news/weather updates at least


There was a story here a few months ago about a low-speed connection thingy that would take a webpage and convert all images into coloured ascii so the formatting of the original page still made sense. Like a Lynx browser but works well on modern pages.

You could install it on your server somewhere and then access it remotely and your server would do the transcoding. Was intended for people on slow connections...

Could be awesome chained with this!

Anyone have the link? Can't find it...

I found this, but I think there was something more recent: https://softwarerecs.stackexchange.com/questions/52166/linux...


Maybe Carbonyl? (https://github.com/fathyb/carbonyl) It's a chromium browser for the terminal. You could use it as a backend.


Browsh relies in Firefox to generate pages quite accurately I think


Isn't SMS a more expensive transport usually? Wherever I've seen paid and limited SMS, they were ridiculously expensive for the amount of data transferred (e.g. in Indonesia).

Do you have a specific "target market" in mind, with cheap / unlimited SMS but censored internet access? E.g. China, Russia, Turkey?


I know for sure that what you're saying was true around 10 years ago. Once apps like WhatsApp began to gain popularity in some countries, though, I've seen a few examples of carriers allowing for unlimited SMS and switching to focus on charging for data, like what has happened in the US. But off the top of my head, I can't think of a specific example. I don't think that my ideal "target market" would need both cheap SMS and censored internet access; for me, those are two different applications. If your internet access is censored, even just a small amount of information could prove to be very useful. The other application is countries with cheap SMS plans but expensive data plans, which, like you pointed out, has a pretty limited application.


True here in India as well. The cheapest pack offers 2GB data/ day and 100 SMS/ day for 3.5 USD per month


SMS messages in Russia are expensive by default. Most carriers offer cheap-ish unlimited SMS packages but I've no idea who they're for. Or how unlimited they really are.

As far as censorship goes, a VPN is better by all metrics than this.


Absolutely - this isn't a VPN replacement, especially with all the limitations it has. It's more of an alternative to cases in which a VPN might not work (or work well) depending on the situation.


Russian internet censorship is very simple. They don't seem to be using DPI to target VPN protocols at all, instead trying to block IPs/subnets of known commercial VPN providers. Blocking IPs, subnets, and domains is the primary operating mode of Roskomnadzor.

There allegedly is DPI, but it only comes into play as an extra measure against things like domain fronting, proxies, and plaintext HTTP. It'd look at the SNI and inject a RST packet to drop the connection. As it does not even try to implement TCP, analyzing each IP packet in isolation, there are various interesting utilities[1] that mess with it by, for example, splitting the TLS ClientHello into two TCP packets in the middle of the domain name.

[1] https://github.com/ValdikSS/GoodbyeDPI


Quick somebody PoC this with SMSGTE over APRS! (So you can browse the web over ham radio. Ridiculously impractical but delicious!)

https://smsgte.org/


TCP/IP over HF is already a thing! (And also ridiculously impractical :) ) https://www.isode.com/whitepapers/stanag-5066.html


Nice! One small thing I noticed: it looks like you don't have compression turned up all the way. You might be able to save a lot of bandwidth if you use the most aggressive brotli settings for everything that's sent over SMS


I just ran some non-scientific tests with compression quality set to 11 (max), and these were the results:

frogfind.com - default: 3 messages :: new: 3 messages

lite.cnn.com - default: 43 messages :: new: 44 messages

text.npr.org - default: 8 messages :: new: 8 messages

"NFL Suspends 5 Players for Gambling" article: default: 15 messages :: new: 15 messages

Seems like either I haven't properly implemented the dictionary, or the default compression level is already set to max.


Worth sharing these here:

https://blog.wturrell.co.uk/text-only-news-websites/

https://news.ycombinator.com/item?id=35313232


I actually came across that post earlier too and included the most useful and applicable links inside of the app's welcome page under Quick Links!


Thanks, I didn't realize that Brotli had multiple compression levels. I'll look into that!


Thanks for this, it reminded me of working with WAP (Wireless Application Protocol)[1] a long time ago.

[1] https://en.wikipedia.org/wiki/Wireless_Application_Protocol


It looks like HN successfully killed the server, I'm on my way to investigate. Thanks all (:


Have you considered using T-Mobile's WebDigits ( https://webdigits.t-mobile.com ), AT&T NumberSync for computer or whatever Verizon and other carriers have to offer for handling SMS on a computer perchance?

You could also use multiple phone numbers to send SMS to avoid SMS ratelimiting. FYI most carriers consider Person to Person messaging to be below 1 message per minute and run between 40% inbound to 60% inbound messages.

One thing to watch out for is some carriers like Cellcom if I recall sell unlimited texting plans that actually have a 20,000 message cap then $0.01 per message thereafter. A cousin of mine hit this about a decade ago as she was part of many MMS groups.


Those look like interesting options, and they're surely be more reliable than relying on the phone's own SMS capabilities, but I wasn't able to find a public API for any of those services. As far as SMS rate limiting goes, I agree that multiple numbers would be good. Perhaps if there's a demand for it, I can create a feature that switches numbers from the database on the fly as it notices some are hung up.

>FYI most carriers consider Person to Person messaging to be below 1 message per minute and run between 40% inbound to 60% inbound messages.

Yep, I've read into that a little. I wasn't able to find anything explicitly forbidding a service like this in their terms, although there's a lot of legal documents that carriers have so I could have missed it. I read that you should expect about 1 SMS/second on consumer carriers, but it seems like I'm able to get about 2-3 SMS/second in my testing.

I didn't know about carriers having an unlimited SMS threshold... I'll have to look into that, thanks for the heads up!


Yeah, neither WebDigits nor other carrier offered services seem to offer a public API, it would be a matter of reverse engineering their website :c


SMS is limited to 1 message per 3-6 seconds. The modem incidentally limits throughput.

Also, Google Fi has unlimited text/call for $20/m. Their SIM does work with USB GSM modems.


GSM is not available on AT&T or Verizon and T-Mobile is ending GSM support in under a year. Right now T-Mobile's GSM is running in a 200Khz carrier in the guard bands of LTE, making it perform worse due to the narrow bandwidth which means your allocated fewer timeslots which limits message throughput, and interference from the LTE carrier directly beside it degrades signal quality as well.

Nevermind most GSM stacks crash when receiving/sending many messages quickly, they aren't designed for high throughput messaging.

Sending via a carrier's multi-device calling/texting option where you bypass their cell network and connect to their servers over the web, or using VoLTE or VoNR is going to perform much better than GSM


The server is back up and running - it turns out that CDMA networks continue to be an issue, and the queue of WebViews was being depleted as any WebView that received a malformed string failed. I fixed that, but the string should never have made it all the way to the webview in the first place, so that was also fixed.


Could be usefull for places where government have shutdown internet for weeks https://www.thehindu.com/news/national/other-states/sambalpu...


My thoughts exactly - other places like Iran have also experienced deliberate government internet blackouts recently. Thanks for sharing.


That's really cool! There are situations where data is unreliable or congested, having a backup is very handy. SMS is very common LMICs as data can be expensive, nevermind the devices themselves are difficult to come by. I could see a UCS-2 implementation being a great feature for a project like this.

I built something similar recently (inspired by this project), a SMS to DuckDuckGo search [1] and weather lookup [2]. The submitted repo's app is Android only, so having a more universal approach is fun to have. I live and work remotely, my closest neighbour's km's away do not have internet but only cell phones with unlimited SMS. With no internet, they would not be able to download the app. Sort a chicken and the egg problem.

[1] https://github.com/snacsnoc/smsferret [2] https://github.com/snacsnoc/tomorrowsms


Thanks for the praise, and yes a UCS-2 based solution would be the natural progression of this project. Unfortunately it becomes much more complicated to encode because of how many code points are available! Your DuckDuckGo and Weather solutions look great, especially since they are focused on specific problems, they are space-efficient and easy to use without an app. It's interesting that you bring up your case of rural neighbors with unlimited SMS but not internet - I do see your point about the chicken-and-egg problem, but my intentions were that since the app is packaged as an apk, the file could be shared around when people meet up over Bluetooth or similar protocols.


Would be super super useful over WhatsApp.

Is it possible?

There are some cases where only WhatsApp is available as free WiFi...


Like this?

https://alexanderell.is/posts/wikipedia-over-whatsapp/

https://news.ycombinator.com/item?id=31463249

I'm sure I've seen what you're describing done before to make use of free WhatsApp in mobile plans in some countries, just can't find it, although I remember that one of the limitations found was rate limiting by WhatsApp, which is to be expected.


Those countries with only WhatsApp available for free over internet should do something about net neutrality. When the telcos can sell app bundles, they will not focus on acting as an infrastructure provider.


or DNS port ? i remember some captive portal would block all traffic except dns request until you login.


Very cool - I've often thought about trying to set something up like this for situations - mostly on airplanes - where you're allowed some kind of "free messaging" but otherwise don't have any data.


Thanks and yeah, that's another good application of the idea. It's somewhat similar to iodine[1] in that respect. On the ground in the US, this would probably only be useful if you're hiking in a remote area or something like that, due to how cheap data plans are. It might also come in handy if your carrier doesn't charge for SMS when abroad.

[1] https://github.com/yarrick/iodine


I know everyone is sick of hearing this but that may be a good use of ai agents, to interface the web over sms


Okay this is actually super clever.. Really nice job !

Edit : I notice you don't have a server in Italy, I might be able to help for a while at least, I'll contact on GH.


Thanks, and that would be awesome! I'll keep an eye out.


Those in countries with a low median income and prohibitively expensive data plans

I suspect SMS is likely to also be "prohibitively expensive" in those situations.

That said, although this is another good tunnel, I wonder if literal "dial-up" going through the (far more likely to be available on a mobile phone) voice channel will provide better bandwidth.


For most cases, I imagine you are correct. There are probably only a few edge cases where this does work out financially. I've explored using the voice channel as an option, but that would require hardware to re-route call audio to the app since no API is available without root.


Reminds of the of the now-defunct JPG Internet project, which was a Facebook page that you could post a link and it would comment back an image of the page. For when your ISP only allows to to view Facebook.

https://www.facebook.com/JPGInternet/


On boats (once land is out of sight) the only connection option is satellite data, which is slow and very expensive. Its been a while, but I remember companies existed offering similar 'compression services' (using a data connection, not sms). TxtNet Browser could be an interesting alternative.


When I started programming, I thought this would be such a fun project. It brings me joy to see someone did it!


I'm glad to hear. This is actually sort of a spiritual successor to the now defunct Cosmos browser app that achieved a similar goal back in 2014, which was the original inspiration for my project!


This is how I used google over the SMS before I had my TMobile G1. Fun memory.


I remember you could text various things to a Google phone number and it would respond.

https://en.wikipedia.org/wiki/GOOG-411

> Google also operated a similar service from SMS number 466453 which has also been discontinued.

I used it occasionally to get stock quotes when I bought my very first shares and wanted to check at lunch.


This reminds me of the times that I used GPRS and WAP to browse the net on older cellphones without an actual web browser. It was extremely slow, but very useful.


What about something more general and even more text-based, like "Execute Console/Commandline Programs over SMS".

Edit: What I mean is sending text commands to a restricted shell, i.e., program launcher, over SMS. Then returning the console text output from the server over SMS to the Android user.


In theory, you could adapt this approach to whatever needs you have. I have heard people suggest implementing IP over SMS, setup as a local VPN. I would say the sky is the limit, but I think with how brittle, unreliable, and slow SMS is, IP over SMS would definitely be out of scope. If you wanted to make a quick jank solution to your problem, this app works with all HTTP GET requests. So, simply host a local webserver on your PC, have your server phone connected to the same Wi-Fi, and provide a text input box that will accept any command and append it to the url with a basic <form> tag. Then the server can generate the output, and send the result back to you.


"What about something more general and even more text-based, like "Execute Console/Commandline Programs over SMS"."

I built this a couple months ago. It's fairly straightforward.


Super cool, I like how you can host a server easily with another phone. Is there something equivalent but normal web browsing that compresses the data on a server before it sends it? For instance, T-mobile has free internet overseas but it’s usually stupidly slow since it runs on edge.


Thanks! And yep, my goal was to make it as easy as possible to host a server without any technical knowledge. I haven't looked into the subject extensively, but I believe those services exist(ed) in the form of Opera Mini and UC Browser. IIRC, Opera Mini's Android app still supports this. Sadly, it's not open source, so you have to trust Opera with all your data, and Opera hasn't exactly had the best track record... I doubt UC Browser is any better in that regard. I used both about 10 years ago when they still had J2ME versions of their apps... It was kind of surreal to use Opera Mini's on a touchscreen dumbphone because they had updated their midlet to look very similar to the iOS 7 version, on a much much inferior device! It worked surprisingly well, compressing text and images.


Yeah, I haven’t trusted opera since they got bought out about a decade ago or so (time flies). Really wish there was a hostable option. Your work is still awesome, I’m gonna give it a go next time I travel overseas!


Opera Mini does this, but IIRC it's not self-hostable.


Very cool! Found this from 9 years ago, deprecated now however. https://news.ycombinator.com/item?id=8304409


Thanks for the link. I actually spoke with the lead developer of Cosmos browser earlier and he seemed enthusiastic about my idea of continuing their work in a way. In fact, before I rebased my app the original PoC was a fork of that app with a bunch of patches to work on newer versions of Android!


Have we come full circle? Kind of reminds of the mid 2000s when I was a kid.


Thank yoh for making this - it's incredible! Are you able to cover you expenses from it or is it simply from the goodness of your heart?


This is amazing! could this be applied to handle payments? Specifically in countries/areas with poor/low internet penetration.


I remember seeing a service that did this over email a couple decades ago.


Very cool, great work!


Are you considering any compression algorithm?


If you mean any kind of custom tailored algorithm, not really. Given I'm only dealing with HTML right now, Brotli has been a good solution. If I end up transmitting other binary data like images, I think well established algorithms like gzip should be okay.


[deleted]




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: