Why am I open sourcing this?
I don't see any other RBI / CBII vendor open sourcing their platform and in the security industry "closed source" can create issues.
But what about business defensibility?
I agree. Open sourcing removes the trade secret aspect that could make a defensible business.
At the same time, a determined hacker would already have my source code. A hacked "free wifi" connection here, a bit of social engineering there, and my so-called "competitive advantage" could be easily removed. Access to GitHub, Gitlab, other accounts would prove no obstacle for someone motivated, and open sourcing is a way to remove the advantage any small group of parties has by keeping it secret.
How do I self-host it?
There's instructions on the repository page.
For the CE release, I deleted the Git history (it became too difficult to deal with branch rewrites trying to remove all the cruft and deployment secret keys etc).
But the actual current working repo for the non-free version has ~ 2400 commits. And the repo that I forked that from (~ 7 months ago), I'd closed 238 issues, and the repo I forked that project from I closed 200 issues. Those previous repos are all my work built from the ground up within this last ~12 months as well. The current working repo has closed 124 issues.
My Gitlab contribs for the last year are at: https://gitlab.com/dosycorp
And it's actually a little bit over 1 year, and if I count the Gitlab contribs show there, it's over 5500.
So, yeah it's been about 2000 hours I think. Working full time every week day and most weekends, and often longer than regular hours (but also dividing the work up in the day into sprints because that's how I manage myself to work best).
It's definitely been a year.
I dig your dedication! You well earned to be on the front page!
Who do you see as the main target audience? And do you have a business model in mind?
Main audience is people and organizations who are having problems with malware and cyber attacks.
Business model I am still working on, but it's a mishmash of licensing (+ maintenance, for hybrid / on-prem) and pay per seat (for cloud-based). Could be other ways to provide value.
What's your background? I'm open to new ideas.
this sounds like (almost) everyone
Mixpanel was a cool interview. I remember the question was about data structure for a rank system (it was a heap). The office in SF is beautiful, and the lunchtime catered food, was great.
I guess I'd have more to say about MA if I knew more about it! I did not try it yet and I don't know how they make Chrome faster in the cloud and deliver a seamless broadcast of the screen to the user, but with enough smart people, I'm sure it's possible.
As I said in a previous comment, I definitely think the future is mreo 'app virtualization' in the cloud, in some sense perhaps MA is starting at the beginning of that curve.
Why would you think that? Where do you see this trend? Their app looks incredibly stupid to me and it is hard to imagine any large number of people that would use it.
Your project aims to provide security, which is an interesting goal, that I think security-conscious people would want to use.
But to virtualize a browser... for performance? For browser apps (which are already "virtualized" desktop apps, of sorts), trading off enormous amounts of bandwidth, for what, cheap RAM sticks that are getting cheaper by the month and faster SSDs for quick swap? For what usage case? That one guy who happens to have a chromebook but needs to run 100 tabs actively?
You could just run "unload tab" extension already...
Or like just have a desktop that you VNC into... Which all of the people that need this have been doing for decades...
For that being said, I agree that virtualizing apps "right now" is stupid outside a niche unless you have some secret sauce (like Mighty must, otherwise why?), because it's too much overhead.
Preserving resources is why I run headless, and that's why I avoided WebRTC/VNC/video instead of judiciously sent screenshots only on change.
this will also give absolutely new meaning to "deplatforming" which will now mean that not only you can't post on some social media website but you can't use any computer at all.
It might be work stopping the browser 'loading' itself inception style with some kind of blacklisting of loadable urls - I assume you already have something to protect against SSRF type attacks on the platform itself.
I like it can load itself.
What's the biggest problem you've had with that ability?
Edit: It's getting kind of slammed, right now. I might need to resize it up again in a while.
Edit2: Alright, I have to resize it, it needs more power. Hold on while it comes back up. Queue is rebuilding @23:13 ET.
Edit3: Okay, back up @23:20 ET. Please enjoy!
Nobody sane relies on anything that's closed-source.
Edit: OK, nobody prudent.
It works by providing a thin client over the web that you connect your regular browser to. The thin client provides an interface to a remote browser that you interact with the browser the public internet.
This is significant because the internet is a cesspool of attacks. Malware, ransomware, virii, tracking, exploited PDFs, ways to deliver device zero days over the web, browser zero days. All these things can lead to the security of your device and network being compromised, causing significant inconvenience, distress and loss for you.
BrowserGap and the RBI methodology acknowledges that not all threats can be detected and neutralized (such as by virus scanners), in order to face that reality, RBI adopts a "isolation" posture towards threats, effectively isolating them in the remote machine and preventing them from reaching your device.
Cloud-based internet isolation is another name for this security practice and it is an emerging industry. Symantec recently acquired a company in this space, and Menlo Security was awarded an agreement to build a CBII prototype for DISA, after a June 2018 request for RBI solutions that could eventually serve 60% of DoD's ~ 3 million users.
See more about RBI.
I mean, in theory the web is a cesspool of malware, but with reasonably good content blocking (I’m not even in the completely-disable-JS crowd) and conscious avoidance of shady sites, I managed to pretty easily stay clear of all attacks so far, at least over the past decade.
Those way more paranoid than me still have the option of using local VMs/containers without too much compromise. Then the attacker really needs an exceptional exploit chain to escape all the way; it’s hard to imagine any group blowing such a valuable chain on a drive-by.
So, why would anyone sacrifice the ability to interact with text, resolution, color accuracy, frame rate, etc. to reduce the minuscule chance of drive-by attacks (assuming otherwise reasonable opsec)? Extremely high value targets?
But then, why would extremely high value targets trust a MITM? (Self-hosting apparently changes that to some extent.) Also, even if you run your browser in the cloud, that browser could still be hacked and leak sensitive information or actively modify traffic, no? So this isn’t even bulletproof for high value targets.
I currently use a proxy and have instructions on how to use FoxyProxy to access each env's environment. This will provide for a much nicer UX where you simply click a link and you're brought to a virtual tab in that env. I'm sure some things will break, so the proxy is a backup, but for 90% of our work I think this is amazing!
Solves any app problem where you have the same hostname per environment
> sacrifice the ability to interact with text, resolution, color accuracy, frame rate, etc.
it’s very much an understatement... Pretty sure your devs/testers won’t appreciate the experience. Frontend devs in particular can’t possibly work with this.
I fail to see why it’s hard for you to spin up (possibly gated) dev/staging instances; certainly much easier and much less resource intensive than something like this.
Anyway, your use case only makes sense when the code can be self-hosted, but apparently this product / product category has customers before the source is opened up, and that’s what I’m curious about.
Customer base is people and organizations who are having problems with malware and cyber attacks.
> Have you tried using this? When I said
> sacrifice the ability to interact with text, resolution, color accuracy, frame rate, etc.
> it’s very much an understatement... Pretty sure your devs/testers won’t appreciate the experience. Frontend devs in particular can’t possibly work with this.
I totally agree the image quality can be much improved. So I'm really sorry you had this experience today trying it out!
Would you be unwilling to mail me firstname.lastname@example.org and I can contact you if and when I have image improvements to share?
Initially, I used JPEG for all clients, then for clients with browsers that support WebP (chrome) I switch on WebP since the quality increase is a LOT (but WebP in FF looks pixelated, so I hope I can find a way around that), even tho the bandwidth is the same.
For Safari and iOS the quality is on JPEG. It sounds like it has sacrificed the ability ot interact with text, resolution, color accuracy and frame rate, etc. I'm really sorry about this.
Some people seem okay to roll RBI out in a test deployment, without the code being open-sourced. I can't speak directly for them, but I assume that Symantec (who bought FireGlass Browser), Menlo, WEBGAP, Light Point, Ericom, Authentic8, Citrix all have some customers even tho they are not OSS. I think that, often, as long as the contract provides the ability to examine the code if required (due diligence) even without publishing it openly, sales happen.
It sounds like you're unfamiliar with RBI, is that right? This is still an emerging industry so it makes sense to me that even if you are in security you are unfamiliar with RBI.
Now I can see that while this would probably be an overkill security-conscious individuals, it might make sense for organizations because there are always employees who can be easily tricked into clicking anything. I do wonder whether it's more effective and productive to instead enforce host-based blocking + browser-level content blocking + lightweight virtualization (like Windows Sandbox? Not sure how well it works since I'm a Mac user for the most part), but I'm in no position to evaluate for organizations.
Having checked Symantec's website, they seem to advocate falling back to a remote browser when the site is potentially risky, which sounds reasonable.
> then for clients with browsers that support WebP (chrome) I switch on WebP since the quality increase is a LOT
Yeah, I first tried the service on my iPad Pro, image quality was terrible. I have since tried it again in desktop Chrome and it's definitely passable. That's unfortunate.
Anyway, I'm probably not in the target market, but best of luck to your business.
Interesting hearing you know about RBI. Did you evaluate any of the other services? What did you feel about them?
I definitely think the approach you say (host level blocking, content blocking and some lightweight virtualization, like Edge/Windows Sandbox, or a local VM) is a valid one that reduces risks.
I think it comes down to considering, when attacks inevitably occur, where do you want to be doing the cleanup? Zapping a few containers, or instances in the cloud and starting them frehh, or decontaminating the local machines and network?
That's the biggest convenience for me so far.
> Genuinely curious: who's your customer base?
Anyway, thank you so much for being interested in this product, especially for helping make the space for me to speak about the type of customer, the risks they face, and their reasons for adopting BrowserGap. I really appreciate your time on this!
Would you like to email me email@example.com and we can discuss something tailored towards this? :)
I'd be really happy to help you.
It sounds like with reasonably good content blocking a person can avoid all attacks, and it looks like using local VMs/containers mean then the attack needs an exceptional exploit chain to escape all the way, and it's hard to imagine any target being caught in a drive-by.
I agree. Also, internet-based attacks are a real problem for many businesses and organizations. For example, in 2016 Singapore government mandated that RBI had to be used because of ongoing attacks.
How can it both be true that most people avoid any attacks whatsoever (and therefore that most simple measures are sufficient) while at the same time, malware is a real industry inflicting real damage on organizations?
I think of malware as an "industry" and as a collection "criminal enterprises" and from that viewpoint the malware industry has certain goals and markets it seeks to penetrate. If you don't find yourself in one of those target groups, that's a good thing. If you do, then you probably are already exploring RBI or CBII to some extent.
So your logic is correct and it looks like you are simply not in the target group.
At the same time, I think categorizing the only victim of web-based malware that can benefit from RBI/CBII as "extremely high value targets" is misleading. Perhaps the finance department in a Fortune 500 company is an "EHVT", but there's a lot of web-based attacks that succeed at targeting businesses and organization units of many different sorts, and the costs inflicted are significant and important, not just to "EHVTs".
Your concern about trust and the cloud is valid, and perceptive, as is the solution you propose (self-hosting). Self-hosting is indeed the right choice for many. That's one reason I think OSS has a role to play in RBI/CBII.
I investigated this path after PoCing the original (you can see the code in a directory plugins/appminifier and public/voodoo/src/plugins/appminifer, I think) but there's all sorts of interaction issues that arise when you attempt to filter the HTML in this way.
Efficient in terms of what? Bandwidth, from a certain point of view but you lose the "source of absolute truth" that a screenshot is, and at the cost of interaction quirks. Also, not necessarily CPU efficient, as you have to do a lot of bookkeeping to transmit events to the right places and keep the local tree in sync with the remote tree.
The main reason I avoided it was the security holes introduced by breaking the strict isolation model of pixels, and the complexity.
You're welcome to fork and improve on the work begun in appminifier! If you go down that path, just know, there be dragons, and good luck!
> it would be far more efficient than sending back a pixel video stream
Yes it would also deliver back exactly all the vulnerabilities to you that this project was specifically made for protecting against... Are you sure you have read its description? The whole point is to do dumb pixels that can't be exploited.
I am a pretty technical person but also didn't know (and couldn't find) info on what RBI is and how it can be used - would love to see this information highlighted on GH and the website.
What really excites me about this is the ability to deliver a secure "workflow automation" solution to people over any device.
Would be really cool to set this up and set the server up to do some proxy-hopping to make IP tracking more difficult as well, regardless of client device and when roaming.. While I'd be more leaning towards self-hosting, if you set this up as a subscription service, your users will also benefit from sharing the same pool of IPs (though I imagine you'd also face issues with getting flagged/blacklisted/CAPTCHAd a lot through abusers and bots, that will be significant work to polise if you go that route).
Thanks for sharing your feeling about this.
It looks like you're working on almost exactly the same project, and that the big challenge is getting access to server hardware for webbrowsing, because not only are AWS etc expensive, they primarily offer "webservers" optimized for very light not very CPU intensive workloads, and needless to say they also don't offer hardware accelerated video decoding.
Wow! Sounds like you're doing something interesting. Are you uninterested in collaboration? I was thinking of ways to make the video better, but right now I'm basically just using DIY "MJPEG" over websocket.
As for the server hardware, I was decided to join the Stripe Atlas program at the start of 2017, and from that I was able to get 5K in AWS credits, and then more Google Cloud credits, and I also applied to IBM and Digital Ocean on my own and got credits from them as well.
So, so far I have been able to develop and then demo this (like today) without significant monetary cost.
I also have some tips for you, because resource usage was one of my concerns, but TBH I find Chrome headless actually always uses less CPU than I imagine. It's all about the page that it is rendering. The page determines everything, but Chrome itself is very light. So when I've budgeted for like 1 CPU per user, it's actually possible to get much more than that. And memory is the big thing that Chrome does lightly, it uses barely any RAM even with 100s of users on a machine. I was surprised by that. 100 users running Chrome and only ~ 20 Gb of RAM used.
Also, regarding video, because I'm avoiding expensive video encoding (just sending screenshots) I avoid the CPU load of doing that. I've experimented with doing more processing of the frames, but it just throws the load way off.
I chose to keep it simple and I'm pleased with that. At the same time, I want to explore ways to improve image quality.
here's some personal notes from this. i didn't work on containerization, just raw VM setup. had to get x11 up and running and then set ffmpeg up for hardware streaming.
It has too much latency, text doesn't get entered, scrolling doesn't work, etc... Is the experience from US similar? Or is this just b.c. of server latency?
If you want, I can test it. I'll open up an "East Asia" instance and we can see how if it's any better.
Edit: Okay an instance in East Asia is up and running!
It’s now ‘usable’, but definitely not a good environment. Scrolling is unbearable, and once you start scrolling it doesn’t stop, so it’s a pain to navigate.
But I'm really sorry about the scrolling. Sounds like it's unbearable. I need to fix that!
I added a "fast mode" for scrolling where if you scroll more than 40% of the screen in one go, it accelerates the amount, so you could try scrolling smaller, but I don't feel that's a satisfying solution for you. Scrolling is really important to get right, I'm very sorry about that!
If you want, email me at firstname.lastname@example.org and I will work on it and let you know.
Edit: I've had other reports about Safari / iOS being really unusable today. I just tried turning up the image quality for iOS / Safari it should make things better.
Thank you so much for sharing this.
I'm looking at your app and I love the long scroll feature. How did you do that? It's so cool how you can scroll down the page natively, and the image updates, that's really incredible.
And I'm reading the initial discussion, and it's ... in 2015! Wow, how did you do this back then! I think thing's are so much easier now with all the features in the protocol.
I am really interested in how you did this and I love the site. It's very cool and I prefer it to my own work in many ways. Would you be a terrible idea for you to contribute to BrowserGap?
About half way through development, I was travelling and buying 4G data sims and I also thought I needed to use it for that (easily use 50Mb just on a news site).
So I made a HTML only version (no images, just stripped back HTML, you can see the work in the various 'appminifier' subdirectories somewhere in the repo). It saved me data, but introduced lots of quirks. At some point I realized it was too difficult, and I was committed to another idea with it, rather than this low bandwidth, so I stopped working on that feature.
Also, I love the Open in new tab? feature you have. This really rocks. It made me so happy to see this work! Thank you so much for sharing with me. :)
I wanted to get a scrolling feature like you have and I couldn't think of a way to make it work. If yow could do that in BG I'd love it!
Liked your product from your previous submission and liked this one as well. I think it can help some people where censorship is present, but not particularly for me.
My target audience is software developers, QA engineers, and Ops people who want a predictable isolated browser environment for doing various forms of testing / hacking.
Am I missing something?
I don't know if you're missing anything but this feedback about files is on point. I really appreciate it. And I'm surprised no one raise this until now. Thank you for your time thinking about this and for making the space for me to speak about it.
> Users need to download files, open them with local apps, upload local files.
Ideally, user's don't download files, they use the Secure remote file viewer (which currently handles PDFs, DOC/X, XLSX, etc), so that helps with exploits from there (such as the Chrome zero day from PDFium that recently occurred). No configuration is required, it automatically jumps in whenever a download starts.
Also, because the browser is running in the cloud, that "download" actually only happens between the web and the cloud. The file literally goes down to a temporary directory on a server in the cloud, before being sent to the secure file viewer. That file never touches the client's device or network. And the secure file viewer only sends pixels to the client, because it converts all documents to images, and then, the browser sends a screenshot of that page. So it's like... two layers of images.
Anyway, that helps mitigate the RCE threat from exploited file objects, browser and device zero days. And no HTML,JS,CSS from the browsed page is ever sent to your device.
As for opening with local apps, that's debatable with things like G Suite and Office 365. But we can integrate with a corporation's SWG (secure web gateway) and file policy so BG doesn't degrade their existing security, but it does provide an extra layer.
As for uploading that is absolutely required, otherwise many things would be unusable. I don't pretend that BG provides any sort of malware or virus scanner (mainly because there is not download), but as for uploads, it's possible to integrate into an organizations' existing SWG technology to gate-keep content that leaves, and also white and black list accessible sites.
> Current malware codebase might get stuck with it, but it's a matter of time and adoption.
I agree that to some extent, security is an ongoing "arms race". But there seems to be limits to what malware can achieve through the exploitation of pixels sent to the device. It puts a big limit on their attack vectors.
It sounds like there's no point taking any steps, because malware can always find a way through. When you say something like this, I feel like I'm wasting my time talking security, because it looks like you'd never adopt a mitigation anyway.
> Other threats like clickjacking, cryptomining, phishing would just work as before.
That's a great point. I don't think this tool can prevent against social engineering threats like phishing, fraud and deception. It may even may them worse by allowing users to feel "more secure" and therefore act more rashly.
No tool provides perfect protection, but BG can reduce the attack surface and isolate and contain many threats away from the device and network of the client. In the case of clickjacking, older browsers can be vulnerable because of CSP headers, but with BG you always proxy through the latest chrome.
As for cryptomining that will simply not work well at all. We have monitoring software that puts hard limits on CPU, memory and bandwidth for each browser and each user. Please, go ahead and try it.
But I see a certain segment of small business users who have everything cloud-based, where this might take off.
I'm interested in what you said. Would it be a terrible idea for you to tell me some more about this certain segment of small business users?
They keep client PCs as "thin" (read "cheap") as possible. They don't have a "SWG" or "file/firewall policies" or anyone who can implement and enforce it. It's just stock desktop AV software perhaps with some initial tuning. And this resource-hungry beast is there only to scan incoming files for the ransomware-of-the-day, either downloaded from the Internet or copied from USB thumb drives. If they could deny users from downloading anything, disable thumb drives and drop AV entirely, they'd be much happier, especially that poor IT guy.
Hack one instance and get access to hundreds of users browsing the internet.
Even in this free demo, every user has their own browser process, with its own uid owner, and that OS uid has its own limited permissions.
At the same time, it's not an insignificant risk at all and you raise a very good point, which I'm surprised no one brought up before. Thank you for bringing it to everyone's attention.
An instance is a single point of failure, it's also less attack surface. To some extent, that's a tradeoff. Relative to all devices and network infrastructure in a typical company that access the public web, there's less attack surface if all web access funnels through a BG instance. On the other hand, it's a concentration of the risks into one place. My belief is that makes it easier to manage, and that the "gap" between the client infrastructure and devices and the cloud (through which only pixels, and a wire protocol of user intent pass), makes it more secure than accessing the public web directly.
Even tho it's a single point of attack, a compromise of a cloud machine, is not the same as a compromise of a device in a company intranet, or a mobile phone of someone in the company. In order to exploit the user's local machine or their organization's network, an attacker would still need to convert any instance access they had into access of a company device or network. This could happen through attack vectors in the pixels for the screen view (less likely) or through compromising the source code that serves the thin client (more likely). This is why monitoring of source code integrity is important. Open Source is an important part of that.
At the same time, in these free demo versions the browsers only exist for 10 minutes, and, exactly as you say, hundreds of strangers are all browsing from the same machine together.
TL;DR - It's a tradeoff of centralized infrastructure. There's less attack surface, but there's also a single point of failure.
Also, if you want to responsibly dislcose any security vulnerabilities you discover, please report to email@example.com and if you want I can acknowledge you here https://github.com/dosycorp/vulnerability-reports
https://ungleich.ch/u/blog/how-to-run-your-browser-in-the-cl... (which will ultimately lead you to https://guacamole.apache.org/)
Maybe this is the Google AI team trying to get more people to solv'em? ;-)
A workaround in the meantime is to enter a URL in the box instead.
Edit: Switched to DDG as default search provider for this. Back up at 21:24 PST.
Edit @23:00 PST: I've opened an issue with Google Cloud Support (the system is hosted on GCP even tho it is cloud-agnostic) and I don't expect they will be able to provide a resolution because this is probably the CAPTCHA behaving correctly.
BTW, it could be the AI team getting some more data, who knows? ;)
Somehow tho I think whatever we do is simply a drop in a bucket for them.
Two thing I noticed. You have to enter address including "http(s)" to avoid searching it in DDG. And more annoyingly I couldn't select text on web page.
As for text selection, the best I have so far is
right click to open context menu
select "Copy text from here"
And you should get a box pop up with text you can select. :)
Also it gives me a feeling that I'm not in control of what's happening on the screen. Could you please let me know how is this solution better (or more secure) than using remote desktop with disposable VMs?
Edit: the HK site is back up. https://hk.cloudbrowser.xyz
There's a couple of factors that could be playing into this. Primarily it's likely just the application itself. It is more slow, and less responsive than using a regular browser on your device.
The frame-rate is capped very low, the image quality is lower, and there's more lag to each interaction since it involves (at the very least) a WebSocket round trip and a screenshot.
Secondly, you could be affected by geography, which has a very significant effect. If you are close to the primary server (US East, Virginia) you'll have a faster more responsive experience.
In a few minutes I'll have the HK server (Asia Pacific) back up ( I was just resizing it down, it was seeing significantly less use than the US server), and if you're closer to that you can try there.
Also, the free demo has many caps (so as to control costs). I cap the outgoing bandwidth of each user to a very low 3Mbit/s, and I use multiple ways to cap CPU usage, including (in extreme cases) killing the process. All of this means that if the page you are using wants to eat a lot of CPU (happens sometimes) then the app will slow right down for you (to preserve resources for everyone else on the system).
I can say confidently that it is not about the number of users. We had More 100s at peak before and a single browser still felt snappy. So if you're getting slow down I think it is (to summarise), either:
- You are experiencing the app for the first time, it is different to using a normal browser, interactions are slower and more choppy (but page loads should be as fast or faster).
- The page you are browsing is hitting the resource monitoring and being downregulated.
- You are link-wise far from the server (which is often, but not always, related to geography).
If you're interested to give it another try I'm at another time I'm happy to arrange that. Would you be unwilling to leave your email at this form, and I can let you know a quieter time? Also, if you just email me at firstname.lastname@example.org and let me know your approximate location, I can set up a server near you and we can attempt to work out any leg issues still occurring.
First up, have you had any issues with site banning or CAPTCHA?
After I saw your question I wanted to know so I just tried signing into my LinkedIn from https://hk.cloudbrowser.xyz and I'll share my experience.
And first they sent a code to my email because "something seemed suspicious":
When I put in the code from my email I could sign in and it worked as usual.
I have noticed that everytime I land at https://bloomberg.com I get a CAPTCHA (1 only) and then I could read the site.
I opened a support ticket with BB but they said they don't need to do anything right now.
I felt OK with that. 1 CAPTCHA is not too bad.
Also, I noticed I spent a lot of time maintaining the demo instances (resizing). In a real deployment the number of users per machine is pretty much static, but here I've had to deal with scaling and spikes.
It occurred to me today that I could probably put the free demos behind a load balancer (smaller basic machines, and scale them up or down), so that I don't have to manually resize the instance.
I've taken down the two demo sites (free & hk) for now, while I work on the load balancer setup. Should be back up in a couple hours.
I moved from a single massive instance to a target pool behind a load balancer with health checks based on if there's available queue.
I have not worked out how to geographically load balance both of those from a single domain based on which you are closest too, but I want to see if these new smaller instances in load balanced target group pools can scale to take the load like a larger instance.
Namely, load balancing and scaling based on CPU is not a good metric, because new instances (which are still serving multiple users), will keep absorbing new users before the metric is triggered, and even when it is triggered, a new instance will take a while to spin up and build some browsers, so scaling lags too far behind load and the effect is existing instances get and stay overloaded.
So, I have an idea for a new autoscaling system that puts 1 tiny machine per user, but it will take some reconfiguring. So, in the meantime, I'm switching back to the old system (massive instances, vertical scale).
I'll do that now and the servers should switch to the new system in about 30 mins.
on the other hand I have been using novnc to do similar things for my own testing purposes(running chrome inside chrome remotely), it worked very well for me so far.
I'm very serious about usability issues, would it be impossible for you to provide some examples of sites where the click failed and what you clicked on?
I try to get those things fixed ASAP because the experience of using is so important, I think it should be as familiar to a regular browser as possible.
As it gets possible to build very large and powerful computers more cheaply, they'll still be expensive and only the biggest corps will deploy them. And the experience deliverable from virtualized apps in those clouds will be vastly better than anything that can be run on a device. Considering the impact and requirements of AI only makes this more likely.
So after briefly becoming all about "edge computing" and "fat clients" in the recent past, I think the future is going to see a swing back toward massive centralization. This will also only be compounded by the next advances in wafer process and quantum, and the increases in bandwidth to allow richer experiences over 5G.
Our pocket devices might be "supercomputers" but the real supercomputers are still going to be in the cloud and capable, I think, of running everything.
TL;DR - the next big advance in computing tech will disproportionately allocate towards the cloud rather than the device, is my bet. And we'll all have "dumb" terminals streaming us VR/AR + AI experiences all the time.
Disclaimer: that future is not why I made this, I still think it is some time off and it just seems obvious this will happen. Rather than becoming all about "more powerful devices, less powerful servers", I think the future will be the other way round. :)
Imagine instead of google docs you had microsoft excel but surrounded by a browser window, everything else is the same. I know people would pay for that. (even if I personally prefer google docs)
> self-host on your own machine (at home, or in a VPS, VPC or the public cloud)
is more secure/private then just setting up a VPN on that machine?
So this is nothing to do with a VPN as such, but of course you could host it in the cloud, or run a VPN to a cloud endpoint.
What this author is providing is a similar and all-in-one npm solution for the above. Also has use-cases outside of secure runtime environments...
I'd like to ask for your advice, I'm at email@example.com .... Would you be uninterested in discussing over email?
Well, I'm hearing you right, then this (BG) is definitely a potential solution for your problem. I think it would be lighter weight than Guac since it only runs a browser (not a whole desktop).
My email is firstname.lastname@example.org ... Would you be unwilling to talk more?
After Screenshots are queued, they're throttled to a low framerate, and then each frame is compared with the last sent, before it is sent to the client, and dropped if there's no change.
I got rid of CAPTCHA by changing the search provider to DDG.
Thank you for your comments, I really appreciate it! :)
I have not thought about Flash, but I totally get the point you make. It looks like it would be very useful as they are big attack vectors and usually not supported on mobile.
Thank you for this feedback.
If an attacker has a browser exploit, they can still..
- read all your email
- log all your passwords
- wait for you to login to online banking and transfer all the money.
You also have to worry about the host being compromised, or the operator logging your activity, etc.
If this is something you really want for some reason, why not just RDP/VNC/Chrome Remote Desktop back to your office/home network.
I am in no way endorsing this product (or the category of products), but it is incorrect to say that it doesn't protect you from exploits.
It protects against your machine being compromised, and while the browser VM can be compromised, it can theoretically be ephemeral, possibly even only having the lifetime of a single tab (some products in the space does this IIRC).
It's all just a matter of what your threat model is.
> If this is something you really want for some reason, why not just RDP/VNC/Chrome Remote Desktop back to your office/home network.
You could make a minimal linux VM with only a web browser, keep good discipline and never do anything else on it, and RDP into it.
However, these products commonly provide much smaller attack surface than such a setup, and combined with something like VM-based tab isolation, you wouldn't be anywhere near the security features of such a product.
For concerns regarding what you say, it's true. The worst case is a browser zero day that escapes the browser, enables privilege escalation, and goes on to compromise the entire client instance and lurks in the server catching everything. That's definitely a possibility on the free demo.
However, for real deployments there are several layers of mitigation. On the most drastic end, we run each browser and server pair inside its own docker container. In that case, the exploit must, first escape the browser -> escape the Docker container -> gain code execution -> gain privilege escalation -> compromise whole instance.
I would never say it's "impossible" but I would say that we have brought effective defenses against that.
Another layer of mitigation is that each browser is totally scrubbed each time it is used. This is a tradeoff as we lose all session cookies and cache data, but we can do this, if required. You can even scrub it yourself while using it (right click/top hold to get the context menu, and select "Wipe everything".
A further layer of mitigation is we completely reprovision / scrub the entire server every hour / day / week.
I think you are getting the idea.
By separating the client's devices and network from DMZ where "web work" occurs, and containing the attacks within this "air gapped" satellite, we greatly limit the attacker's scope to cause harm, plus we greatly increase our ability to deliver mitigation at any scale and schedule we choose.
Another advantage of this is, you don't need to download anything, RDP/VNC requires some extra app or setup, this is just connect to a web site and you're good to go.
You can self host to manage trust. Organizations typically want to partner with a provider even if they do self-host or go hybrid, because they don't want to manage everything themselves.
Thanks for the message. I like browserless!
I don't use puppeteer. I use Chrome DevTools Protocol heavily tho. I started using chrome-remote-interface but hit limits in what it can do with Targets (specifically, flat session mode) and the latest versions of the API. Now I just use the WebSocket directly.
I'd like to partner. Email me email@example.com
Thinking back to when I started this I initially just wanted to keep everything simple and so I avoided putting in a large and high-level lib like pptr, and went with chrome-remote-interface.
I looked at pptr and IIRC at that time (~ 12 months ago) there was not a clear way for me to handle multiple tabs (a key "real UI" use case). The same goes for Cyrus' lib too.
With Cryus' lower level lib I could hack around that, by doing my own target and session management, but at some point in the last couple months I hit a wall with chrome-remote-interface. Cyrus' lib was not up to date with the latest ToT API (specifically flat session mode) and I worked out I could replace the entirety of chrome-remote-interface with some simple code that sent messages down a WebSocket, saved a Promise (by message id) and returned it, and resolved that promise when it received back a message tagged by corresponding id. It was also simple to write an 'on' function to add listeners for various events. So that was that.
Basically, the DevTools protocol is a well specced, well tested, simple protocol and all these libs (like pptr and chrome-remote-interface) began simply as wrappers around the WebSocket, with an API to map function calls to protocol messages and add listeners for events. PPTR has evolved into much more than that now, and during the same time period, I evolved my own "BG protocol" atop the CDTP (Chrome DevTools Protocol). It became easier to deal with the single source of truth that CDTP is, and get the full expressibility of the latest ToT protocol than deal with the limitations and abstractions of other things built atop that.
Specifically, PPTR did not (and I believe probably still does not, tho I have not deeply checked) an easy way to control and manage multiple tabs. And even if it does, I'd have no use for it, because I already have the code that does all that anyway. Scanning PPTR docs now I see that I prefer the abstractions, naming, etc of the CDTP protocol itself, rather than the ones PPTR provides. Like I said, the CDTP protocol is very comprehensive, consistent and makes a lot of sense, and I know it very well. For me and my use case, it's just a better fit.
The way I think about this is not that "PPTR" has some problem, it's that the "BG protocol" and PPTR (et al) are trying to solve (basically) fundamentally different problems. PPTR (et al) try to provide a clean developer experience for common tasks related to browser use cases (such as automation, getting screenshots, PDFs, testing, etc). That's a particular domain, and not exactly the same as what BG protocol does. BG protocol attempts to provide as realistic and familiar as possible experience of using a browser (when you're actually controlling a remote browser through the CDTP). That's not entirely the same domain, because some things that users want, are not required in automation, and some things that automation does are not required or done by users.
One of the ways I code is by picking the right tool for the job, and if that tool doesn't exist, or no longer works, I build the tool. I want to work with tools that fit right. So for this domain and use case BG protocol is a better fit than PPTR.
pptr and BG both use Chrome DevTools to communicate with the browser, so there's that commonality.
I haven't looked extensively at the pptr source but I imagine both pptr and BG do some bookkeeping of state related to the sequence of commands (BG certainly does), rather than a purely "stateless" command queue.
For instance, for some things you need to keep track of which session is associated with which target. For other command sequences (such as Runtime.evaluate) you need to know which execution context to evaluate in. And you can keep track of open execution contexts by tracking various Runtime domain events (such as executionContextCreated, executionContextDestroyed, etc).
So to provide a sophisticated level of interaction with the page, some amount of chattiness and state is required for the protocol on top of the DevTools wire protocol.
What I'm getting at is, if there are hurdles, they will likely emerge from the different ways in which pptr and BG handle state and that chattiness to achieve particular user intents.
Also, BG does not require pptr to do automation. It's possible to simply record and replay (again with a further layer of book-keeping that's too involved to get into here), the BG command sequence (which itself is a superset of the DevTools protocol).
All the same, I've often thought about providing a functionality to "export to puppeteer" (or to nightmare, or to phantomjs, etc) in terms of getting a transcript in a widely used format that people can take and run anywhere. That's one thing that excites me about pptr X BG.
In any case, automation is not something that is currently provided in the CE, but it's in the paid version at https://browsergap.xyz
That seems less secure than facing the risk of any exploits being able to escape the hypervisor in the cloud, someohow come through the text protocol connection to your computer, and exploit you there.
The extra layer of security provided by the remote cloud is important.
If, on the other hand, you mean using Guacamole to connect to a desktop running headless in the cloud, I'd say that's a similar level of security to BrowserGap.
It's a big oversight, I'm sorry about that. Let me think about it.
There must be a way to do this.
But, haha, I'm really glad that little modal dialog bubbled up the layers! XD It's kind of crazy when you think about what's happening ~~ you're chaining 7 browsers in row, using each to automate the next. It's crazy! Haha! XD
You really got me with this, it's so funny! Thanks for giving me a huge smile.
Saved = (total resource weight of original browsing) - (screenshots + protocol)
Things like privacy extensions, screen readers, crawlers have various user agents.
At the same time, you make a good point. I'll consider your reflection if I come to re-decide if this is a sensible default.
If you do this, please post an issue or PR on the REPO. I'd love to include something cool like this in a "built with BG" section.
I don't have an iPhone next to me right now, and I have not tested in Safari for about a week. Would you mind sharing a screenshot?
I know that because iOS Safari does not support WebP, I'm just using JPEGs which means the quality is worse there.
As a short term workaround, I'll now turn up the JPEG quality. This will take some time to propagate to browsers.
Again, I'm very sorry you had this experience today! I will test on iOS and improve the usability. I've had other reports that scrolling is terrible in iOS.
I'm just adding the iOS issues to the GitHub repo now. It could be great to contribute your screenshot to there! :)
Edit: Also, I'm sorry but would you be able to tell me are you using https://free.cloudbrowser.xyz or https://hk.cloudbrowser.xyz?
Because I am testing serving HK with imports rolledup for speed (less requests), and wonder if this caused a further issue.
Sorry I don't have a random email, but here you go: firstname.lastname@example.org
Can anyone explain the purpose of Google putting up an apparently impossible barrier like that? Is it because my searches are comingled with dozens of other people attempting Google searches at the same time from the same IP address? Or Google decided to blacklist his IP address?
Are people really that eager to send google all traffic information? Is this really the best solution? Couldn't one for example parse the actual content that the bots are trying to create and filter/ban them based on that? I believe spam detection is quite effective these days.
This whole captcha hell is making web really annoying for normal users and mostly just benefits google.
It really fucks with you when you're on TOR, especially.
Looks like it happens not only in that case.
Also, that's terrible! I'm so sorry for you that you had that experience today. You must feel pretty angry to sit through 15 CAPTCHAs!
I have to solve the CAPTCHA problem. I'm sorry I can't explain why this happens or how the CAPTCHA system works. I don't know anything about it.
I'm OKAY with the occasional CAPTCHA, but I'm not satisfied that people are getting hit like this.
I've been told that VPNs have a solution to this (since they need to deal with their clients getting faced with CAPTCHAs as well) but I'm sorry I have not reached out to any VPNs yet about this because, last time this problem occurred, after an hour or two I checked and it stopped happening, and there were no more CAPTCHAs. So last time I did not have to change the default search provider to DDG.
I'll send an email to NordVPN now and ask them if they have any idea what I can do.
Just kiddin'. Sometimes, I end up having to do unreasonably more captchas. The reason is that the system verified I am human, and it wants me to cooperate with it for free in order to authenticate more captchas. Machine learning, they call it. I call it un(der)paid labour. Never had 15 in a row though, so I suppose the already mentioned reason is more plausible.
Just try doing a reverse phone number search on truecaller.com. It will work flawlessly on Chrome but use FF or any other browser and recaptcha will fail you saying you're a bot (at least happens to me evey time I've tried). Same for many other sites where you see that "Checking your browser" page.
So even though I want to use FF I can't since most of the web is unusable because of this stupid Recaptcha unlesw you're using Google Chrome :(
Google probably identified you as computator prior to the capcha but that profile had no value for seemingly impossibly.
>Can anyone explain the purpose of Google putting up an apparently impossible barrier like that?
Your video will play automatically (after these fourteen short messages from our sponsors)