I ran the servers for our networks, binary exploitation, and intro systems programming classes, and this thing is a major annoyance. It is because of this stupid RAT that students do not understand how to use the OpenSSH client.
I've tried a few things to fix this:
1., I set a motd on our class servers asking the students not to use the VSCode remote server plugin.
2. I ran `ncdu /home` in front of the class and demonstrated how, without exception, everyone with >100MB of disk usage on the class server was a VSCode user.
3. I set a user process limit of 45, because the VSCode RAT (somehow) uses ~50 Node processes. When students inevitably ignored the motd and the in-class warning, they hit the process cap and had to ask us to kill their processes in order to get back onto the system.
4. I replaced the process limit with a script that kills all the .vscode-server RATs every 10 seconds.
Your comment is bringing back so many memories of being in university and working around the anachronistically strict limits the uni sysadmins had on the network
Ah yes, one of my favorite skills learned in school was how to work around arbitrary rules made up by cranky sysadmins! I still use it all the time.
The smart kids (whom I’m sure look like they are learning that precious openssh client) are doing their assignment locally or on a free tier VPS with VS Code and scping the thing over when it’s done.
They’re also smart enough to learn openssh when they need it IRL.
In the class which I have managed the infrastructure for, we have 1 to 2 students per VM, so this is less of an issue and there aren't any restrictions.
One of the reasons we provide the VMs is so that students can experience working in a remote server environment. The concern that I have is that these remote ssh tools allow you to bypass learning/practicing how to perform basic actions, e.g. cd, read/edit files.
Granted, as mentioned, you can scp/rsync (or git pull), but at least this seems to be more appropriate when you eventually need to interact with a real production server.
This isn't something specific to VSCode being popular. When I was in college over a decade ago, there were students using Sublime with an SFTP plugin, coding locally and copying stuff with FileZilla or other similar GUI clients to transfer files (I distinctly remember seeing some program that was named something duck-related but for the life of me I can't remember any other specifics). Sublime in particular could be frustrating in the class I TA'd where they had some assignments to deal with processing some (very basic) machine code to simulate running the tool the class used to assemble and run the code from the (similarly basic) ISA they had been taught, which involved them using hexdump or something similar to understand how the bytes in the files worked, but Sublime "helpfully" would render the object files as as their text representation of the hexdump, with extra spacing for readability and swapped endianness compared to the way the bytes would show up if hexdumped on the school's Linux server. There would always be several students every semester who would show up at office hours unable to figure out why their code that was written to try to read an ASCII string like "AD DE EF BE " but instead would instead find some text they didn't recognize because they didn't think to look at the actual byte values, which would always just happen to start with 0xDE, 0xAD, 0xBE, and 0xEF.
The issue is not about live editing on servers, it’s about running a process out on the server that spawns,according to GP, 50-some processes. VS Code remote editing and Sublime/any SFTP/SCP-style deploy are on completely different levels.
I was responding directly to the parent comment's decrying of VS Code as responsible for why their students don't know how to use ssh clients, not the general issue, which I agree is not specifically about live editing.
Off-topic: just one look at that homepage, the icons in particular, brought back such fond memories of the pre-flat design time. Those halcyon days when it was normal for buttons to look like buttons. (Maybe I'm just using the wrong apps these days.)
Can you explain why though? I get that you had to put a lot of effort into blocking VSCode but it's not clear what have VSCode caused to motivate this.
It sounds like they’re operating at larger scales, with a large number of users sharing a resource. Each user spawns 50 processes and consumes 100MB of space if they’re using VSCode. Assuming a low count of 1000 users, that’s 50K running processes and 100GB of space consumed. To enable a text editor.
100GB of storage is… not much for 1000 users. A 1TB NVMe SSD is $60. So 100GB is a total of $6 of storage… or about half a penny per user.
And that’s for SSD storage… an enterprise-grade 14TB hard drive is only $18/TB on Amazon right now, less than a third of the SSD price per TB. Call it 100GB = $2 of storage, total, to enable 1000 users to run the editor of their choice.
So, no, I’m not seeing the problem here.
If you really wanted to penny pinch (one whole penny for every 5 users), I think you could use btrfs deduplication to reduce the storage used.
Very apparently you're counting as a singular physical person. In a big organization there's always an overhead both in time, and money, and more importantly there are more problems to solve than there are resources. So one has to arrange priorities, and just keep low-prio things in check not letting to boil over the lid.
Preventing people from using their preferred tools — tools which are extremely widely used in the real world — does not seem like a useful application of time and effort.
Usefulness depends on conditions of which we don't know a lot here. Sure, there are situations when counteracting pressure is more expensive than expanding capacity. But frankly I doubt this particular case is one of those.
How many concurrent users can you run off a single NVMe SSD?
How many students leave their coursework to the last minute?
How do you explain that the server went down during the last hour before submission deadline again, and that everyone gets an extension again, because you cheaped out on putting the cheapest possible storage into a system that has to cope with large demand at peak times?
How many students now start to do worse because of the anxiety caused by these repeated outages?
How much more needs to be invested in the university counselling services to account for this uptick in students struggling?
No… it’s not. To quote the message earlier in the thread, that message said “everyone with >100MB of disk usage on the class server was a VSCode user.”
100MB * 1000 users is how the person I responded to calculated 100GB, which is storage.
Most of the RAM usage would likely just be executable files that are mmap’d from disk.. not “real” RAM usage. But, also, the 1000 users in question wouldn’t all be connected at the same time… and I honestly doubt they would all be assigned to the same server for practical reasons anyways.
It’s not easy to estimate the real RAM usage with back of the napkin math.
Depending on what they're doing, it could easily be multiple Gb per user. When you do VSCode remoting, pretty much everything but the UI is running on the server. This includes stuff like code analysis for autocompletion, which - especially for languages that require type inference to provide useful completions - can consume a lot of RAM, and a fair bit of CPU.
> I honestly doubt they would all be assigned to the same server for practical reasons anyways.
The computer science department at my university had multiple servers. All CS students got an account on the one same server by default. Access was granted to other servers on a case by case basis, based on very course-specific needs.
So yes, in my case, all CS undergrads used the same one server.
> 1., I set a motd on our class servers asking the students not to use the VSCode remote server plugin. ...When students inevitably ignored the motd...
The provider of the servers? If the students want to use their own server, I’m sure they’re free to do whatever they want. Also VS code has plug-ins that just SFTP the files up.
We encourage students to use their own machines. We even loan out old ThinkPads and MacBooks for students to run Linux on. Very few (3-5 per term) take us up on this offer. Most of our students would rather we do the sysadmin stuff for them.
The only requirement is that the code needs to run on our server, which they can easily check before submission.
You sound like a painful teacher. Who cares if the students are using vscode+remote? Are they learning the material? Are they building good software? This is uni for christ sake, not an apollo mission. The entire purpose of a university environment is to foster learning and development, to open up a safe space for learning. Not to suffocate them with arbitrary and excessive rules.
Things need to be introduced at the right gradient. If you pile it on all at once, nothing is learned. If you want to teach someone how to use SSH to interact with a server, then that should be a dedicated chapter or portion of the curriculum. Once you are beyond that and it comes down to building programs, you need to eliminate that part of the gradient and focus 100% on mastering the task at hand.
My experience participating in Google's program has been pretty good. The reward money is a nice supplement to my grad student stipend. I got a free trip to DEFCON out of it, too.
Yeah, I should really rewrite the README to bucket bugs by behavior instead of server; there's a decent amount of redundancy in the set of discovered bugs.
This makes sense. Nginx is reliable and well-tested, and OLS is neither.
See my recent Shmoocon talk (yes yes, self promotion) for an example of a vulnerability that would be caught immediately in Nginx, but persisted in the OLS codebase for >=10 years: https://yewtu.be/watch?v=aKPAX00ft5s&t=2h29m28s
I've spent the better part of the year messing around with every HTTP server I could get my hands on. I would not recommend OpenBSD httpd.
It's supposed to be simple, and it is simple compared to Apache, but it also has way fewer eyes on it. Ultimately, a big pile of string-handling C is likely to have some problems. There was a trivial-to-exploit server-crashing segfault in httpd's FastCGI implementation that was only fixed in the last month or so. There were also recent issues related to null bytes and line feeds in headers causing strange, exploitable misinterpretations of incoming requests in relayd. Much of this is now fixed, but I wouldn't be at all surprised if there were more low-hanging fruit remaining.
If you're looking for a lightweight HTTP server written in C, I recommend Lighttpd. It's older, more widely-used, and more standards-compliant. I'm not trying to dump on OpenBSD; I run it on my primary laptop. I just wouldn't use their HTTP tools for anything mission-critical yet.
No HTTP server software can be guaranteed to be completely secure, but OpenBSD httpd is at least privsep, always runs in a chroot, and each process is pledged quite tightly, that means http/tls protocol speakers can't write to the filesystem, can't fork any processes, and can't execve(2).
Surely the most secure would have to be a server written in a memory safe language? While you can do stupid things in any language, I would jump to Caddy (Go) or a Rust based toolkit before anything written in C.
You can do stupid things with "memory safe" languages too, and nothing protects you from misconfiguration. The bsd approach of just locking it down seems to be to be at least as secure if not more to me.
> I love how having a couple of new languages make older, hardened programs insecure in an instant.
As someone who has been dealing with Unix since the early 90s, most of those old C programs were always security nightmares. There were a few exceptions: djb's stuff, dovecot and (surprisingly) Apache. But most of the other popular C servers were absolutely riddled with buffer overflows and other security problems. The Morris worm, the entire rsh family, you name it. Sendmail was awful for a long time, too.
Memory corruption errors still make up 70% of security holes in modern C and C++ code, and that's after decades of improvement. That entire set of vulnerabilities could be avoided in the 90s by using Perl 5 or Java, which were insanely popular. (PHP came with its own supply of security holes, both in the interpreter, the language design, and the awful database APIs.)
We've had popular memory-safe options for decades. All Rust brings to the table is the ability to be memory safe without needing GC (which is great). But with rare exceptions, the popular C servers were always full of holes.
There's no reason you couldn't also pledge an HTTP server written in a safe language too.
One key thing to keep in mind is that exploit mitigations are largely about limiting degrees of freedom. It's quite hard to contain an attacker who has gained code execution in a process (such as due to memory corruption) due to the massive amount of control they have. On the other hand, if you're able to prevent them from gaining such a strong foothold to begin with (such as by using safer languages which don't have such issues), you're in a much better spot because now they don't even gain any control over the process.
They have become insecure not because of new languages but because web is no longer just serving a bunch of files. You are delivering executable text mixed with untrusted input that must not improperly interfere with other users and for that you need impeccable string handling. Just reliably segfaulting every time there's a problem won't do.
Because they used to have Apache in base from 1998 to 2014 (the 5.6 release) and found it frustrating to maintain? Given the developers and what they do with the operating system, I can understand why they would consider having an HTTP server in base something within the scope of the project.
It is an open source project with both commits and discussion happening in the open (tech@ in particular), so the barrier to understanding is reasonably acceptable. At some point it was also debated to move to nginx, but in the end httpd(8) was championed as a natural offshoot from relayd(8) which was already in base.
You are indeed correct. I recalled that the nginx migration was in progress until around the time of 5.4, but did not remember that it was in base at the time. Thank you for correcting me, it was indeed in tree from 2011 to 2014.
OpenBSD maintained several local patches to nginx, such as chroot [0] by default and reallocarray fixes, but it was rejected by the upstream and too big to maintain locally.
OpenBSD httpd would probably have never existed if nginx upstream reacted differently.
For the same reason that CVS is included in base: it's used by the developers for the continued development of the project. OpenBSD.org runs on httpd(8).
I've tried a few things to fix this: 1., I set a motd on our class servers asking the students not to use the VSCode remote server plugin. 2. I ran `ncdu /home` in front of the class and demonstrated how, without exception, everyone with >100MB of disk usage on the class server was a VSCode user. 3. I set a user process limit of 45, because the VSCode RAT (somehow) uses ~50 Node processes. When students inevitably ignored the motd and the in-class warning, they hit the process cap and had to ask us to kill their processes in order to get back onto the system. 4. I replaced the process limit with a script that kills all the .vscode-server RATs every 10 seconds.