I'd like to take this opportunity to highlight BSDPAN, which is how FreeBSD integrates her own package database with Perl's native module installation.
Since 99% of package management is just files on a filesystem and a bit of dependency analysis, for the purposes of easing installation, permitting bidirectional awareness of state, and alerting administrators to security updates.
I would commend any OS that has the smarts to hook into the package ecosystems of her guests. RubyGems, CPAN, npm, PEAR, PyPI into APT, RPM and what have you. Wouldn't it be great if.
Here's an edge case, though. In the specific world of both Ruby and her fat offspring Rails, the proliferation of versions (and the widespread separation of sysadmins from developers) means that in practice many Ruby applications have the runtime language binaries and package dependencies installed in app-specific or personal home directories, via the likes of rvm. Stick that in your package management pipe and solve it.
I know this is a bit of a me too post but this is something I have thought should exist for years now.
Another benefit this would provide would be that you'd only have to learn one set of incantations rather than forgetting and looking up whatever subsystem it is you're messing with.
For the life of me I don't know why this does not exist yet. I have such a desire for it that I thought long and hard about building something like this myself but balked at the prospect, for yea it is daunting.
There are others as well: LaTeX (TeXLive) has a package system too, and I am sure there are lots more.
There are workflow issues to generalising the case. In particular, since this is a devops issue any such solution has to be based on satisfying the needs of developers (e.g. ease of deployment, version specificity), and the needs of admins (ease of patching & audit, dependency management) across a wide variety of packaging mechanisms for both base OS and many language-specific ecosystems.
It's no surprise to me that the unit of deployment, for many sites, is becoming the virtual machine template. Since you don't even need to bother maintaining it; just keep your data separate from your code, rebuild the image when necessary with latest libs & pkgs, and throw away old ones. This is how many PaaS providers are doing things.
Wise decision. Ruby has it's own package system. I believe every developer using rvm/rbenv for managing ruby versions. Rails should be installed using rubygems. I always cry seeing pretty outdated rails packages in distributives. I don't care.
Unfortunately this leads to language binaries & libs being distributed by the developer with the application. Sysadmins then install the whole massive hairball in /opt as a single unit and never update it until the developer provides a point release. Which never happens because the dev team has been disbanded, they're all on new projects now.
If you don't think that's a problem in the real world, you ain't seen 95% of enterprise IT.
I appreciate your butt-kicking energy! However, I must note: configuration idempotency tools that regard infrastructure as code are not new. I think you'll find that the inadequacy of cfengine, puppet & chef to the task is why they are not universally used. Devops is a very young discipline and these tools are the stepping stones toward a more coherent model of component deployment. I'm not saying they're crap, just that we can and will do better. Chef is probably the best effort so far, although I am unconvinced by the client/server variant.
In particular, none of them yet solve the problems of client application deployment. They are also implicated in the VM sprawl many cloud users experience.
Finally, they have convinced some developers that systems administration is something they can do themselves. Experience consistently proves this wrong; I've lost count of developers out of their depth, asking for rescue from suboptimal deployment decisions and needing rescuing. Either that or they discover a penchant for systems admin as well, and thus become devops people (this is what happened to me). I've seen the latter happen most often in the ISP world; perhaps network engineers have that mindset.
Personally I foresee a stronger role for packages as an output from continuous integration pipelines (another young discipline).
Ah, so if by client you mean end-user devices ... yeah. I am not sure Chef or Puppet solves it. On the other hand, my world has been focused on web and mobile apps, so end-user apps tend to use browsers that are already on the end-user device.
I'm not sure what you mean by clients though and how that relates to VM sprawl.
It's true that some developers are convinced that sys admin is something they can do themselves. However, it goes the other way around too, that sys admins think they know what is best for the app. It's better all around if devs actually gain an understanding at the issues with sys admin, and sys admins get some development skills.
As far as the server-side goes, once nice thing about using something like Chef or Puppet is that you can create a miniature version of the production environment on the developer's laptop. You won't run into the same kind of issues when you are scaling out, but you can at least make sure that you are running everything rather than always developing things in isolation.
Can you tell us more about this problem of client application deployment, please? I work with CFEngine quite a bit and it can deploy server and client components. What issues are you running into?
Yes. Maintaining a canonical set of ports for frequently changing language-managed packages is like getting into a phonebook reprinting business. It would be a far better use of precious volunteer time for any given language package to push OS specific changes to that upstream rather than grow downstream collections. The *BSD community could also learn something from the arch AUP as a way to end-users to roll their own packages to make it easier to contribute and reduce their own deployed systems' entropy ... because having a singular ports collection is like SVN or top-down planned economies... they are labor-intensive to support and their decision making is distantly divorced from the intended use-case. Different from AUP would be human-sensible configuration management that doesn't require manual intervention or too much tweaking for deploying 1k+ nodes.
Yep. I don't know why anyone would want to install something like Rails, Django etc... via their distribution or OS's built-in package management system. I think it's silly actually.
Ruby has rubygems, Python has pip, Perl has cpan...
As somebody who occasionally uses software, besides just developing it, learning the command syntax and idiosyncrasies of a half dozen package systems is a pain in the butt. I want to type "pkg_add things I want" and be done with it.
Not to mention that package management isn't a problem that should be solved at every single level possible - package management for the OS, package management for developing in $LANGUAGE, package management for extensions of $PROGRAM written in $LANGUAGE, etc....
Aside from the fact that that's redundant, it's highly error-prone.
As a long time Rails dev, I never trust the package maintainer's OS packages for Rails. I need specific versions of Rails and gems, often times specific git commits.
In places where the IT staff insists on making sure this isn't "redundant", they usually use something like fpm to create a deb or an rpm out of the gem specified by Gemfile. Which you then have to drop into a custom deb/rpm repository. On top of that, the ops staff don't know how to do that properly, so they end up with a bunch of testing and development dependencies that the Rails application doesn't need.
... or, you have the deploy process call, "bundle install"
The redundancy is in the OS packages, not in application dependencies. So it is actually better to automate such with something like Chef and Puppet. From there, I can define exactly what the application needs from the OS.
I have seen this play out in a lot of ways, devs vs. sys admin. In the past, sys admins "win" since compute resources were still scarce. We're in the age of virtual machines now and configuration automation. Applications get their own VMs, sometimes in multiple nodes. We don't have Unix high priests guarding the altar with SSH keys, hand-deploying things in secret early-morning ceremonies. We have automation to deploy for us, which means we want to make it easier for the automation to deploy, not necessarily for someone to shell in and maneuver around with hand installations.
Things being application-centric, the "right" thing is to conform to the requirements of the dev team's app, not the other way around.
Therefore 'pkg_add' needs to be able to hook into 'gem install' and 'gem list' and save you (and me) the bother. And this needs to be solved for apt and yum and homebrew and so on as well. I don't know, something needs to be done definitely.
I've been messing around with FreeBSD for a few days and it has RubyGems set up very conveniently. Root installed packages (like the ones from ports) would already be attended through the root installed RubyGem.
Aside from OpenBSD which has their security procedures for installed software it doesn't seem like any ports distro really need to have copies of gems in their package tree if the RubyGems package is set up correctly. They could delegate that function to RubyGems, it'd be transparent to the user.
Some package managers go down a more drastic route and simply tell the user to install the dependency through the correct package manager (Homebrew does that with Node, top of my mind). Users can still do it locally with rbenv/rvm/chruby if they want, it doesn't change much.
The bigger I'd think is that they don't want to rely on external package managers for the stability of their system. Which is a very valid reason, if RubyGems is out you can't install certain ports. Or worse, the gems aren't safe. The system is then self-contained and tested, proven to work with the provided copies of gems.
But as a simple dependency resolution perspective I don't see it being the case.
I can tell you that deploying a Rails app needs to come from using rubygem and bundler, never the OS's package management.
As for security concerns: you can and should vet gems, but it won't matter if the application itself has vulnerabilities.
We don't have very many Rails app written as a packaged system for deployment. The kind you see are often SAAS apps or something to interact with people internally. They are often bespoke and written for specific needs.
Non-OS installation is a huge barrier to new users of a Ruby application. I installed Gitlab recently. Take a look at the length of the installation guide [1]. There are seven numbered steps, some of which involve multiple commands.
Upgrading from 4.0 to 4.1 was about the same complexity.
Thanks to the thoroughness of its author, the guide largely worked as advertised. But being able to install and upgrade with apt-get would have saved me hours.
It is the case for now, but won't be much for longer. I expect application developers to start versioning Vagrantfile, Berkfile, etc. so that someone can bring up at least dev versions of apps.
This is encouraging to see. If a given port isn't being maintained, and its security is haphazard to begin with, removing it is a very prudent course of action.
While I know I can't trust Ruby and the Rails communities to do the right thing, I know with much more certainty that I can rely on the OpenBSD developers to.
> While I know I can't trust Ruby and the Rails communities to do the right thing, I know with much more certainty that I can rely on the OpenBSD developers to.
I find this intriguing. It seems to me that all of the recent Rails security issues have been communicated and patched quickly by the Ruby and Rails communities, while the ports maintained by the OpenBSD developers remained out of date and insecure, which argues for the opposite conclusion to the one you have drawn. But I don't think it's any knock on the OpenBSD developers, keeping up with patches in a fast-moving project like Rails is a losing proposition, so they're absolutely right to remove the ports and cede that maintenance responsibility.
In the OpenBSD world, security isn't something that comes later via an endless stream of patches, like it does within the Ruby community.
Security is done proactively in the OpenBSD realm. Care is taken to develop software that's secure from the very beginning, with security-related patches being a rare occurrence later on in the extreme case that something was accidentally overlooked.
The Ruby and Ruby on Rails way is incompatible with the OpenBSD philosophy. Were Ruby, Rails and related software developed properly, there wouldn't be the need for constant hand-holding from the OpenBSD package maintainers. I don't think that the OpenBSD developers should be held responsible in any way for the negligence of the Ruby community.
Getting rid of these questionable ports is a good example of the proactive approach to security taken by OpenBSD. Constantly patching low-quality software is not the correct way of dealing with the situation. Essentially getting rid of this code is the correct approach, and that's why it is good to see the OpenBSD developers following that path.
If you're going to follow that rationale to it's logical conclusion -- that software not adhering to the OpenBSD philosophy of security first, bar none, be excluded from ports -- then there are a lot of ports that should be removed.
I'm not defending the Ruby/Rails/Rubygems community here. The problems we're facing are a result of decisions to ignore important security concerns when designing software. I'm just don't like to see people piling on. I think this is a revelation for the Ruby community. Rubygems is not just some package, it is the primary package source. This incident was as far reaching as it gets in the Ruby world. No one is claiming any different.
It's also worth pointing out that the Ruby community aren't alone. This doesn't make the decisions right, it just makes it easier to understand the context in which they were made. I don't know how much progress the Python community has made, but they're facing similar challenges:
I realize this response is a bit late. However, it's worth mentioning that there's been quite a bit of movement here from the Python community in the past two weeks. No doubt this is a response to what happened with Ruby. A proper cert for pypi.python.org is being rolled out this week and pip should shortly have cert checking.
Even more than the security, the reason I use it at home is because I'm lazy and I don't want to go hopping about applying patches to lock things down. Most things take the least amount of effort to configure and, probably most importantly, things are predictable.
There's no "magic", everything must be clear, documented and open.
An old friend of mine also runs OpenBSD on his machine and I don't think he's restarted in 2 years. Granted, he's running ancient software, but it works, he's using sane configs so it's secure, although he hasn't taken his eyes off the news in case any patches are released. That's really the best you can do in the end.
As much as I feel bad for the Rails team, it may hopefully be a blessing in disguise in the end. Complacency is never a good thing.
"An old friend of mine also runs OpenBSD on his machine and I don't think he's restarted in 2 years"
I regularly reach 6 months of uptime on my Debian desktop and I've got a Linux server which reaches 4-digits days of uptime.
I only reboot when I need to physically move the machine or when a remote-exploit affecting my setup is discovered.
OpenBSD takes this even further and more power to them. My "todo list" since a very long time is to install a firewall running OpenBSD. I should really take the time to do this.
May I recommend PFSense rather than OpenBSD proper, all the power of PF wrapped up in a gui that lends itself directly to firewall configuration. I really enjoyed setting up CARP with it.
PFSense is using an older and patched version of OpenBSD PF and as you said all the configuration is done using the gui. PF configuration in OpenBSD is very simple using the cli and following the official FAQ: http://www.openbsd.org/faq/pf/index.html
Since neither Ruby nor Rails are part of the OpenBSD base system, this point is largely moot. Practically the entire ports tree consists of software that was not developed with careful security from the beginning. I think yours is a straw man argument, and it is not a surprise to me that you felt the need to fling invective at the Ruby community as backup.
Not sure where anyone said it was part of Base, I am sure it was generally acknowledged as a port that was falling behind, and therefore presenting a security risk if installed, and a burden to maintain, given most users fall back to Ruby Gems anyway.
It is rather sad that any story with 'ruby' in the title seems to bring out people who are quick to shout, stamp, accuse and drown out any voices that question how things are currently being done in the Ruby world.
I would question one of your points though, you said "Practically the entire ports tree consists of software that was not developed with careful security from the beginning" - I presume you sat with each and every developer of each piece of code involved to question whether security was on their mind when they sat down to design and code, or do you feel "the need to fling invective" as you mentioned earlier?
The guy above clearly applied the coding standards of the OpenBSD base system to a contributed package and used that as a basis for argument. Irrespective of how Ruby's developers do things, it is sloppy thinking.
As for going through the ports tree, your presumption is odd, of course I haven't sat with each and every developer. Perhaps you intended irony. Ho hum. But where's the invective? It is exceedingly rare for software to be written with as much security-consciousness as OpenBSD. I don't think that's a controversial statement.
- gems shouldn't be installed via ports in the first place
- the ports are not being maintained by BSD
- the gems/ports in question would need to be patched
You turned that into "rails sux, BSD rules." Please stop.
The choice of "security as an afterthought" and "never had any security holes ever" is a false choice between two extremes that don't actually exist (well, at least the second). The poster is referring to two very different approaches to software security. OpenBSD's approach is considered to be the most uncompromising in the industry, and goes further than probably most of us would prefer to go, but nonetheless serves as a good example of what's possible. You can read about it here: http://www.openbsd.org/security.html and there's also some good papers/presentations here: http://www.openbsd.org/papers/ .
It's not about total and complete prevention, it's about reduction of risk. Your logic taken to its logical conclusion would argue against practically any risk mitigation measures at all. For instance, even SSL/TLS have not been immune to exploits.
I've used a lot of different languages, libraries, frameworks, and whatnot over my career. They all have their own problems.
Some, however, have far, far more problems (and more serious problems) than others. JavaScript, Ruby and PHP are three examples of very troubled languages. The languages themselves are filled with rather stupid flaws. Their communities are toxic, and in many cases ignorant. The software written in such languages generally exhibits poor performance, poor security, poor maintainability, and various other issues.
Call it "FUD" if you want. I see it more as the expression of truths that some find painful to acknowledge. Some programming languages and their surrounding environments are much, much worse than others. I'm not going to pretend that they're good when they aren't.
Well since your affirmations are never argumented nor sourced, I have hard time qualifying then as "expression of truth". They are much like "immovable opinion of a troll" to me.
Again in this comment you blame Ruby/PHP/Javascript without any detail:
> poor performance, poor security, poor maintainability, and various other issues.
Hum, well ok ... compared to what ? All these properties seems relative to me. And I really don't think that a langage / platform can combine all of them. Just like a database can't be CAP or like the project triangle[0]. Engineering is all about tradeoffs.
> Their communities are toxic, and in many cases ignorant.
Hum, even better... Even from Theo de Raadt this sentence would feel arrogant to me.
Just to be clear I have no problem with you having this opinion, and I don't really want to debate about it. I just wanted to know if you had some rational behind it. Now I have my idea...
I think he was actually pointing out a difference in approaches. OpenBSD tend to take the conservative line, the Ruby crowd seem to take the front-door open with Yaml parsers blazing ready to run arbitrary code line.
I am sure it is possible to write conservative, stable, secure frameworks and tools in Ruby, but it is rather telling that we don't.
I did for a while, but as the patch suggests, it is probably a better idea just to install using gem. There is really no need to have rails in the ports as it doesn't require any special compilation to run on OpenBSD.
Ironic to see this suggestion when this post is immediately below one on the front page saying rubygems are not safe to install. It would be too bad if the security savvy BSD folks pushed their users into a worse situation by using gem.
It's not the role of OpenBSD or any other distribution to provide those gems.
I don't know anyone that rely on system packaging to get gems or eggs or CPAN module. And it would be silly because you can't run an arbitrary ruby/python app with and arbitrary version of gems.
And IHMO Debian should take the same decision and stop packaging gems and eggs.
Well, I would expect the OpenBSD folks to think that the Rails folks will eventually get it together or die off. I would also imagine, given it takes no special code to run Rails on OpenBSD, that they feel the Rails folks have more resources to push out releases and OpenBSD folks shouldn't wait on a port.
The situation currently is an unmaintained and guaranteed insecure package, so I don't think having users rely on something else will make it worse.
> These ports haven't been maintained and have been subject to numerous security issues over the past couple years (including a very critical one that made the news this week)
Sure. We had a number of Rails apps running in our production environment. The recent weeks have been a very stressful time. A critical Rails or Ruby related vuln is discovered, and then we have to make emergency changes to try avoid the app or server getting compromised.
Of course, given the large number of these critical issues, the business decided to pull and rewrite the apps based on input from the It Security team who described the current Ruby developer culture as being immature, ignorant of software engineering principles, unaware of secure coding, too enamoured with shiny/magic/convenience to care about longer-term implications in their design choices, thinking that constant band-aid fixes are the right way forward rather than acknowledge and fix poor design and implementation, and basically a liability for the company and it's data.
> Of course, given the large number of these critical issues, the business decided to pull and rewrite the apps based on input from the It Security team who described the current Ruby developer culture as being immature, ignorant of software engineering principles, unaware of secure coding, too enamoured with shiny/magic/convenience to care about longer-term implications in their design choices, thinking that constant band-aid fixes are the right way forward rather than acknowledge and fix poor design and implementation, and basically a liability for the company and it's data.
And you're switching to PHP? Someone should be fired for that decision.
Dude, every one of your posts on HN I've seen has been about how Ruby sucks and should be thrown away. Either you're a troll or you're a very angry and upset person. Either way, I feel sorry for you.
I don't mean you any offense, but your response is the kind that makes the entire Ruby community look very bad.
It is perfectly normal for responsible software developers, system administrators and business leaders to get angry or upset by security vulnerabilities. The impact such vulnerabilities can have on an organization is staggering.
When they're exploited, there can be huge sums of money lost. There can be serious legal ramifications. A company's reputation can be destroyed by a single incident. And those are just a few of the consequences.
Even when these vulnerabilities aren't exploited, there are still significant costs associated with fixing them, testing such fixes, deploying the fixes, and so forth.
Software is supposed to bring benefits to its users, not problems and costs. Costly problems with software will make many people angry.
Ruby, Ruby on Rails and some related software have had a very bad time lately. They have caused a lot of problems for a lot of people, and this has indeed resulted in anger and wasted monkey. Some people are making a sensible decision to look toward alternative technologies, to try to minimize their losses.
Instead of labeling such people as "trolls", the Ruby community as a whole needs to engage in some significant self-evaluation. Try to understand why people are expressing what they are, and perhaps even try to learn from what they're saying. Security is important, and the Ruby community needs to learn this fact.
Er, I'm sure this is true, and I'm no Rails fanboy (indeed I've never even used Rails, and haven't written more than 10 lines of Ruby in my life), but the grand-parent poster really does come off as a troll or someone with an axe to grind.
He's vague and inflammatory, and avoids technical details in favor of generically insulting language ("the 'IT security' team describes ruby culture as immature"), apparently more aimed at denigrating a particular community than clarifying any issues or solutions. Much of what he writes is borderline absurd ["Rewriting in python and PHP"... really? PHP is their choice for a better-"engineered" replacement?!]
Everybody sometimes writes vague and snarky posts, but doing so repeatedly is a warning sign...
I was less confused by his claim that the ruby community isn't security oriented (or whatever) by the fact that they decided to switch to PHP, which has a much, much worse track record.
As of this writing, the grandparent was downvoted; this post is a response to that situation.
> Ruby sucks
Ruby actually does suck. I don't like it; that's my personal opinion.
But I can't say that. All the cool startups are using it, so Ruby has become one of HN's sacred cows: There are people on here, like the parent, who take offense when someone dares have anything but the highest praise for Ruby and Rails.
I've looked at Ruby. The syntax is ugly; the language is about as bad as Perl in this regard. I've deployed Ruby applications; it's complex (although, to be fair, this is endemic to Web development). Rails is a nightmare.
A lot of HN'ers seem to like it. That's fine. But we should listen to unpopular opinions; important truths aren't always immediately universally accepted as such.
Problem: Maybe Ruby is actually the best thing since sliced bread, but I don't understand its greatness because I haven't seen the right tutorial. Solution: Write a better tutorial, improve existing ones, or point out your favorite beginning Ruby tutorials on HN or your blog.
Problem: Maybe the syntax of Ruby is really what makes me uncomfortable. Solution: A Python-to-Ruby bridge would let people like me experience the goodness of Rails without the pain of learning Ruby's extensive syntactic "innovations."
Problem: The grandparent's employer was uncomfortable with the number of security vulnerabilities recently found in Rails. Solution: Maybe having a stable branch which didn't innovate quite as fast and focused on security would make enterprise users more comfortable with Rails. Or maybe the Rails community needs more thorough vetting of new features for potential security holes, perhaps through requiring separate reviews for security and functionality before patches are officially accepted.
But if we just dismiss anyone who criticizes Ruby as a troll, how are we supposed to recognize and respond to actual problems like these? The grandparent's issues were a legitimate criticism of Ruby, and should not have been downvoted.
Ruby isn't my first or only language. I'm pointing out that there doesn't seem to be much of any analysis or reasoning behind his posts. That's it. It amuses me that people automatically assumed I meant Ruby > all. I haven't claimed that and definitely do not think that. I think this is actually a sign of how people judge other languages really badly.
While you may not agree with static_typed (his name suggests he's probably not a huge Ruby fan), the Rails community (& to some degree the Ruby community) should listen to folks like him. This is because more and more people would agree with static_type's statements.
The high chance that more catastrophic rails exploits will be found in the next month (if all of the paths to YAML.load have been found, I'll eat my hat) will increase the number of people making or agreeing with statements like static_typed's. After an exploit or two have been found in February or March, people will be pointing at Rails and saying 'I told you so'.
Rails is almost entirely a wrapper for these components like ActiveModel and Sprockets and such. You can use them apart from Rails but that's why they are all on the ports tree. It's probably the only thing that uses them.
This is very much not true for many of the components. I have a few gems that rely on ActiveSupport and ActiveModel, Sprockets has integration with other frameworks, etc.
That said, you're right, the 'rails' gem is really a meta-gem that installs all the right versions of the other ones.
Yeah but are those other gems in the OpenBSD ports tree? The point is not that they are not used for other things but that they are probably not used in OpenBSD
Since 99% of package management is just files on a filesystem and a bit of dependency analysis, for the purposes of easing installation, permitting bidirectional awareness of state, and alerting administrators to security updates.
I would commend any OS that has the smarts to hook into the package ecosystems of her guests. RubyGems, CPAN, npm, PEAR, PyPI into APT, RPM and what have you. Wouldn't it be great if.
Here's an edge case, though. In the specific world of both Ruby and her fat offspring Rails, the proliferation of versions (and the widespread separation of sysadmins from developers) means that in practice many Ruby applications have the runtime language binaries and package dependencies installed in app-specific or personal home directories, via the likes of rvm. Stick that in your package management pipe and solve it.