
Mongrel2 Sqlite3 Config System Working - twampss
http://sheddingbikes.com/posts/1277495906.html
======
cageface
This claim seems completely specious:

 _With a config file you have to hunt around for the right stanza, type in the
right exact change, restart the server, and pray it works. With SQL you can
actually query the database, and run updates (then pray that works) which are
much easier to get right and do._

With apache, I make an edit, check the syntax with apachectl configtest, then
restart if the syntax checks out. The kinds of errors that get by the syntax
check (bad regex in rewrite rule, typo in a host name etc) are just as likely
in SQL. Text files allow me to use the same version control tools, text
editors etc I use for the rest of development. Deployment is just a matter of
copying a simple text file around etc.

Putting this all in SQL is a solution in search of a problem.

~~~
zedshaw
Nope, it's a different solution to the same problem. Like you said, if they're
equivalent solutions, yet the sqlite3 one is simpler (code is smaller by far)
and has other advantages (migrations, tooling, schema) then sqlite3 wins.

Also, I still stand by that getting a SQL update correct is much easier than
editing an apache config file. Only The Spaghetti Monster knows how Apache
configs actually work (and he works for Microsoft).

~~~
cageface
They may be equivalent in terms of functionality, but I'm having a very hard
time guessing how they might be equivalent in ergonomics.

For instance, how would you represent something like this in SQL?
<http://topfunky.net/svn/shovel/nginx/conf/nginx.conf>

~~~
zedshaw
Ohhhhhh, I see you want huge swathes of commented out sections, bizarre syntax
for routing, and lots of nesting that makes no sense and has lots of
repetition. I'll get right on that.

Seriously though, SQL's been proven capable of doing all of this, and for the
half-assed turing tarpit if-statements in these files you'll get something
better. I'm aiming for Lua or similar to let you really control it.

But if your worry is that there will be some monster data structure you have
to understand and work with manually in order to do anything then have no
fear, you will get good tools.

------
amix
I like how nginx configuration works. It's simple, it's a DSL, it's a text
file that's viewable by anyone, you can version it easily, you can comment in
it, you can quickly take backups or revert back to old configurations, you can
send a kill -HUP to reload the configuration without taking your site down.
The bottom line: The nginx, and the usual text based approach, is simple and
it works, I can't really see what he is solving with this database approach
other than adding a lot of additional complexity.

~~~
zedshaw
So, everything you said applies to SQL. SQL is a DSL (and a standardized one).
SQL is text, so it can be versioned, you can back it up, and you can do it
also the .sqlite file. You can also do -HUP to reload configs, nothing about
SQL or SQLite prevents that, and in fact, nginx has to completely restart its
world to make this happen. Mongrel2 will probably be able to reload without
kicking over the process.

The problem I'm solving is one of operations automation and design. Go take a
look at the nginx code for loading configurations and then go write an nginx
module. Figuring out how to get configuration settings out of the thing is
horrible. Then, come talk to me when you can change the nginx config on 100
hosts based on the hostname and three sections of the configuration file using
sed and awk.

Hell, let me know if you can load a random nginx config and tell me what
version of nginx it works for. That'd be a trick.

~~~
cageface
_Then, come talk to me when you can change the nginx config on 100 hosts based
on the hostname and three sections of the configuration file using sed and
awk._

If I have a section of a config file that's host-specific I'll bust it out
into an include and generate just the relevant parts.

 _Go take a look at the nginx code for loading configurations and then go
write an nginx module. Figuring out how to get configuration settings out of
the thing is horrible._

Putting settings into SQL doesn't magically solve this problem. As an admin
I'm still going to have to know what fields and values each module expects.
Much worse, I'm also going to have to manage tree-structures in SQL for
directives with subdirectives, which is something even full-time SQL people
often struggle with.

~~~
zedshaw
First, no, if you actually read the nginx code for modules they each have to
define their own little schemas in raw C code using some undocumented or
barely documented C structs. In the sqlite3 database any modules I do will be
equal citizens, able to have their own tables, and query the database. Much
more orthogonal solution.

So yes, putting them in SQLite (not IN AND RDBMS), does magically solve it,
and does it for all the languages that know sqlite3, which is all of them.

As for having to know about the structure of the database and know trees and
stanzas and things like that: Did you RTFA? I said there'd be handy tools for
dealing with it so you don't need to know this stuff. Seriously, you think I'm
going to foist a full on SQL schema on anyone?

That'd be about the stupidest thing ever. Of course I'm going to have tooling
that makes this easier to work with, and in fact _you_ can write your own
tools to manage it.

~~~
cageface
_As for having to know about the structure of the database and know trees and
stanzas and things like that: Did you RTFA? I said there'd be handy tools for
dealing with it so you don't need to know this stuff. Seriously, you think I'm
going to foist a full on SQL schema on anyone?_

So you _are_ planning on wrapping this in some kind of reasonable DSL? If so,
the fact that you're storing an intermediate representation in SQL seems
uninteresting, although you also seem to suggest that the advantages of this
approach are realized via direct manip via SQL.

I can see some room for improvement and standardization in config languages,
but any server that doesn't allow me to do config purely via text files and
that doesn't have a config language rich enough to encompass directives with
subdirectives and conditionals isn't useful to me. Why not just make
everything Lua?

------
koenigdavidmj
Mr. Shaw, do you ever sleep?

~~~
mbreese
I heard a rumor once that he slept for 45 minutes. When he woke up, he sat
down at his computer and typed out the most elegant program ever written. He
had composed it all in his dreams. It was in pure x86 assembly. Then, looking
upon the goodness of it all he said "fuck it, they don't deserve this" and
then deleted it.

At least, that's what I heard...

~~~
pjscott
Reality produces things that are even stranger: Ken Thompson and Dennis
Ritchie once accidentally wrote the same program. It was 20 lines of assembly
code, and what they had written was character-for-character identical.

~~~
stcredzero
Never play those two in Pictionary! You'll be sorry if you do.

------
stcredzero
With Go's fast compile features, why couldn't a server start consist of using
the configuration data to generate code which then gets compiled into a hella-
fast optimized server? Paths could be highly-optimized hard-coded pattern-
matching finite automata.

Someone would have to up and write a Go web framework to use with that. They
wouldn't have to, but it would be a shame not to.

~~~
zedshaw
That's possible, and probably not too hard, and man oh man would the hackers
_love_ a compiler right in their web server. :-)

~~~
stcredzero
They already pretty much have that with Perl, PHP, Python, Ruby, Smalltalk,...
Not necessarily in the front end web server, though.

Anyhow, that's easily solved. It wouldn't have to be in a process doing any
web serving. The compiler would be available in a process that just does the
starting up. The worker processes would then know which dynamically loadable
module they need to load. So, no there wouldn't have to be a compiler in the
web server.

------
ErrantX
_INSERT INTO host (server_id, name, matching) VALUES ( last_insert_rowid(),
'mongrel2.org', '(.).mongrel2.org' );_

I love this.

(I can't think of any other way of putting that :))

~~~
cageface
I'm having a very hard time seeing this as an improvement over
apache/nginx/lighttpd text-file configuration. Web server configs also have a
tendency to be heirarchical, which sounds extra-painful in SQL.

Let's take a real-world nginx conf for example (comments and blank lines
elided to save space). I can't imagine how trying to shoehorn something like
this into sql is a win. Even Java-style XML situps seem preferable:

    
    
      http {
      include /usr/local/nginx/conf/mime.types;
      default_type  application/octet-stream;
      log_format main '$remote_addr - $remote_user [$time_local] '
                      '"$request" $status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
      access_log  /var/log/nginx_access.log  main;
      error_log  /var/log/nginx_error.log debug;
      sendfile on;
      tcp_nopush        on;
      tcp_nodelay       off;
      gzip            on;
      gzip_http_version 1.0;
      gzip_comp_level 2;
      gzip_proxied any;
      gzip_types      text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
      upstream mongrel {
        server 127.0.0.1:5000;
        # server 127.0.0.1:5001;
        # server 127.0.0.1:5002;
      }
      server {
        listen 80;
        client_max_body_size 50M;
        root /var/www/apps/mysite.com/current/public;
        access_log  /var/www/apps/mysite.com/shared/log/nginx.vhost.access.log  main;
    
        if (-f $document_root/system/maintenance.html) {
          rewrite  ^(.*)$  /system/maintenance.html last;
          break;
        }
        location / {
          proxy_set_header  X-Real-IP  $remote_addr;
          proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header Host $http_host;
          proxy_redirect false;
          proxy_max_temp_file_size 0;
          
          if (-f $request_filename) { 
            break; 
          }
          if (-f $request_filename/index.html) {
            rewrite (.*) $1/index.html break;
          }
          if (-f $request_filename.html) {
            rewrite (.*) $1.html break;
          }
          if (!-f $request_filename) {
            proxy_pass http://mongrel;
            break;
          }
        }
        error_page   500 502 503 504  /500.html;
        location = /500.html {
          root   /var/www/apps/mysite.com/current/public;
        }
      }

~~~
ErrantX
When you have to regularly deploy maintain an nginx instance that hosts a
large number of small sites you will understand :)

~~~
pedrocr
This was the use case I was imagining would be easier with the sql solution.
But wouldn't that be best handled by splitting the single .conf file into an
hierarchical set of simpler files in directories? Apache vhosts are already
handled like that.

~~~
ErrantX
> Apache vhosts are already handled like that.

And they are not much better :) (actually I prefer Nginx - better syntax)

