
Show HN: Rambler – A simple and language-independent SQL schema migration tool - elwinar
https://github.com/elwinar/rambler
======
sbuttgereit
I much prefer the methodology in Sqitch to the migration pattern.

[http://sqitch.org/](http://sqitch.org/)

While I wish the tool had been implemented in something other than perl (with
10 million CPAN dependencies), I like that DDL files are versioned and
organized in ways more meaningful to the structure of the database rather than
being bound to the timeline of the database.

If I am going to do migrations, a tool like Rambler would have to compare
favorably to Flyway ([https://flywaydb.org/](https://flywaydb.org/)), which is
pretty solid.

~~~
jdc0589
i both like and dislike that the registry database is separate. i honestly
can't with it how i feel about it.

~~~
sbuttgereit
The intrusiveness of the registry in terms of database configuration depends
on which RDBMS you're using.

I use it with PostgreSQL and I don't use a separate database for the registry.
I do have a separate schema for the registry in the same database as the data,
but I would want that sort of organization no matter the case. If the database
you use doesn't include some sense of "schema" below the database level of
separation, I could see how that would be annoying from an
administrative/maintenance point of view (or on hosted database solutions).

------
aikah
Basically a copy of SQL migrate :

[https://github.com/rubenv/sql-migrate](https://github.com/rubenv/sql-migrate)

I'm not sure which tool came first in the Go community. While I'm not a big Go
fan, this is precisely how Go can be handy sometimes, by allowing easy
distribution of tools that don't depend on an interpreter or a JVM...

~~~
TeMPOraL
Wait, doesn't Go require you to install a Go runtime?

~~~
aikah
The point is you can release binaries for every plateform thanks to cross-
compilation, no need to install Go on your machine. And no need to install a
runtime no, the runtime is embedded into the binary.

------
jdc0589
I'm a biased fan of migration utilities like this (I've written a couple). I
like using plain old sql for migrations, it makes more sense to me. Sure, you
loose some DB mobility you get with the projects that have their own DSL, but
then again I don't have to learn a new DSL.

I kind of wish I had kept developing
[http://jdc0589.github.io/mite/](http://jdc0589.github.io/mite/) the feature
set is pretty solid even though much of its isn't documented in the site, but
I got to the point I wasn't working with sql databases much anymore, so the
interest kind of died off.

~~~
elwinar
Rambler doesn't have any custom DSL, it use plain SQL. Just like Mite :p

~~~
jdc0589
yep, thats why I automatically liked it.

------
xomateix
It looks interesting and I like the simplicity of the tool, looking forward
for seeing more databases supported.

Besides needing the jvm installed, are there any differences between rambler
and flyway command line [1] that make it a more suitable choice?

[1]
[https://flywaydb.org/getstarted/firststeps/commandline](https://flywaydb.org/getstarted/firststeps/commandline)

~~~
axelfontaine
Flyway author here.

Besides supporting many more databases, Flyway's parser is also much more
robust with support for all kinds of database specific things like changing
delimiters with MySQL, Oracle PL/SQL, PotsgreSQL COPY FROM STDIN, T-SQL, ...

And you can use truly plain SQL files like the ones generated from your DB's
dump tool as a starting point, no special comments required.

(And of course you also have things like repeatable migrations, Java-based
migrations (great for complex data transformations), API and build tool
integration, ...)

~~~
ojiikun
All due respect to someone who has created a tool that does genuinely save me
some work when deploying, but I can't internalise calling flyway a "migration"
tool when failed deltas require such catastrophic manual cleanup. One typo or
semantic error in a file and I get a corrupted stack that requires me to edit
not only the file but the metadata table. I have started double-testing
deployments to mitigate the awfulness, but are there any plans to ever make
failed changes more graceful?

~~~
axelfontaine
No need to ever manually edit the metadata table. That what Flyway's repair
command is for.

However if failed migrations are a big concern for you, do yourself a favor
and consider moving to a database that offers proper DDL transactions like
PostgreSQL, SQL Server or DB2. Flyway runs every migration inside a
transaction and this way changes become truly atomic without any sneaky
implicit commits (I'm looking at you Oracle and MySQL).

------
arekkas
how does it differentiate from

* [https://github.com/mattes/migrate](https://github.com/mattes/migrate)

* [https://github.com/rubenv/sql-migrate](https://github.com/rubenv/sql-migrate)

* [https://github.com/pressly/goose](https://github.com/pressly/goose)

~~~
elwinar
Goose does too much IMHO, and sql-migrate is mainly a wrapper around goose. I
looked at it for inspiration, but disagreed with many choices.

Migrates is another thing, and I like it more except for the lack of
configuration file.

------
mixedCase
I've been eyeing Pop[1], which is a data-mapper/ORMish library wrapping around
sqlx[2]; and it happens to contain a new DSL called Fizz[3] that allows for
database-independent migrations.

To be honest, I have always tried to avoid Go ORMs after a bad experience with
gorm, but the whole package is looking very interesting for reducing the
boilerplate involved in Go CRUD applications without having to buy into a
complexity risk. It's worth a look.

[1] [https://github.com/markbates/pop](https://github.com/markbates/pop)

[2] [https://github.com/jmoiron/sqlx](https://github.com/jmoiron/sqlx)

[3]
[https://github.com/markbates/pop/tree/master/fizz](https://github.com/markbates/pop/tree/master/fizz)

~~~
kornish
What was the bad experience you had with gorm, out of interest?

~~~
mixedCase
I don't remember the details, it was around a year and a half ago. But I do
remember I had some trouble trying to optimize a few queries with gorm
complaining about something and the other guy I was working with didn't
understand it much and since we were early on development we scrapped it and
went with sqlx.

------
awinter-py
Hmm -- most migrations don't need to be reversed, not sure what the 'down'
section does in those cases.

The more important problem in DB migrations is transforming the content
(usually involves application code) and resolving relational changes (i.e. 'in
v2 all users are a member of a group').

Some large companies handle migrations by versioning the data and branching
the code to handle every version. Another alternative is to 'migrate on read'.

Both of those options sound worse on paper than an all-at-once migration but
one-shot upgrades can be expensive and dangerous on large databases. Dealing
with data-versions explicitly also makes sense for data that's stored client-
side and periodically synced.

~~~
GrinningFool
> Hmm -- most migrations don't need to be reversed,

This is true - but when something does break unexpectedly in a production
migration, the last thing you want to do is figure out how to do your
revert/'down' on the fly.

~~~
awinter-py
Okay, but are you testing every 'down' command on your prod DB? Large untested
operations in prod may not be the best answer in an emergency.

------
dwb
I wrote one a little while back, for MySQL and PostgreSQL, that's just a bash
script. I've found it handy a few times.

[https://github.com/dwb/dogfish](https://github.com/dwb/dogfish)

~~~
elwinar
I had one of those at the very begining, but then I wanted to do things that
are more pleasant to do in other languages, so I switched to go. But I can't
agree more with your README, my focus stayed on getting out of the way: drop
your requests into a file, dump your credentials in a configuration file, run,
be done with it.

------
shawndellysse
I've used (and committed a few small fixes to) Rambler in a few of my
projects. My team and I chose Rambler over the alternatives for a few reasons,
the primary one being simplicity all around, in the sense that there is no
runtime needed for this tool, just a binary that you run with your deploy
scripts.

------
sametmax
But does this mean your migrations are limited to only one type of DB ? So if
you switch from sqlite to oracle, you loose all your migrations?

In that case, I'll stick with my ORM solution.

~~~
elwinar
If you switch you vendor, you will probably have to fix manually many things,
be it migrations, code, etc. An ORM-based solution is a thing, but then you
lose the ability to use vendor-specific behavior, unless your ORM have
specific extension for those, and then you're back at square one.

