
Pgloader: A High-speed PostgreSQL Swiss Army Knife, Written in Lisp - wtbob
http://pgloader.io/
======
dim
So the Quick Start is now back available at
[http://dimitri.github.io/pgloader/howto/quickstart.html](http://dimitri.github.io/pgloader/howto/quickstart.html),
and I made a release with files hosted at github for your testing pleasure:
[https://github.com/dimitri/pgloader/releases/tag/v3.2.1.prev...](https://github.com/dimitri/pgloader/releases/tag/v3.2.1.preview)

You have self-contained binary files allowing you to play with
/usr/bin/pgloader (or ./pgloader) without having to know anything about Common
Lisp.

Enjoy ;-)

------
dim
Here's another link to the video while the all static website seems to have
some sort of an issue:
[http://medias.ircam.fr/stream/ext/video/files/2014/05/13/ELS...](http://medias.ircam.fr/stream/ext/video/files/2014/05/13/ELSAA_6_mai2.mov.webm#t=57,357)

Also see the github page for the project at
[https://github.com/dimitri/pgloader](https://github.com/dimitri/pgloader)

the main docs at
[https://github.com/dimitri/pgloader/blob/master/pgloader.1.m...](https://github.com/dimitri/pgloader/blob/master/pgloader.1.md)

Check out the USAGE section (it's a quickstart really) at
[https://github.com/dimitri/pgloader/blob/master/pgloader.1.m...](https://github.com/dimitri/pgloader/blob/master/pgloader.1.md#usage-
examples)

------
bronson
Wow, switching from Python to Lisp produced a shocking speedup. I liked Lisp
in college but I probably never would have considered it nowadays... If I
wanted a quick runtime, I probably would have fallen for Go or Julia or, after
some maturing, Rust.

Lisp! Gonna keep that one in mind.

~~~
wes-exp
Here's Lisp (SBCL) vs. Python 3:
[http://benchmarksgame.alioth.debian.org/u64/compare.php?lang...](http://benchmarksgame.alioth.debian.org/u64/compare.php?lang=sbcl&lang2=python3)

Summary: crushes it.

~~~
Erwin
Well, the SBCL code includes code for literally generating assembly code
specific to x86-64 to get that performance and is 3 times as long as Python.
Presumably the inline assembly code is not part of any LISP standard but
highly specific to that specific LISP implementation.

~~~
lispm
That's only the mandelbrot benchmark.

------
dim
So a new mirror is up now, enjoy!

[http://dimitri.github.io/pgloader/](http://dimitri.github.io/pgloader/)

------
craigkerstiens
Here's a link to the github repo for the project as well -
[https://github.com/dimitri/pgloader](https://github.com/dimitri/pgloader)

------
SixSigma
I've written a PostgreSQL protocol client in Limbo [1] that uses CSP /
Channels ... long long ago in the time before Go-lang ...

I can say that libpq is _not_ the best example of how to write a client. It
uses all text, it doesn't use the asynchronicity very well.

I think there is some room in libpq for improvement, if only someone would
take it on :)

[1] [http://plan9.bell-
labs.com/sources/contrib/maht/limbo/pg/](http://plan9.bell-
labs.com/sources/contrib/maht/limbo/pg/)

------
dim
BTW if you're going to FOSDEM check out
[http://www.postgresql.eu/events/schedule/fosdem2015/session/...](http://www.postgresql.eu/events/schedule/fosdem2015/session/827-migrating-
to-postgresql-the-new-story/) where I will host a full talk about pgloader ;-)

~~~
ubercow
Any idea if the talk will be recorded? I'd love to listen to a full talk but
some space and time restrictions prevent me from attending in person.

~~~
dim
I'm told by one of the FOSDEM PGDAY conference organizers: "I believe they
won't be". Sorry about that. The slides are going to be available on
[http://tapoueh.org/](http://tapoueh.org/) and
[http://pgloader.io/](http://pgloader.io/) though.

------
pjmlp
Checking the documentation I've found out it supports dBase!

------
hittaruki
Is it possible to do streaming replication to PostgeSQL, from another
database, say MYSQL?

This would be the best way to do a migration without braking an existing
system.

~~~
dim
Well we can consider that pgloader already implements the first half of the
streaming replication: the base copy to stream changes onto.

Adding support for the MySQL binlog replication format is on the TODO list, if
you're proficient in it I would welcome hints to implement it correctly.

In terms of priority and organisation, it's Open Source so you know how it
works right? Contributions are welcome, as are feature based sponsoring.

~~~
hittaruki
I would love to contribute to it, but I am not an expert on the subject. Its
just something I tried to do a while back, but couldn't find a clean way.

Let me see if I can figure it out and will try to give you a pull request.

~~~
dim
Awesome, thanks already for putting some time on it. You will easily find my
email if you need help playing with pgloader internals and stuff.

------
jaryd
Can this be used to perform a high speed recovery from a pg_dump?

------
jbverschoor
Would love mongo import :)

~~~
dim
It's on the TODO list ;-)

------
Chmouel
cool! awesome video as well i recommend watching it,

