
PGX: Write Postgres extensions in Rust instead of C - adamnemecek
https://github.com/zombodb/pgx
======
jarym
Guys, this is really really cool. Been looking to learn Rust and it just so
happens I need to write a Postgres extension in the near future so two birds
and one stone.

Btw - Also, I've seen ZomboDB and while I've not used it I must say it looks
EXCELLENT. So congrats on that too.

~~~
zombodb
_blush_

------
gavinray
This is really cool! The only other similar thing I know of is WASMer's
extension (which only supports Rust atm I believe):

[https://github.com/wasmerio/postgres-ext-
wasm](https://github.com/wasmerio/postgres-ext-wasm)

One question though: If you built and compiled an extension with this, how
would you deploy it on top of a Postgres Docker image?

I see instructions for installing and testing locally, via the cargo command,
but couldn't find anything on this.

Does it spit out a static asset that you put in the project folder, and copy
over during the build pipeline? Something like:

    
    
      FROM postgres:12-alpine
      COPY ./my-rust-extension /some/container/directory
      RUN install-command-here

~~~
zombodb
That's a great question, and one probably best answered over on pgx's GitHub
page.

But! I plan on adding a command to "cargo-pgx" to package up the extension for
you into a directory structure (or tarball, maybe).

The idea is that you'd just run: cargo pgx package

And it would just build a --release library, and create the proper directory
structure (based on what pg_config) says for the shared library and the
associated .sql.

I actually need this ASAP for supporting ZomboDB proper, so... Coming Soon!

~~~
gavinray
Awesome, look forward to it =D

~~~
zombodb
[https://github.com/zombodb/pgx/releases/tag/v0.0.7](https://github.com/zombodb/pgx/releases/tag/v0.0.7)

!boom!

------
faizshah
Interesting, it makes coding a postgres extension seem delightfully simple. I
have been looking for something to try using rust on! Thanks for posting this.

~~~
zombodb
Thank you!

If you decide to jump into it, definitely let us know any pain points you
have.

It takes a bit of time to work out the kinks in a thing like this.

------
rukittenme
Just out of curiosity, what sort of requirements compel a person to write a
postgres extension? I know of things like PostGIS but I'm not well versed
enough in this world to know the totality of the problem.

~~~
zombodb
Author of `pgx` here.

We developed pgx so that we could rewrite "ZomboDB"
([https://github.com/zombodb/zombodb](https://github.com/zombodb/zombodb)) in
Rust. ZDB is a custom "Index Access Method" for Postgres. Think btree index,
but stored in Elasticsearch.

So that's definitely a thing.

Other ideas might be custom data analytics/transformation/processing engines
that you'd prefer to run in the database instead of externally.

Custom data types to represent, for example, street addresses or genetic
information.

The only limit is yourself! ;)

~~~
boulos
> The only limit is yourself! ;)

For folks who don't get the joke, please visit zombo.com before Flash is
finally dead :).

~~~
mumblemumble
Alternatively, there's [https://html5zombo.com](https://html5zombo.com)

------
touisteur
Recent similar thing in the Ada world: [https://github.com/AdaCore/gnatcoll-
db/tree/master/pgxs](https://github.com/AdaCore/gnatcoll-db/tree/master/pgxs)

------
fnord123
Also of interest: rpgffi -
[https://github.com/posix4e/rpgffi](https://github.com/posix4e/rpgffi)

"R(Rust) PG(Postgresql) FFI (Foreign Function Interface)"

~~~
zombodb
That is interesting.

pgx does a lot of that too, but for only pg10/11/12\. And then it post-
processes the bindings and builds a "common.rs" for symbols that are identical
across versions.

That way you can target all three versions at once, and then you can use
Rust's #[cfg(feature=)] directive to opt into support for specific versions.

pgx has also done a little bit of work to mask differences between the three
versions so they appear as part of "common.rs" anyways.

I wanted a framework that let me know, at development time, if I was trying to
do a thing that isn't supported (or the same) across various PG versions.

------
darksaints
This has been on my back burner for about a year now...glad someone has gotten
to it. It already looks full featured. I've been thinking that with the right
library subset (just like libcore or no_std), it would be a very safe
possibility for a trusted language...accessible to users that are not
superusers. That would be extremely valuable to RDS users that would love to
use something fast, as opposed to the standard trusted languages like
pl/python.

~~~
zombodb
I've noodled the idea of a pl/pg_rust, but the whole "Cargo.toml" and
downloading dependencies from the internet seems bad.

I see what you're saying about no_std tho. Could be doable.

------
3ace
At first I thought it is a Postgresql driver for Go that has the same name

[https://github.com/jackc/pgx](https://github.com/jackc/pgx)

~~~
Snelius
Yes, it's always mess like sqlx.

------
giovannibonetti
It would be awesome if someone with some Rust knowledge could make an
extension to create a compressed string type like Antirez Smaz [1].

I know Postgres compresses TEXT types by default, but only when the data
exceeds TOAST_TUPLE_THRESHOLD (default 2 kB)

Some examples on the project page [1] show reduction of size between 10% and
more than 50% for some small strings (less than 100 bytes).

[1] [https://github.com/antirez/smaz](https://github.com/antirez/smaz)

~~~
stubish
I think the existing C API would be a better fit, and the work would likely be
a fairly simple modification to the citext extension found in contrib section
of the PostgreSQL source if anyone is game.

~~~
zombodb
I'd be inclined to agree.

That said, it'd be easy to prototype with pgx, assuming that compression
library exists in the Rust ecosystem too?

~~~
giovannibonetti
I guess this compression library should do the trick [1], but I haven't tested
it yet.

[1]
[https://docs.rs/compress/0.1.2/compress/entropy/ari/index.ht...](https://docs.rs/compress/0.1.2/compress/entropy/ari/index.html)

------
tcyrus
Is there any way to make custom types using pgx?

~~~
zombodb
Yes there is. It's not documented/example'd yet tho.

There's a derive macro called #[derive(PostgresType)]. Combine that with
serde's Serialize, Deserialize, and you're gtg.

I'm going to be working on more docs and twitch streams over this week.

------
ficklepickle
Typo in the title: oc->of

~~~
dang
Fixed. Thanks!

