

High concurrency friendly IDs in Rails without SQL locks. - codebeaker
http://lee.hambley.name/2012/2/27/friendly_id-and-parallel-processes/

======
jeltz
To solve this problem is why many databases support sequences. Sequences are
number generators which work outside the transactional system (e.g. sequence
incrementations cannot be rolled back) so the locks are very short.

To use sequences in PostgreSQL use simply call nextval('name_of_sequence') in
the database. Rails uses the SERIAL type for the primary key column which is
implemented with a sequence.

------
ExpiredLink
> FriendlyId v3.x prevented this by keeping the slug sequence in a separate
> place in the database, and operating upon it within a transaction, thus
> guaranteeing atomic access to reading, incrementing, and updating the slug
> sequence.

So he created a 'solution' where no problem exists. At least, TIL an new word:
slug.

~~~
zaptheimpaler
>So, the problem is since friendly_id v4, the gem no longer guarantees two
really important principles:

>No guarantee that we're the only person reading this value, right now

>No guarantee that the ID we'll choose will be unique a few µseconds from now.

The problem exists in the newer version. If anything, the developers created a
problem where none existed, not him. They traded consistency for simplicity
without a clear mention of the possible race conditions.

~~~
ExpiredLink
>> No guarantee that the ID we'll choose will be unique a few µseconds from
now.

That's why you let the database atomically create the next id for you. There
is no problem. You either get the next unique id or you have to try again.

~~~
dasil003
So the obtuse point your trying to make is that slugs are unnecessary and we
should just use an auto-increment or uuid?

Sorry, but slugs serve a real purpose, so your solution is only a solution in
the sense that walking is a solution for a broken down car.

~~~
codebeaker
> Sorry, but slugs serve a real purpose, so your solution is only a solution
> in the sense that walking is a solution for a broken down car.

This doesn't replace slugs, at all - the slugs are still generated, this
replaces the SQL unique sequencer. The SQL sequencer which isn't universally
supported, and has a different API on every platform.. many modern
applications have Redis or a similar atomic structure store which could
provide an interim solution, short of building a purpose built ticket
provisioning system such as that employed by Flickr.

------
codebeaker
I'm weighing in here on the comments about "Solving a problem that doesn't
exist" - the database is quite loaded, and we have Redis in the stack anyway.
The solution to this problem is any backend which creates unique numbers
atomically; I happened to choose Redis because I'm interested in it
technologically and I like the API.

It's worth mentioning that Rails doesn't encourage access to the underlying
database features - and that stepping outside of the Rails/ActiveRecord
results in features which are difficult to test and more difficult to maintain
in the face of an ever changing API.

I don't care where the next atomic ID comes from, as long as it comes from a
datastore that is fast, and that I can trust.

~~~
kstrauser
> I'm weighing in here on the comments about "Solving a problem that doesn't
> exist" - the database is quite loaded

Can you give a ballpark estimate of your write load? And in any case, a
server-side trigger executed on insert could handle the process much closer to
the data, with fewer round trips and much greater performance.

> I happened to choose Redis because I'm interested in it technologically and
> I like the API.

With due respect, you've gotta get over that. I'm all about experimentation,
but there's a time to break new ground and a time to use the tools that others
have already developed and tested.

> It's worth mentioning that Rails doesn't encourage access to the underlying
> database features

That may be true, but if you're pushing it near the edge of breaking, it may
be time to bite the bullet and move small amounts of logic into the database.
It's only very slightly harder to test - the general principals are the same -
and I'd dare say far easier to maintain.

------
jlouis
I would definitely contest that you have to lock the whole database table for
readers. Modern SQL systems will not have readers waiting upon writers. And if
you use a SEQUENCE, like you probably should in this case, then you only need
a sequence-lock once in a while.

There are also fully independent schemes for this, but they cannot, usually
give you ordering properties of your counter sequence. And those who can will
have rather large keys (large in size of bytes)

~~~
gaius
The only DB I'm aware of that still requires a table lock is SQLite (!)

He's invented this ridiculous Rube Goldberg contraption because his knowledge
of RDBMSs is 20 years or more out of date...

------
sanxiyn
How Flickr does this: [http://code.flickr.com/blog/2010/02/08/ticket-servers-
distri...](http://code.flickr.com/blog/2010/02/08/ticket-servers-distributed-
unique-primary-keys-on-the-cheap/)

Simple, really.

~~~
codebeaker
And overkill for almost all intermediate sized installations, but a great use
of their technology to meet their requirements, thanks for the link.

Of course Twitter have snowflake: <https://github.com/twitter/snowflake> and
many other tools such as UUID[4r] - see an old discussion on HN:
<http://news.ycombinator.com/item?id=1396667>

Of course I wanted sequential, small IDs, within a small margin of acceptable
loss (if a save fails that ID is never used), that were't taking up a massive
amount of space.

------
VeejayRampay
Can't fibers be used to that effect?

Something like an infinite with Fiber.yield returning a unique ID or
something.

------
fleitz
This problem can also be solved with SQL. An after trigger on the table and an
indexed computed column would all work. If you're using Postgres probably just
a regular INSERT with a call to curval.

This seems like an awful lot of work to avoid a stored proc / trigger,
computed columns, or custom INSERT statement. Even in MySQL a stored proc
would work, or two INSERT statements in a transaction.

~~~
jeltz
Minor correction: You call nextval in PostgreSQL to increment a sequence and
get the number. curval is for getting the last value returned by a nextval
call within the current transaction. Since PostgreSQL implemented RETURNING
for INSERTs and UPDATEs curval has become more of a utility function for use
with psql and similar clients.

