
Database “sharding” came from Ultima Online? (2009) - mpweiher
https://www.raphkoster.com/2009/01/08/database-sharding-came-from-uo/
======
fcatalan
I personally remember that couple of years, 95-97, as ripe for coming up by
yourself with things that are now considered "obvious" or just the way things
are done. Things happened fast: by the start of 1995 the concept of networked
computers was a vague thing from the movies to me and by the end of 1997 I was
an "experienced" web developer. We had websites, but we didn't have good
search, so you still had to learn many things the old hard way.

The biggest thing I remember to have "invented" by myself in early 1996 (at
the same time as thousands more) was the database driven website: I was tasked
with creating a "virtual campus" kind of website with forums, chat,
assignments, calendar, grades, news... and the sparse examples and models I
had found for all those functionalities used plain text files as storage.

After completing a prototype I was obviously sick of dealing with those files
and suddenly thought: "hey, dumping all this crap on a few tables in a
database would make everything so much easier!" It felt a bit mind-bending,
because you were taught that databases were for internal "databasey" things:
Accounting, addresses, customers, widgets and their prices, that kind of
stuff. Using a database as storage for a forum or a public-facing website felt
almost like a revolutionary concept.

By the end of the project I was quite bored of writing plain SQL and I had
something very close in spirit to many ORMs/ the Active Record stuff from
Rails. So I sort of "invented" that too.

Another thing to reflect on is that I wrote all that as plain C through CGI.
The database was mysql. I tried to run it a couple years ago and it compiled
and ran almost straight on a modern Linux system, after like 20 minutes
fiddling with Apache and the makefile, 20 years after the fact. My current
projects seem to rot every time I take a short vacation.

~~~
ericol
To that vein... I invented AJAX :P

I remember it being around 2001 - 2002 and seeing the first implementations of
Microsoft for Javascript requests and at the same time the DOM was started to
be a thing you could actually manipulate and I started experimenting on doing
programmatic page updates for a system I was developing for the office I was
working for (I worked for a state office back then).

Then the big crisis of 2002 hit (I live in Argentina), I lost 2 of the 3 jobs
I had at the time (And the one that I kept was the one bringing in the less
amount of money) and I ended up moving to Spain for a few years.

When I finally managed to land a job coding (2007) AJAX already had its own
name and dynamic pages were taking the world by storm (And to hell, via
callbacks).

P.S.: That "I invented AJAX" was obviously tongue in cheek. I'm pretty certain
that there was a lot of people doing the same thing all around the globe, and
very likely that was the idea that got that implemented on the MS side of
things in the first place.

~~~
ksherlock
Before AJAX, it was DHTML (D for Dynamic). Checking O'Reilly Safari, that term
started gaining use in published matter in 1999.

~~~
moron4hire
AJAX and DHTML were different things. DHTML was just scripting for DOM, it
didn't include making requests back to the server without reloading the page.

------
Alex3917
Out of curiosity, has anyone made an MMO better than UO yet? I stopped playing
those types of games when I got into high school, so I don't really have any
concept of if EVE and WoW and the like were actually definitively better, or
just different and more successful do to being easier for beginners.

I haven't played any of the Larian games (e.g. Divinity: Original Sin II), but
noticed that only recently with these sorts of graphics do these new 3D
isometric games actually have the same kind of feeling that UO had at the time
when it came out, if that makes any sense.

~~~
chongli
_has anyone made an MMO better than UO yet?_

Nope. UO was one of a kind. What made it special is the way it brought
together people with many different play styles and allowed them to form their
own communities which were self-policed.

Every multiplayer game since UO has tried to learn lessons from it by policing
the players centrally, through the limitation of player interaction. The
result is that all of the different play styles have gone their separate ways
to games which cater specifically to them.

This phenomenon mirrors the filter bubble phenomenon that resulted from search
engines and social media recommendation engines giving people more of what
they want.

 _Edit: for examples of the games that cater to the play styles I’m referring
to, look at Fortnite, League of Legends, Minecraft, Stardew Valley, Diablo
3... When I look at this list it’s rather shocking to me that one game could
house all these diverse play styles. The magic of it was that it worked, for a
time, until more specialized games came along and started drawing away the
player base. Then EA brought in Trammel and split the player base which was
the beginning of the end._

~~~
Alex3917
The interesting thing also was that most of these play styles were forced to
work together also. E.g. you couldn't be a fighter without blacksmiths to make
armor and weapons, and you couldn't be a blacksmith without miners who were
gathering ore. Because leveling up in one skill caused your other skills to
atrophy at a high level, it prevented players from being able to become
isolated from each other.

------
gavman
This was discussed by one of the game's creators in the Ultima Online episode
of Ars Techica's War Stories (around the 5:10 mark)
[https://www.youtube.com/watch?v=KFNxJVTJleE](https://www.youtube.com/watch?v=KFNxJVTJleE)

------
tlarkworthy
I always assumed MMO shards came from database literature. My mind is blown if
it's the other way around. Though not the first time gamedev is at the edge.

------
staycoolboy
Garriot talks about this in the documentary "Get Lamp!", which is about the
history adventure games. He talks about the challenges of maintaining a huge
MMO, especially how they were broadsided by the challenges of managing an
economy because everyone was so greedy. He mentions how the admins came to
work one day to find that players had killed every living thing on the planet
and could no longer generate income. Kinda like a miniature "dark forest"
theory.

~~~
pradn
Computer games work best with numbers, so RPGs end up being about numbers. So,
naturally, people try to min-max because 1) it's something to do 2) you
progress that way 3) you can show off to others. Maybe a child would be
hesitant to kill a video game creature, but it seems like, since pretty much
every video game lets you do violent things, that taboo against violence is
erased very quickly. So, all these combine to, what looks like to people who
don't play games, a propensity to kill en masse.

It's a shadow of the same situation one encounters in an abbatoir. Maybe
killing the first few animals makes you feel terrible, but you get out of it
quickly. The human ability to adapt to any circumstance has this disturbing
side to it, too.

~~~
staycoolboy
Which is ironic given the nature of U4 and it's moral system. That game REALLY
floored me the summer I spent playing it (literally the entire summer, it was
before I was old enough to get a job).

> RPGs end up being about numbers.

For me, this was the final, sad conclusion to computer-based RPGs after nearly
40 years playing them.

In tabletop D&D, a +1 sword is a rarity, and +2 is super special; magic users
focused more on creativity because the cost to the caster for Lvl9 spells (if
you ever got there, which was never in real games) was prohibitive to use them
on a regular basis (memorization, physical aging). In fact, the level tables
in AD&D stopped around 10 or 12 I think because that much XP simply didn't
happen. Computer RPGs turned it all into numbers games, as you can see in WoW
where the top tier armor today has stats like +30,000 attribute, when in the
first release, top T3 geat from C'thon was like, +50 key attribute per piece.

Am I ranting? I'm ranting.

~~~
pradn
A lower variance in peak skills/weapons is a good way to go. Then, you could
feasibly have tons of weapon types, with each player picking the type they
like. I, personally, have always been partial to the rogue archetype,
especially dual-wielding fast weapons. I'd love to play that way. But in some
games, like Path of Exile for most of its existence, it's just not in the
meta. I'd love to be competitive, even 5-10% below the top tier, if my
favorite playstyle was possible. In PoE, you'd be like at 20% of the best
builds if you played that way.

Plus, in DnD you have a lot more scope for actually creatively role-playing if
skills and gear were less numericized.

------
eordano
It seems that SHARD is a term used already in 1988, meaning "System for Highly
Available Replicated Data"

Source:
[https://scholar.google.com/scholar?cluster=14914487445955020...](https://scholar.google.com/scholar?cluster=14914487445955020264&hl=en&oi=scholarr)

~~~
teraflop
As I pointed out a while back
([https://news.ycombinator.com/item?id=22974882](https://news.ycombinator.com/item?id=22974882)),
the "SHARD" system described in that paper didn't actually have anything to do
with "sharding" as the term is currently used. It was designed to replicate
data, but it didn't do any kind of partitioning; each replica stored a copy of
the entire dataset.

For that reason (in addition to the low number of citations), I think it's
very likely that the name is a total coincidence. Pretty much any word you can
think of has been used by somebody as an acronym for some project.

~~~
scott_s
I'm the person you replied to in that thread, and in support of your point:
after that discussion, I spent some time crawling through the proceedings of
Very Large Databases (VLDB) and the ACM Digital Library, and I could find no
instances of "shard" used to mean the partitioning of a database prior to
2001. (That paper is "Minerva: An automated resource provisioning tool for
large-scale storage systems" in Transactions on Computer Systems, free-to-read
at
[https://dl.acm.org/doi/abs/10.1145/502912.502915.](https://dl.acm.org/doi/abs/10.1145/502912.502915.))

 _Other the other hand_ , I found many papers citing the SHARD paper - more
than the official count. That's a difficulty with citation counts of old
papers: a lot of the papers citing it are also old papers, and we're not
consistent at tracking the citations of old papers. Personally, I don't have a
conclusion. The SHARD paper is decently cited, and its usage is close to the
modern one. On the other hand, I can't find any smoking gun pre-1997 usage of
"shard" in the modern meaning.

~~~
teraflop
Interesting, thanks for putting a lot more effort into answering this question
than I did!

------
dws
Google mentions sharding in a 2005 patent.
[https://patents.google.com/patent/WO2007011957A2](https://patents.google.com/patent/WO2007011957A2)

[http://www.startuplessonslearned.com/2009/01/sharding-for-
st...](http://www.startuplessonslearned.com/2009/01/sharding-for-
startups.html?showComment=1231392480000#c9054865638836100787) notes the term
in use within Google by 2003.

~~~
NelsonMinar
I worked at Google 2001-2006 and we definitely talked about "database shards",
particularly for the AdWords MySQL database. (this architecture has long, long
since changed). At least from 2002, and the database was partitioned far
before that. I remember at the time thinking the term "shard" was odd but
perhaps Ultima Online inspired.

I'm curious now about the history of database partitioning. So much of the
early work was about trying to hide the partitions, to pretend it was all one
big reliable system. It took awhile for folks to realize that was foolish.

------
pixelmonkey
This comes up from time to time. Here is a Twitter thread with some more links
about the various usages of "shard" in the context of early internet and
gaming services, including Ultima Online (UO), other early MMORPGs, and even
MUDs.

[https://twitter.com/amontalenti/status/615116231092371456?s=...](https://twitter.com/amontalenti/status/615116231092371456?s=20)

~~~
Raph_Koster
Everyone just called them "servers." Not shards.

------
mfontani
In the year 2000, my (telnet + MCP & MCCP & other acronyms-supporting) C-based
MUD boasted a _web_ interface.

With colors, buttons to move, and even _images_.

A web request is just a TCP socket, and browsers were gracious enough to start
displaying stuff as soon as it came in (so long as tags were properly closed).
So long as you sent stuff fairly routinely, the connection wouldn't be closed.

Frames helped... well, "frame" the interface: main frame would be the mud's
output; another frame for user input - both using a shared "key" to ensure
output and input were linked to the same user.

Users' input could therefore easily just be sent on another web request,
checked for key correspondance, and the command ran as if the user input the
same command via telnet: action -> response.

A sprinkle of that new-fangled JavaScript helped ensure the screen would
scroll properly as soon as new output came by.

Having previously abstracted all color-related stuff (to ensure strings like
"&WFoo&GBar&X" would send ANSI colours to users opting-in to them, and nothing
to users opting out of them) meant that it was "easy" to send "<span
color='white'>Foo</span><span color='green'>Bar</span>" to the HTML interface
instead.

Websockets? We don't need no stinkin' websockets! _really long_ polling
suffices ;)

------
efitz
I hadn't ever heard of the term "sharding", but the team I was on was
"partitioning" databases in the early 2000's (effectively sharding at the
table level so that we could drop table instead of deleting rows).

~~~
mywittyname
So my understanding is that there is are practical differences between a
partition and a shard. They both are ways to stored separately, based on the
partition key. But a partitioned table has one table, while a shard uses
multiple tables which share the same schema.

For example, with a partitioned database, you might have a single
`transaction` table, partitioned by date, but with a shard database, you'd
have multiple `transaction_${date}` tables which needed to be queried from,
but they'd share a common schema.

~~~
greenshackle2
Last time I did partitioning by date in postgres (on postgres 9.?), multiple
transaction_${date} tables is exactly what I got.

They are exposed as a single 'transaction' table but they were really
different physical tables.

My understanding is that partitioning is a more general term. Partitioning can
be on one or multiple servers. Sharding is a type of partitioning that is
always across multiple servers.

------
artemonster
queue in "stones theme" of the login screen and nostalgic goosebumps... The
article is quite old though - maybe someone from HN can provide clues for this
interesting coincedence ^^

~~~
outworlder
I've made people go back to UO just by playing Stones. I'm not sure what's up
with that song.

