
How Shopify Manages API Versioning and Breaking Changes - tomnewton
https://engineering.shopify.com/blogs/engineering/shopify-manages-api-versioning-breaking-changes
======
Judson
It's interesting to compare and contrast this method of API management with
Stripe.

As far as I understand, the Stripe api would continue to work indefinitely so
long as you lock your api version, whereas Shopify would eventually break the
app as they essentially backport breaking changes to older api versions.

Initially, I thought Stripe's method was superior, and would provide the best
API experience, but realized that Stripe and Shopify have different incentives
w/r/t their api.

For Stripe, breaking a functioning site harms revenue, and generally the
developer is the Stripe customer.

For Shopify, their customer is the store owner, and for the most part, the
store will continue to function because that is mostly controlled by shopify.
The developer api is for added functionality, and it is in the interest of
Shopify and the merchant that those apps continue to be updated and utilizing
the latest features.

So, two different ways of managing breaking changes, but both are ultimately
centered around providing the best customer experience.

~~~
mr__y
>those apps continue to be updated and utilizing the latest features.

while this is orthogonal to Shopify API or your post at all, since you
mentioned the need of updates, I just wanted to use that opportunity to vent
my frustration with the constant push to update everything all the time and
judging any piece of software by using "when was the last update" as a metric.

The problem I see is that not all apps or libs need (frequent) updates, many
(maybe most) do need them, but some don't. They provide some functionality,
they do that well and you could call them "complete". Maybe some security fix
could be needed from time to time, but with a mature code being in use for
many years event those are not frequent.

For example, consider something like ping utility. It does what it does for
many decades. There was a need to add IPv6 support, but that was almost two
decades ago. Why would anyone need to update it? I do not want any additional
functionality, I don't want it to send emails or have social media share
button. I want it to send ICMP echo requests and receive ICMP echo replies and
nothing more. Aside from some security fixes no updates should be needed for
10+ years. This utility is done. It should not be thrown upon just because
there were no updates for many years.

While of course neither ecommerce or Shopify platform are "done" and they get
many updates now and will get updates in future it does not mean that some
functionalities could have reached "done" stage.

For a "complete and done" addon, there could be a need for a security fix from
time to time. There could be a need for some adjustments if a major browser
introduces a new deviation from JS/CSS/HTML standards and forces everyone to
update their code. But those events happen from time to time, possibly not
that frequently. This means that some addon/plugin would not require any
updates during the periods between those events and those periods could be
many months/years long. But hey: "this addon did not receive any updates for
13 months, it must be really bad and should be avoided". This leads to a
situation where a competing solution with tons of bugs will look better just
because it receives two updates a week.

~~~
pc86
Well first of all I think it's a straw man to imply that anyone would want to
send emails or share to Facebook out a ping utility. That's sort of a big
thing that makes _utilities_ different than _applications_.

Edit: I deleted a couple sentences and realize now this might not convey
exactly what I meant. Utilities are easy to call "done," applications are not.
Applications interact with external forces who _do_ change constantly (other
software, business processes, law and regulation, etc). I think in general,
updating applications is a necessary thing, bordering on good, regardless of
circumstances.

~~~
pilom
I think the parent is saying that API's should be utilities and not
applications.

------
circular_logic
Glad to see that Shopify has better API versioning on their mind. When I used
there API a few years ago, it was one of the worst APIs to depend on. To the
point, we had to architect our system to alert us for unannounced breaking API
changes so we could fix and replay the JSON back.

\- Moving JSON fields in and out of nestings didn't seem to be counted as a
breaking change.

\- Changes were rarely announced, and there was never a changelog as to what
had changed (they look to have started one starting 2018 [1])

\- When we contacted support about a brake, they would often be surprised.

\- Often the only sign there would be a change would be that new fields would
start to show up before a larger change.

All this would happen every few months. Reading this article I can start to
see the reasons why this was happening.

[1]
[https://developers.shopify.com/changelog?filter=all](https://developers.shopify.com/changelog?filter=all)

~~~
nozzlegear
I maintain a Shopify API package for .NET [0] and this has largely been my
experience as well. Their attitude toward breaking API changes has caused me a
good deal of frustration in the past. To make matters worse, their docs
weren't (and still aren't, in my opinion) that great; my biggest complaint
being they typically don't document when values can be null or even have a
different type (e.g. property X could be a string or a decimal, but you'll
never know looking at their docs).

This has led me to taking the drastic step of making _every_ property
nullable. It's gross and feels bad to use, but at least it prevents JSON parse
operations from crashing applications when a value is unexpectedly null.

[0]:
[https://github.com/nozzlegear/shopifysharp](https://github.com/nozzlegear/shopifysharp)

------
uyuioi
The Shopify developer experience is terrible. No fluffy blog can change that
fact.

Shopify says the main product is the store owner. But the developers pick up
all of the slack of Shopify.

Recurring payments. App. Store backups. App. Theme backups. App. Order
editing. Came late 2019. Checkout. So locked down. Where’s the API?! Slate
tooling. Abandoned. Starter themes. Abandoned. Storefront SDK. Terrible
documentation. More than 1 variant image? App. Metafields. App. Wholesale.
App. Mailchimp. Removed.

People talk about google abandoning products. Shopify abandons nearly all
developer tooling and is so locked down that it’s a constant “app for that”
for the basics.

The interesting thing is theres been more than a few store owners I know that
use Shopify. They’ve asked how to move off of Shopify.

I guess fulfilment is more important though. Right?!

~~~
theturtletalks
Shopify's business plan is offer minimum functionality for a low monthly fee
and then users use the marketplace to add additional functionality which
Shopify takes a 20% cut.

I'm actually working on an open-source fulfillment and operations app for
Shopify:

[https://github.com/openshiporg/openship](https://github.com/openshiporg/openship)

I use it to build small apps that interact with the API directly instead of
paying and relying on any apps.

~~~
uyuioi
But Shopify hold the transaction fee and charge 2% on top. So really in the
end. Shopify is not that much cheaper. Considering bigcommerce doesnt charge
this. I am bullish on the long term of bigcommerce. Or anything that goes
after the small medium business.

I don’t think Shopify could sustain a real entrance by Adobe with Magento, or
a product in the same space from someone like Microsoft. As these companies
know developer tooling and in the end. Shopify needs developers. Developers
don’t need Shopify.

~~~
stickfigure
_Adobe with Magento_

Eh?? Maybe if they rewrite it from scratch and just recycle the brand...

~~~
uyuioi
They’re already heading towards Shopify’s lunch with commerce cloud. That’s
quite obviously the outcome Adobe is after. It’s a subscription business after
all.

------
MentallyRetired
Shopify made a change to their API that was easily measurable on who it would
affect, but didn't email us. Refused to grant us a temporary exemption (they
would do it for $2000/m they said). The end result? They've sunk my business.
I've replaced shopify now by writing my own but it's too late. My customers
have all gone to my competitors and we're looking at pivoting.

~~~
xal
Can you share more details? I run Shopify and that's not something we would
do.

~~~
MentallyRetired
Hi Tobi,

It was the change on limitations for variants. We woke up one day to product
additions failing. Took about a week for the tech team to figure out that we
were being throttled because we were exceeding 50k variants. Totally
understandable in the bigger picture of things, but I didn't feel we received
adequate warning. Developer support had told us that sometimes they grant
temporary exemptions but in our case they refused. They advised we upgrade to
shopify plus (they quoted us over $2000) to remove the throttling/limitation.
Financially it was out of scope for us, so we had to throttle our own
customers, which led to a massive disadvantage.

I ended up writing our own cart software, which was always part of the plan,
but in the meantime our business suffered.

I don't hold any grudge about it since we were getting incredible value out of
the previous arrangement, and I do think Shopify is amazing software. I've
been a paying customer in multiple capacities since at least 2008 and
recommend it to people all the time. What happened was just unfortunate timing
for us.

Thanks for taking a minute to listen, though. Much appreciated.

~~~
volkk
You couldn't pay the 6,000-8,000 for like 3-4 months and keep your business
afloat until you rewrote it? Kind of sounds like your business was really
really in the red already, or more information is missing?

~~~
jdsully
Not everyone is in the “money is no object” startup economy. Outside of tech
small businesses are actually small.

~~~
volkk
OP wrote that they have employees. if 6k-8k is too much for a company with
employees to literally keep it running, that means it was months away from
going completely bust anyway

------
keithwhor
Hi everyone. As the founder of an API company, this article has me wondering
about the semantics of versioning — so I thought I’d ask the community a
question. I hope the Shopify API team doesn’t mind!

When you all think about API versioning, what makes the most sense to you — a
semver approach (major.minor.patch) a la NPM, or a date-based approach
(2020-01-07) a la AWS? Or is some combination of the two desirable?

At Standard Library [0] we both allow people to publish APIs but also publish
API proxies on behalf of partners (Stripe, Slack + others) using a semver
approach. It’s not perfect but theoretically enforceable (schema parameter
additions can be forced to require a minor update, schema parameter removals
can be forced to require a major update). We’ve just stuck to this semver
approach based on intuition and haven’t had negative feedback about it, but I
do like the idea of time-based versioning.

Would love thoughts! If you want to play around you can build your own APIs
using [https://code.stdlib.com/](https://code.stdlib.com/), which uses the
FunctionScript specification [1] to enforce HTTP request schemas.

[0] [https://stdlib.com/](https://stdlib.com/)

[1]
[https://github.com/FunctionScript/FunctionScript](https://github.com/FunctionScript/FunctionScript)

~~~
thdrdt
Both don't say much without good changelog.

But with semver you can say: ok, I want every new version up to a new minor
version (all fixes). While with a date based version you don't know how
'breaking' the changes will be.

~~~
keithwhor
How often do you feel like you find / consume changelogs when you’re looking
at APIs? I feel like they’re either non-existent or non-obvious for most SaaS
companies.

Do you feel like you’d trust a company’s API more if you could peruse the API
changelog more easily?

~~~
thdrdt
It's not about trusting a company. It's about wanting to know what will break
if you start using a newer version of an API.

And sometimes you also discover new features that can be really useful.

~~~
keithwhor
That’s fair! From my perspective that’s something that’s building trust with
the SaaS provider. We’ve all heard, “ugh! This API is so shitty!” before — I’m
fascinated by what tools and products can be built to prevent developers from
feeling these pains.

Thanks for the feedback. Can definitely see that changelogs are really
underinvested in across the industry, and it’s useful to have people mention
how valuable they are.

------
petetnt
I want to love Shopify but Shopify doesn't want to love developers. For
example when their multi-location offering [0] _in beta_ they also announced
that the Inventory API is going to breakingly change in 2 months and _none_ of
their own SDKs supported locations at that point. This has happened with
Shopify time and time again. Shopify doesn't manage API versioning or breaking
changes: it just forces developers to update or endure with their broken
applications and interfaces.

Sure, it's Shopify's choice. But considering how long their own changes take
(for example multi-language is still in some sort of beta and it's been up and
coming for like 5 years?) the API cycles are just brutal. And the saddest
thing is that Shopify is still the best managed e-commerce platform for most
usecases.

[0]:
[https://help.shopify.com/en/api/reference/inventory](https://help.shopify.com/en/api/reference/inventory)

~~~
james_s_tayler
Where I work, we have found 3rd Party SDKs to be an anti-pattern for this and
other reasons.

------
ch33zer
How do they handle security fixes in old releases? If releasing a security
patch requires backporting it to 5 different active releases then I'm
unconvinced that this is a useful strategy.

~~~
auv5
Great question! As mentioned in the article, when making a change you would
typically add an `ApiChange.in_effect?` check to see if your new functionality
should execute. When we implement security fixes, we do not include this check
and the fix retroactively applies to all API versions.

------
etxm
Aside, the article mentions frozen_record and it makes me wish that DataMapper
had been a more successful project. I really enjoyed working with it.

------
pbreit
Just don't make breaking changes. It's really not that hard.

------
leome
Very nice story, this is why I decided to use Shopify API as less as possible,
so I won't need to care about the version.

------
lol666
shopify is slow it doesnt matter how they manage their api...

------
lol666
i know, i know, be constructive... but hey, shopify with +10k products is so
slow they can version whatever they want...

~~~
theturtletalks
At that point, I would use something like Gatsby which would build static
pages for each of your product pages. You could host that on S3 with a CDN for
cheap. You could downgrade the Shopify plan to $9/month since you're not going
to be using their storefront.

~~~
lol666
thank you, but this doesnt solve the speed issues in my case. i could use
sylius or any other ecom that i could host myself and use custom speedup on
code instead of just on cdn:) shop i mean is in uk and has only uk customers,
no need for cdn as customers arent spread globally or even on the continent..
as for static wouldnt that work for every solution? in that case theres no
need for shopify at all... but as you say i could even keep static code
generated and served from redis instead of s/hdd, that would be even faster

~~~
theturtletalks
The main power of Shopify is the dashboard and the APIs. By having a custom
storefront, Shopify just provides the APIs and a nice dashboard for non-
programmers to use. You can also ditch Shopify later on since the storefront
just builds off their API. Just my 2 cents. For $9/month, hosting something
like Magento or even Woocommerce would be difficult.

