I don't understand why people use config servers like this, instead of keeping it in version control. Keeping it in the repository allows me audit changes of everything. If you move it to an external service, you lose that. And even if the external service has an audit trail, now you have two sources of version control. My assumption why people would do this is because 1) they're told that's how it should be done 2) they have a slow deployment processes, where it takes too long to get the change merged & deployed. Anyway, I've very happily never used a config service, and I still remain unconvinced. :)
Hey thanks for the insightful comment! I'm sympathetic to your thought here on having an external service and going with a simple approach.
Situations with a slow deployment process has been a big motivator for building Config.ly -- and I've seen it happen at companies with medium-sized eng teams (taking hours to swap values during incident fires), and I know it's a problem _all_ iOS developers have due to the app store deploy process.
I also think there's other value here... For instance having a UI means non-technical folks can update it. A lot of my time as an engineer has been spent making copy tweaks (I've felt existing CMS products were too heavy-weight and difficult to plugin).
Also, having turn-key libraries mean you don't have to copy/paste hardcoded values among many clients (or store on the server build an API for _every_ constant). I'm a big fan of DRY.
Big +1 for making it easier for non engineering folks to update the copy strings via UI. This is something very useful for product teams at mid size companies
Some sort of dynamic config management existing via spring cloud or homegrown systems like configbus - easier to implement for larger tech companies, but the small and mid sized may benefit a lot f
in their development speed from this off the shelf solution I believe as it evolves further
Further, a lot of "configuration" can just be the environment (context). Isn't that precisely what DNS is for?
An instance of an app (process, service, whatever) doesn't need to know that it's "dev", "test", "prod". Just have it look for "the database" and change the /etc/hosts (or equiv) as needed.
My impression is that istio (?) is supposed to formalize this approach. That'd be nice.
I'm so done having this conversation with teammates. Yet another reenactment of the "why use version control" war. There's always some stubborn refusal to join the third millennium. Because reasons. It's too complicated. No one knows how to do that. It's hard to debug. Whatever.
Every new cohort believes they're the first to invent configuration management.
Oh yea, mobile apps are totally a great use case. But honestly I would probably just toss a quick route for `/config` in my backend server that returns my config JSON for the app--and then boom we're back to having version controlled configs in my git repos. And obviously, make sure in the app to cache the last version incase there's no internet connection, unless maybe the app requires an internet connection.
I could be wrong, but I think some folks would want their other clients -- even their server -- retrieving that value from that one single source. So when you roll an Android app, you'd also want to implement HTTP + caching... And basically you're on the path to building your own Configly.
It’s really not so hard to roll yourself IF you don’t need a web interface.
In my GraphQL api I have a resolver called appConfig. That enables me to fetch the config I need for a screen in the same request as the data and to reuse any caching/offline/http logic the app already has. Zero added latency. No service can beat that, but then it‘s also just me editing postgres to change the config.
I think even if you do need a web interface, there are enough plug and play tools that will give you something that works reasonably well. That isn't to say that a service like Config.ly couldn't do a much better job.
We do both, to a point. We use Hashicorp's Consul to push config changes to running systems, but we sync it from a repo using an internal git2consul type thing.
It's nice to be able to push things into the system while it's running without hopping through all the hoops of qa, staging, etc. We then usually move it into the applications base configs in the app repo if it's going to stick around for more than a day or two to keep things tidy.
There's nothing wrong with a service to push config into a system at run-time, and you don't have to throwaway the benefits of the git history.
Do you store your API keys and other sensitive data with a site that doesn't even have a page discussing their encryption or security practices? Their privacy policy mentions they secure data with SSL protocol...
Who has access to each client's database? Is it audited? Is it encrypted at rest? I'm sure it is, but Config.ly would be wise to add this information to avoid fears.
Also you can store encrypted secrets in Git just fine, there are a number of methods to do so very safely.
Thanks for the feedback. The goal right now is not to store sensitive data in Config.ly - your read API keys will be on your clients - and so in theory anyone who can read that source code can fetch your keys.
> Who has access to each client's database? Is it audited? Is it encrypted at rest? I'm sure it is, but Config.ly would be wise to add this information to avoid fears.
Vault/k8s secrets for sensitive data--but you know, it really depends on the context for sensitive data, there isn't a simple answer to say what I've done across the board
Berglas is great in that regard, as you can keep the unique names to the sensitive data in version control, but have the actual values sit in behind an acl in a secure location.
You’re able to guard secretes as need, but keep the audit-ability of version control.
We store the vault paths/fields/versions in git and they're dereferenced outside of git (either directly in the software or in an intermediate deploy step).
sops is a great tool for version-controlled secrets - plaintext keys and encrypted values using remote KMS to do the encryption.
https://github.com/mozilla/sops
I’m Jeremy, cofounder of Config.ly. It’s like a CMS for your static variables / copy / constants so you can easily update them from a web UI instead of deploying code.
We’re software engineers and saw that the source of many bugs, incidents and time-sinks stem from hard-coding data. For example - waiting a day or longer for an iOS app store approval of a copy change, waiting hours on an internal CI/deployment process to bump a timeout during a traffic spike incident, or having the same dollar-cost value diverge while hard-coded on iOS, Android and web clients.
In an ideal world, data would be completely separate from code. Databases can do this but often aren’t used that way for good reasons (they can be cumbersome to wire-up, there are scale concerns about adding extra load to your DB, it’s risky to touch a production database, etc). So we built Config.ly.
It has a simple web interface to define Strings, Numbers, Booleans and JSON objects and arrays. We think it’s so simple that even non-technical folks can update basic Config data like copy and colors (so you can focus on code!). We have client libraries (in four techs and growing!) that fetch these values from the server and intelligently handle optimizations like caching.
It’s free to use (with genorous size and bandwidth caps) and getting started from sign-up to fetching values in your client should take < 5 minutes.
I love this idea. I will express the ups and downs of it, and see if I'm missing something.
First, some background: I'm a heavy Django user. Django has settings that are loaded once at runtime and usually they're in version control (they're committed to the repo). That means that, if you want to change a value to a setting, you must commit to the repo and re-deploy. If you're using Kubernetes, this might take several minutes (re build the images, upload to the registry, replicate on all the pods, etc).
That'd be the advantage of this solution. Imagine you have to change a setting IMMEDIATELY (something bad happen). You must:
* commit to the repo and push
* wait for CI (tests, formatting, protocols of PR approvals in a team, etc)
* wait for CD (build image, push to registry, replicate, etc)
It can potentially take several minutes.
Now, the major drawbacks I see with this service are:
* Security: do you guys feel confident/strong enough to store important data of the app? Go so far as hosting secrets for example? (We use the Secret Manager in AWS).
* Latency: in our case we read confs only once when the WSGI server starts. Can you be fast enough?
* Availability: if your site goes down, my app goes down. Can you ensure 99.9999 availability?
Anyways, this is definitively a needed service. It does have its challenges, but you guys are onto something for sure. Congrats!
Thank you for the kind words and really thoughtful comment!
If we build a Python / Django library, would you integrate Configly? If not, what would it take?
I think you hit the nail on the head for a core use-case: deployment processes inhibiting quick data changes. I believe the problem gets worse as you work at larger places... I've worked at [unnamed SF Tech Company with ~1k engineers] where deploying took ~a day simply waiting for the CI along with a backlog of commits from other developers.
I really like the challenges you call out. We don't do a good job of saying this on the site but we explicitly don't want h manage secrets / sensitive data; especially because the API_KEY is on the client libraries - so there isn't really anything stopping anyone from peeking in memory / the source making requests (I think this is different than say a more traditional login-auth system where the secret is stored in a session and only the person with access to the client with that session can access that data).
100% agreed on latency and availability. Config.ly needs be super solid here (sacrificing of course some consistency... slightly delayed updates). Besides following best practices for building/designing/operating high-avail systems (both my cofounder and I have experience here), there are some clever things we can do on the client. For instance, caching should help with both and I thought some of commenters below mentioned had great suggestions for other sorts of clever client fallback mechanisms we can do to help here [1] [2].
So this is a static file server that one can upload a JSON file to? It would be less prone to errors if the JSON was auto-generated based on a dynamic form so the user doesn't have to worry about getting the JSON syntax right.
Firebase’s reliability issues forever fired them from my arsenal. Their uptime dashboard regularly lied about being available, as well. The coup de grace was when they went down during a demo to a customer where I had put in 20 hours of prep beforehand.
Configly looks like it would fill a niche of needing to give levers and dials to a non-technical customer who contracted the technical coding of the app/product out.
When you do, it would be interesting if the form could be generated with specialized widgets from a JSON Schema Validation.
It would probably be a lot of work, but would make life a lot easier for people who use that. Automatically populating enum dropdowns, numeric range enforcement, etc. On the other hand, you wouldn't need to make your own UI to edit the field typings since many JSON Schemas can be generated from code, or with existing visual tools for people who prefer visuals.
So if I understand correctly, I can define bits of JSON / values in your interface. Then when my client app loads it'll hit your service backend to fetch them.
Seems pretty interesting. In terms of how it works it seems similar to how LaunchDarkly fetches its feature flags.
In practice if we configured all small bits of data in here it'll happen at app startup and be on the critical path. Do you have some sense about the latency there?
Hi. Today, our clients fetch values on-demand (so, not necssarily on app startup - it's when you call configly.get(key)). The values are then cached on the clients according to a TTL you can set yourself.
Latency was around ~200ms (I believe ~40ms is on the server). Do you feel it's important to push this number down?
I think LaunchDarkly might do some background fetching. We were slightly concerned about additional bandwidth / battery usage on mobile for this pattern and aren't quite ready for a push pattern.
Speaking for me, a 200 ms delay would be too much when values are fetched on demand. And I assume it‘s higher outside the US?
Imagining a mobile app, adding a 200ms delay to a screen transition that would otherwise not fetch data would not be ok. If it fetches other data, it may be fine. I would rather have the relevant values fetched in the background before they are read.
Because of the latency sensitivity I would probably try to hack this together on cloudflare workers+KV if I needed it. I would expect <100ms latency in most cases on that.
I think this is a general worthwhile service, especially the focus on CMS/copy over feature flag tools, but low latency would be something I would look for in a feature list.
That's something we can definitely give more priority to.
I actually had an iOS library version that would poll infrequently in a background thread and had a synchronous API. Something like:
Configly.init(API_KEY, { values: [keyOne, keyTwo]})
// (other code runs)
print(configly.shared().get('keyOne')) // would be cached by the background thread
but was slightly concerned about unwanted bandwidth/battery usage for mobile users but perhaps that mode would be useful exactly for the situation you describe.
(I don't see an option to reply to your other comment -- I'm guessing it's too deeply nested, so replying here)
>I think if battery/data even becomes a concern, you have already put too much data into config and probably need a database?
> Does anyone really care about a couple hundred KB (at most) these days? Or am I underestimating the config size/usage?
I could be wrong but I think this is a concern some mobile developers have. It came from user feedback but we should definitely look into it more.
Re replying, more likely the issue was the comment was too new, HN doesn't let you insta-reply as a means to make it harder to get into a minimal-thinking flamewar style argument.
This is going to come across as dismissive, I don't mean it to - I'm sure im missing something.
Is this more than a key value store with TTLs? Is it the UI over the top, the CMS that differentiates this from something like redis?
Hey! Doesn't come across as dismissive to me at all. It's a good question.
You could think of this as Redis hosting (configured to be durable!)
+ UI with features like version history, lightweight type checking (and potential for user accounts, schema enforcement)
+ client libraries on a variety of languages for super easy install / integration with all of your frontends. The libraries are smart with things like caching
+ a team focused on improving this feature set / functionality.
I was just thinking that what we really want when we say "no-code" is the ability to think in terms of domain specific data structures instead of programming constructs.
How does the offline story look like? At a first glance, it seems things would not work if you're not connected to the internet or don't/can't have a connection to Config.ly
If you try to fetch values and there is no connection nor ached values, the client won't be able to hit Config.ly's servers. I think it's a good idea to implement Config.ly with graceful degradation if you are concerned about this case (e.g. "configly.get(key) || "default_value"); though it should only happen in this specific use case.
If you lose internet while using it, the values could be cached and it will work well.
We have a graceful degradation feature on the roadmap; something like hard-coding fallback values in case of offline issues like this. Something like:
Perhaps instead of hard coding the fallback, offer a Webpack plugin that fetches the latest config at build. That way long as you've got internet at your build step, you'll always have a recent version.
Support something like local, checked in .env file (dotenv) but with the format you use in your service. Add in export from the UI so you can download it to local before going offline and it's a pretty good fallback.
Hey! Config.ly Co-founder here. I agree that utilizing a database/S3 or rolling your own lightweight version is a big competitor to Configly. I'm not sure if you saw it, but I tried to highlight some of the reasons we think Config.ly is more effective over a DB for certain usage patterns in the intro note [1]
> Databases can do this but often aren’t used that way for good reasons (they can be cumbersome to wire-up, there are scale concerns about adding extra load to your DB, it’s risky to touch a production database, etc).
I can think of other benefits; you generally don't want really anyone touching your prod DB directly -- but _especially_ non engineers. So what if they want to update copy? You could build or leverage a web UI... but now you're going down the path of building Config.ly. You also don't get version history for free with most databases. And once a value is in the database, you also need to worry about the middle layer of getting that value to your clients and having your clients fetch it intelligently/caching, etc.
Cool! Java is coming soon for Config.ly. I think we view having many client libraries as an advantage over something like Spring Cloud Config that I believe only has Java implementation / HTTP -- since many folks have clients in several languages. I could be wrong, but I also don't think Spring Cloud Config has a fancy web UI with version history nor caching (so you can let PMs update copy instead of interrupting developers!).
The hope is that these and more features for Config.ly along with the fast integration will make it a no-brainer over rolling your own implementation as many folks do today.
By default Spring Cloud Config uses git repositories as its backing store, so version history is more or less handled. You can also use file systems, Vault, JDBC, Redis, AWS S3 and CredHub as backends out of the box[0]. You can also mix and match backends, so that (say) regular config goes into Git and secrets go into Vault or CredHub.
It ought to be straightforward to add Config.ly as a backend, which would give you a pathway into a very large developer population (both Java+Spring and .NET via Steeltoe[1]).
For GUI cases, Spring Cloud Azure[2] adds Azure App Configuration[3] as another backend.
Disclosure: I work for VMware, which sponsors Spring development, but not on Spring.
Looks nice and simple. There's been so many attempts at these things: configuration management, api key management, secrets management.... They're all basically the same thing... But I guess the same can be said about a lot of tools
Hey - thanks for the comment. The vision for Config.ly is slightly different than what I think you described. In my mind there's "ENV VAR" which are secrets and has things like Doppler or many of the other things mentioned here.
I view Config.ly as being useful for _anything you'd put in a class variable_. For example text copy or styling. And if you put it in Config.ly instead of hardcoding, you can avoid deploying which can be slow (e.g. for iOS it's days) -- and potentially even avoid having developers do the work altogether.
Does that make sense or do you view it differently?
How easy is it to have different sets of values, running at the same time, so that we can point the client to one of the sets ? E.g. each set can contain a "theme" or can denote a client.
It's very easy to do that. The data stored in Config.ly are key:value pairs and can be a JSON string/array, String, Number or Boolean. So you can create as many keys as you'd like.
I'm not sure if that answers your question. It may be referring to slightly different functionality... We don't yet have functionality to say "for this key, deploy value x to clients a,b,c and value y to clients d, e, f" but it's on the roadmap. Do you think that'd be important?
Hi! Thanks so much for your comment! Could you expand a bit on what you'd like with A/B testing?
All of the values can be fetched across projects / clients, it's essentially a simple K:V store with broad libraries. For example, if you have a key "price" whose value is a number "100", you can call:
configly.get(price)
from all your clients (provided they are all using the same API KEY).
Today, clients download at the granularity of an entire key-value. So if you changed that price to "150", it'd fetch the entirety of "150" and not just the digit flip. Were you envisioning a case of storing a lot of data?
Today, our free service is capped at 100kb total. Was there a use case you had in mind?
A/B Testing for a K:V, key(“button-color”) = “red” for 10%, “green” for 90%
In a Enterprise, where there are multiple teams wanting to manage their own K:V store, having a global K:V is helpful. Also being able to refer to a project to refer to their K:V
if key(“price”) = “100” is set in Global and one of the projects is setting it again to a different val or same show a warning that you’re rewriting that.
You definitely could use S3! I think I view some advantages of using Config.ly as:
- Updating content is way simpler. Even non technical folks can do it. (and soon you'll have revision history)
- I think with the S3 library you can use HTTP caching; Config.ly can do smarter caching (e.g. if you fetch (a,b) and then just a, HTTP caching would fail).
I've seen this built as an in-house tool at every major tech company I worked at. Can't believe this hasn't been built as a standalone product yet. Imo, there are 3 main use cases that make this kind of system a must-have:
1. shipping critical changes that can't wait for CI/deploy pipeline. If you're at a company with more than 5 engineers or so, it probably takes you anywhere between 15 min to multiple days (for mobile clients) to get code from your laptop into production. In time-critical situations, this can be unacceptable. Maybe I have an alert banner that I need to show to all users NOW, (or perhaps a configurable subset of users). Having an online tool where I can change the alert_banner config from null to "We are experiencing outages due to an ongoing AWS issue. We expect a resolution by 3pm PST" is something that cannot wait for that deploy cycle. And sure, I can store it in S3 but then only people with AWS access can set it, which is quite problematic.
2. Non-engineers need access. Maybe I want my marketing person to be able to change the copy on my landing page. Rather than having them bug me every time they need a copy change, I'll just hand the keys to them, without giving them access to the code base or AWS
3. Nuanced codepath management (I don't know what else to call this). Feature flags are great, and every company has some system for this. However, I sometimes find myself in situations where I want finer-grain control than simple boolean flags. An example that came up at work recently: I was working on my company's zendesk integration. Inside of our zendesk webhook, we set a bunch of user tags every time that a ticket gets filed (maybe 30-50 tags). We found that one of the tags ("company_size"), which depended on clearbit to populate, was throwing errors and returning 500 due to frequent clearbit timeouts and improper error handling in our code. This needed to be resolved immediately. The ideal setup here: inside my webhook code, before executing any business logic, I pull a "tags_to_skip" config, and any tags included in that config never get populated. This would allow me to update the "tags_to_skip" config from [] to ["company_size"]. Realistically, we're not going to have a feature flag for every tag. And even if we did, it would not be ideal, because there'd be nowhere we can go to see the list of all tags that are currently being skipped. Eventually we threw too many errors and zendesk turned off our webhook, which caused major headaches for the support team.
I'm not sure if I did a great job of explaining that last use case :/
This type of tool is one of those things that you don't think you need, until you see it. Once you see the tool, you start seeing countless places in your work where you realize that a tool like this is extremely powerful. I just played around with config.ly, and I have to give major kudos on the ease of setup and use. Really great product!
