Or they don't want to spend half their time managing that Jenkins ecosystem when some bash scripts and literally any other CI solution out there gives you very similar benefits for fraction of the effort.
My previous company also turned away from Jenkins to GitHub actions. I'm not an infra guy and the only complaint about the Jenkins pipeline is its slowness as it copies everything from the feature branch into the deployed branch, while only a few files were changed. But I don't think it's Jenkins' fault though.
Thank you.
I'm curious, what tool did you use before to track your services based on tags, and how did it work out for you?
I’m targeting startups/corporates that are over 5 years old but haven’t become large corporates. Employee count from 50-500. At this stage, companies often lose track of their deployed infrastructure, leading to significant costs. Typically, around 10% of the infrastructure is underutilized or unused. And beyond this stage, things are either sorted or way too messy. Additionally, documentation struggles are common, with only a few individuals aware of the full architecture. Democratizing this knowledge can be incredibly beneficial.
To your questions:
1 - The assummption is you already have some tags for your infra. Which can include can include service names, environments, resource types, ownership. And you start from there.
2- Resource could be machines, databases, load balancers, even object storage. If you run unmanaged datastores, you'd have an option to identify that infra. You are right, to be useful on an ongoing basis, we'd regularly have to churn up suggestions based on underutilization.
3- I am not sure this can even be done automatically, but plan is to provide tools to easily document all intereractions and business logic. And depend on service owners to create this mesh.
4 - This has been a constant feedback. Yes something like an approval workflow should do, or may be skip this altogether.
Interesting. Re 1: Wasnt planning on doing this, as creating diagrams thru tags will ensure that you have updates in real time. If you had access to such a tool, would you gives access to tag manager?
Re 3: Not in first MVP, but will be doing it.
Re 4: Fair. I suppose suggestions will be useful by themselves.
we have dedicated finance staff who compile all billing reconciliation end-to-end, any tags we need just gets added as a lookup to their datamart controlled by a unique asset tag. Very hard to justify giving access to compliance staff who always reject such requests
we have a solution based on plantuml deployment/networking api's where we generate static 2D diagrams as .svg from our datasets. We also have a basic 3d representation using python scripting and blender which exports to .gltf 3dmodels which we import into godot game engine web edition.
Wow. Thats some elaborate solutioning. I think the documentaion and cost attribution is pretty sorted at your shop. Does this solution help you save some costs at any point or the primary goal is only documentation?
Managing production IT infrastructure is akin to overseeing a construction project. It’s crucial to refrain from implementing any tasks without comprehensive diagrams. Maintaining cost control is of utmost importance for the longevity of a business. With GitHub Copilot, the automation of complex integration tools becomes significantly more manageable.
We are in the process of developing a freeware Excel plugin for the PlantUML component. This plugin will enable users to generate diagrams directly from spreadsheet templates. If you’re interested in trying out the beta version of this plugin, please send me an email and I’ll be happy to share it with you.
You'd be the primary target customer. The exisitig tools in market are way too costly and number 4 would always be risky without proper ACLs and integration with your existing deplyment framework.
The challenge lies in making architectural knowledge accessible and comprehensible to the entire team, not just a few.
This tool will aim to bridge that gap by providing clear, visual representations of your deployed infrastructure, which can make the complexities of architecture more transparent and easier to understand. By centralizing this information, you can reduce the reliance on a few key individuals and make it easier for everyone to contribute and stay informed. This way, the plan becomes more than just a set of tasks—it becomes a shared understanding.
I don’t disagree that there is value in the accessibility of knowledge.
I cannot read sheet music. I lack such experience. That experience is earned from practice. Likewise architecture is a skill earned through practice. My experience has taught me:
1. It cannot be communicated to the inexperienced developer because they do not understand it. Sometimes they will pretend to understand it and just superimpose a hasty, wrong, interpretation on top of it that just reinforces some singular aspect of comfort.
2. When architecture is defined in terms of goals it is better understood by business owners and project managers than inexperienced developers. They may not understand the code and conventions but they can frame the second and third order consequences in terms of measures, risks, and effort.
3. You can generally identify those capable of architectural discussion by where they invest their interests. When the conversation never rises above syntax and vanity, as in how to write code, the higher level considerations of organization and structure become lost and lofty.
Somewhat. Before I venture off on a tangent I just want to be clear that communicating architecture is a different goal than simplifying architecture.
Simple is not easy. In the context of logic simple just means fewer, or numerically less than something else. Please see this most excellent video by Rich Hickey, the creator of Clojure language: https://www.youtube.com/watch?v=SxdOUGdseq4
The primary problem with simplifying architecture is that simplification segregates a population. For people who are capable of understanding and communicating concepts of architecture simplification dramatically increases expressiveness and reduces labor. For people who have not earned comfort with architecture simplification imposes barriers to understanding. It's the difference between adapting an abstract subject into an essay versus needing to be told what the research material is as well as what words and thoughts are appropriate.
Consider subjects like lexical scope, functions as first class citizens, definitions via interfaces, tree models and taxonomies, and so forth. None of those subjects on their own are challenging, at least for most of us. Most people use these concepts outside of programming everyday without really thinking about it. The challenge then is to get people to think about these concepts intentionally. In most cases people not already comfortable expressing logic in these concepts will only do so if alternatives are removed. This is more challenging than it sounds because it requires separating people from competing institutions they are reliant upon.
In my case, just putting my own bias out there, I am self taught. I learned to program while traveling across Afghanistan, though I had already taught myself XML Schema the year prior. There were many times I did not have internet access so I could not simply access unlimited references or rely upon unlimited dependencies. I just had to figure it out logically and through trial and error. I also did not have prior teaching in the subject so I was a clean slate. The result was to do that which required less memorization and less formal education instead favoring that which was more visual and immediately expressive. Perhaps this is why I find DOM traversal amazing while most people are absolutely horrified by it.
Going on a tangent, this is inspirational. I dont know how you can travel thru Afghanistan, and gain expertise on these subjects. And express it with this clarity of thought.
Coming back, simplifying architecture will never be easy. And if that is the case, how can communication of architectures be made simpler? Is it possible to build abstractions and represent them as such to make it simple for most to understand? Are there any tools have come closest to solving this? In a world full of complex micro services.. someone has to come and solve for this problem.
I was called to perform a cyber security deployment in the US Army for a year from summer 2009-2010. I traveled across Afghanistan visiting various major US Army bases providing security assessments for the US Army's 1st IO Command. Active duty personnel were performing these missions in all areas that did not require family separation and used reservists for places like Iraq and Afghanistan.
The prior year I was involuntarily reassigned from a web design position to a front-end developer position at Travelocity. That year at Travelocity I was taking my first baby steps into programming, as required to retain employment, but did not really understand what I was doing. I learned use of basic functions and syntax. I learned much more trying to write original programs on my own in my down time in Afghanistan.
I suppose an excellent comparison to software architecture is public speaking or writing for publication. You know talent when you see it and it looks so incredibly simple when performed. The real challenge is attempting to communicate that simplicity in a meaningful way because not everybody is willing to consume the challenging material that allows for such talent. I don't claim to be talented, but I know what I have performed relative to other people. There are some people I can explain this to with immediate consumption. There are some people who are capable of listening to it, but find the effort required untenable. Then there are people who are not capable of consuming the material in any form.
Yes, they should. However, I've often seen teams lose track of all the services and components deployed. There is usually a dependency on one or two engineers, and when they leave, it introduces a significant knowledge gap in the team. Organizations are typically focused on growing the business, and it's usually only the cloud provider who benefits the most from the lack of visibility and optimization.
