
Ask HN: How do you keep track of the components you develop and run? - petr-k
What methods or tools do you use to keep track of the various components (whole systems, microservices, libraries, ...) and their interdependencies in your organization?<p>When the number of these components grows rapidly, it can become increasingly annoying having to search for these components, navigate between their source code, documentation, CI setup and history, recent log entries, deployment information (number of instances, endpoints), metrics, monitoring dashboards, etc.<p>I know Spotify tries to approach this with an in-house tool called System-Z (see https:&#x2F;&#x2F;dzone.com&#x2F;articles&#x2F;modeling-microservices-at-spotify-with-petter-mari). Have you seen any other interesting tools in this space?<p>Do you think it is beneficial to have tools that make this easier and show all the necessary information in one place (via UI or an API), possibly along with visualizing application dependencies? Or do you believe this is basically a non-issue, given how lots of organizations are moving into the cloud, where vendors can provide a lot of these tools for you?
======
BjoernKW
For most organisations traditional tools like IDEs, version control systems,
logging and monitoring should indeed suffice. A great IDE such as IntelliJ
IDEA and a tool like GitLab help a lot when it comes to keeping track of
source code dependencies and deployments.

Having a tool that visualises all dependencies in one place is nice to have
but it shouldn't be essential to run a system.

If you're running a larger number of services service discovery and properly
naming services is vital rather than being able to visualise them.

------
aprdm
We use rez for package management (libraries) and it does have tools to show
every package that depends on your package, or, every library you depend on
(and their dependencies) in a node graph.

We use consul UI to show all the services that we have, where they're running,
number of instances and so on.

We use ELK for service specific metrics / centralized logs.

We use Confluence WIKI for user / dev docs. Plus the README on the project
git.

~~~
petr-k
My point was, do you ever feel the need to integrate all these pieces of
information from the tools you mentioned, into one place? So that you do not
have to keep several tens of browser tabs open whenever you inspect a
particular set of services?

~~~
aprdm
Hmm not really because most of those things have alerts to warn us when
something is wrong.

Also, we have a Kibana dashboard which does have information from different
sources and it easily tell us if everything is good or not.

------
git-pull
For [https://devel.tech](https://devel.tech) and
[https://www.hskflashcards.com](https://www.hskflashcards.com), I split the
common JS, SCSS, and Python code into a git submodule.

I named my common library "based":

\- Uses reStructuredText + docutils to display content in a consistent fashion
([https://devel.tech/features/django-vs-
flask/](https://devel.tech/features/django-vs-flask/),
[https://www.hskflashcards.com/downloads](https://www.hskflashcards.com/downloads))

\- Common fabric source code for deploying Django websites, updating the sites

\- Customized crispy-form templates and SCSS files

\- Using Yarn "Workspaces" to load the common JS/SAAS code
([https://yarnpkg.com/lang/en/docs/workspaces/](https://yarnpkg.com/lang/en/docs/workspaces/))

\- In Webpack, I share common webpack code via wrapping the configs in a
function and passing in dependencies.

\- Common Python code is available to be accessed for reusable Django
applications, but also plain-old python

\- And use of "import_string" in django settings to load additional plugin
configurations [1]

[1]
[https://docs.djangoproject.com/en/1.11/ref/utils/#django.uti...](https://docs.djangoproject.com/en/1.11/ref/utils/#django.utils.module_loading.import_string)

P.S. I am writing an article (in draft atm) about import_strings in Django at
[https://devel.tech/tips/n/djms3tTe/demystifying-djangos-
impo...](https://devel.tech/tips/n/djms3tTe/demystifying-djangos-import-
strings/), intended for people new to Python and Django. You can also use
import_string in your own Django code to 'opt-in' to any sort of feature you
keep in a common codebase. In my circumstance, I use it as a way to register
docutils roles and directives via django settings. This pattern also holds
true without Django, too
([https://docs.python.org/3/library/importlib.html](https://docs.python.org/3/library/importlib.html)).

P.P.S. I highly recommend Yarn Workspaces. It's fast, and really makes it a
lot easier to share front-end code.

