Not trying to sound condescending, but do you have any working knowledge of containers?
That's why I said you need to go revamp the design of your app first, going from a "monolithic design" to a close-knitted, loosely-coupled "microservice-based design":
- split the functions of your app into pieces, do that on paper
- draw the interaction between all of those components with arrows, to see the inter-dependencies, see what functions need to read/write from a database and which need to poll another function for checking the state.
- read a bit about sharding (both app and database).
- start "breaking" the above functions into separate entities, containerize them (see what runtime you need, what extra libs, what OS would you prefer in case you need to troubleshoot, make sure you won't have a multi-gigabyte image, optimize as much as you can, you can do it later but you won't, let's be honest, that's how technical debt comes into play) and try to have them play nice with each other.
When you managed to do that, try to use docker-compose and do some testing, to see how the performance is doing, bugfix, do some more testing, then some more bugfixing afterwards and then, when you're happy with what you have, read about Kubernetes / Openshift / EKS / AKS and do an analysis on what you need from a container orchestration system, have some requirements then compare those with all the above examples I've wrote and get the right mix of cost per functionality.
I can't really recommend any reading for it, there are a lot of resources on Google for the theoretical stuff, but to put them in practice, either bring in someone experienced from outside your organization (if that person doesn't exist inside of it) or try to put together an innovation team that'll do this, but with a lot of "trial and error" sprinkled all over it.
OK. Looks like I did not make my question clear, sir. Here is the thing:
1. I have a huge monorepo for the backend, written in NodeJS and Express. I have used Bitbucket (Git) for VC. Inside that monorepo, there are 5 application drivers. Those drivers perform their own unique tasks.
i. Validator - validates txs
ii. writer, etc. You probably got it I'm sure.
Now, I have written the CI/CD pipe already. I have a private Docker Hub repo. There in, I have pushed that entire monorepo from Bitbucket. The bitbucket-pipelines.yml file is written for the whole monorepo and not for individual applications drivers that I mentioned above. Now, there is Docker Image for that entire repo. The thing is, in the private Docker Hub repo, I want to store them as separate images - inside that single repo, because right now, I can't separately access them. Those 5 of them I mean. However, I can't separate those drivers as micro-services of their own ( I mean in separate repo because they share NGINX, mongodb, logdna, etc).
My question is:
1. Is it possible | logical to write 'Dockerfile' inside each application driver and would that suffice me to see them as separate images inside the Docker Hub repo?
2. Or is there any other way I could do it?
3. Can I use docker-compose.yml file which is available for the monorepo as a whole to connect those drivers ( using the Dockerfile inside each of them?)
