Yep, you have it correct. I've got a note at the bottom of the README that I'm considering adding a config field to keep the .git directory, but I'm trying to keep pretty far away from git-in-git consequences/use cases. I said the same in another comment here, but I don't envision vdm becoming something that's git-specific or developmental -- it's really just intended to be a getter, not a writer, and the functionality reflects that.
> I've got a note at the bottom of the README that I'm considering adding a config field to keep the .git directory, but I'm trying to keep pretty far away from git-in-git consequences/use cases.
Maybe a better approach would be to rename the `.git/` to (for example) `.vdm` in each submodule? Each `vdm` command would first rename it back to `.git`, execute the git commands needed, and then rename it back to `.vdm/`.
This gives you the ability to implement `vdm history` or similar command while still keeping the submodule invisible to the parent?
The vdm submodule directories would probably have to be entered in the gitignore anyway. You wouldn't want to commit the submodule source files either. Won't the .git directory get excluded that way?
>This tool looks like "submodules, but lighter", while repo is "submodules, but heavier". Looks to me like the motivation is for dependencies that are not hard enough to justify a submodule.
I think you nailed how I was feeling in much fewer words!
Sure, but not all my code in a single repo is monolingual code, and not all of those languages have cooperative (or existing) package managers. Bash is a great example of something I wish I could stop copy-pasting between repos, and was actually the original motivator for vdm (along with protobuf files, for the same reasons).
Thanks! That AOSP `repo` tool is one I'd not heard of, so thanks for sharing!
I actually haven't really put much thought into collaborative/mutlirepo development work using vdm -- the original intent was for it to strictly be a retriever of what the user specifies. I think the majority of both my frustration and complexity of other tools is because they're trying to solve for a lot more than at least I personally usually want to use them for. It's like, I just want a `pip install/requirements.txt/go.{mod,sum}` for any kind of tool, not just the language that takes up the majority of my codebase.
One of the thoughts I had, though, was to maybe add a field for each remote section called e.g. `try_local` that takes a filesystem path, and if that doesn't exist then fetch the remote. That way, your development workflow is exactly the same as usual -- developing a dependency is just doing work in the actual repo directory nearby. I'm not married to the idea though. I just REALLY don't want to have it be in the business of managing git-repos-in-git-repos, because vdm isn't really intended to be a Git tool, if that makes sense.
The parent isn't talking about e.g. EC2 as a virtualized platform, they're talking about EC2 not being a virtualization platform. With few machine-type exceptions, EC2 doesn't support nested virtualization -- you can't run e.g. KVM on EC2.
That's really not that hard to add above. A lot of folks act like it's impossible to handle errors etc. in bash, but it's pretty straightforward -- certainly no more difficult than in any other language. The hard part, like with all languages, is deciding how to handle errors cases. The rest is just code.
On mobile so no idea if this a) looks good or b) runs (especially considering the command substitutions, but you could also redirect to temp files instead), but it's just something like this:
You forgot "-f" flag to curl, which means it won't fail if the server returns an error. Also "jq" returns success on empty input, pretty much always. Together, this might mean that networking errors will be completely ignored, and some of your data files will mistakenly become empty when server is busy. Good luck debugging that!
And yes, you can fix those pretty easily.. as long as you aware about them. But this is a great illustration why you have to be careful with bash: it's a 6-line program which already has 2 bugs which can cause data corruption. I fully agree with the other commenter: switch to python if you have any sort of complex code!
set -euo pipefail # stop execution if anything fails
cleanup () {
rm $tmp_file
popd
# anything else you want to think of here to get back to the state of the
# world before you ran the script
}
trap cleanup EXIT # no matter how the program exits, run that cleanup function.
But why bother? The moment you start doing all that, all the arguments of "oh look how much i can solve with my cool oneliner" goes away. The python version of that code is not only safe by default, it's also shorter and actually readable. Finally it is a lot more malleable, in case you need to process the data further in the middle of it.
for N, url in enumerate(Path("links").read_text().splitlines()):
resp = requests.get(url)
resp.raise_for_status()
prop = resp.json()["data"]["someProp"]
matches = (line for line in prop.splitlines() if "filter" not in line)
Path(f"{N}.data").write_text("\n".join(matches))
I'm sure there is something about the jq [] operator i am missing but whatever. An iteration there would be a contrived use case and the difficulty to understand it on a glance just proves I'm not interested. As someone else mentioned, both curl and jq requires some extra flags to not ignore errors, i can't say if that was intentional or not. It would either way be equally easy to solve.
There's real value in 'one-liners', though. A 'one-liner' represents a single pipeline; a single sequence of transformations on a stream of data— a single uninterrupted train of thought.
Chopping a script into a series of one-liners where every command in the pipeline but the first and/or last operate only on stdin and stdout, as far as is reasonable to do, is a great way to essentially write shell in an almost 'functional' style. It makes it easy to write scripts with minimal control flow and easy-to-locate effects.
Such scripts are ime generally much easier to read than most Python code, but I don't find Python especially easy to read.
I think it's very much your last theory -- used everywhere but not as interesting to tell people about as it might have been a decade ago. Queues are now Boring Technology(tm), and that's a good thing.
I never really even entertained the idea of agreeing with this until just this week actually, when I was putting log statements into bash functions that were supposed to return strings -- and failed for reasons I could not understand. Six full hours of debugging later, I realized that the logs were writing to stdout, and thus were cluttering the expected return values.
I'm still not fully sold on the idea, but I now have at least one tangible example of when you would want to keep stdout clear of commentary.
Cool info though, thanks for sharing!