If you think this is useful, you might also like codeatlas.dev and its Github Action (https://github.com/codeatlasHQ/codebase-visualizer-action). It currently does not support per-contributor activity, but we put a lot of effort into making the diagrams beautiful to look at and the basic approach of using treemaps for visualisation seems very similar. In fact, could be cool to collaborate on this, DM me if interested!
Are you sure ? I don't have a chance to test right now but from the brief look i had there's nothing specific for GitHub.com. as i see it uses the git log file.
Edit: based on your profile I see you're quite anti Microsoft for some reason. I think one doth protest too much in this case though.
The GitHub Actions on only references `/github/` and is only available through the proprietary forge. The gallery only shows `$USER_ORG/$REPO_NAME`. While seemingly subtle, it's this sort of behavior, treating Microsoft GitHub as the only place projects live, that is taking the distributed nature of Git and trying to centralize it. This is the same issue with Vim plugins giving priority shorthands for plugin names or npm having some shorthands like `$FORGE:$USER_ORG/$REPO_NAME` for dependencies, but if you drop the $FORGE you get GitHub as a default. This makes folks feel as if using an alternative host or self-hosting is inferior or less supported. This leads to newcomers thinking GitHub is Git. And that's not how distributed version control is supposed to be.
Yes, this is how critical paradigms like decentralization are gradually broken by Microsoft. Things are built and people have a brief look, but don't actually test them on other platforms. Don't criticize people by looking at their comment history, it's petty and frankly seems tribal. Write about the topic being discussed.
Looks pretty. I can't scroll on the gallery page on mobile, and tapping anywhere on the page (including the top example) just brings up the navigation menu. Switching to desktop mode works.
This is similar to the Structure dashboard in Repography [1] which is a web app I built, with the added advantage that it's kept up to date automatically so you can embed it in your README.md and not have to keep updating it yourself.
> Also shout out to the (mostly useless but cool looking) git history visualizer, Gource[0].
I'd say that Gource is actually pretty useful for figuring out where most of the effort has been concentrated recently! For example, when added to a new project, I might run it against the repo to see which packages in the project have been changed the most in the past month, what people are working on and so on.
Computing diffs is what takes large amounts of time as the object database is used intensively along with limited efficiency of object caches.
I couldn't resist and threw `gitoxide` at it, and it turned out to be more than 2x as fast (even though it uses way more CPU to do that, there is definitely room for improvement).
File "git-database.py", line 263, in <module>
main()
File "git-database.py", line 247, in main
last_commit = handle_commit(cur, lines)
File "git-database.py", line 178, in handle_commit
handle_match(cur, matches[i], commit_lines[2+i], fields)
File "git-database.py", line 197, in handle_match
p, n = secondary_line.split("|")
ValueError: not enough values to unpack (expected 2, got 1)
I wonder if this can somehow be used as a tool for quick visual inspection of architecture compliance. Maybe the size of the box could be selected before display to denote a custom metric. For example, if the size of the box for each file was proportional to the (number_of_lines_edited) * (current_date - file_creation_date), then modularity could demand that the size of the boxes remain small. (pardon the musings of a non software engineer on a Saturday night)
The size of the boxes is fairly customisable on the back end, the webpage just needs to be sent the right JSON data, and the only real criteria is that the size of each directory is the sum of the sizes of its children.
https://github.com/nixos/nixpkgs/ would be a great benchmark for a tool like this :) One of the larger repos on github, close to half a million commits by a large set of contributors to thousands of files.
If you are an org with a large codebase and want to get many more insights, there is Seerene, it's commercial but takes the idea of repo analysis to the extremes (not affiliated, just a happy user)
Great that is one of the tools I had somewhere on the back of my head.
Nice to see which parts are changing the most in a project to maybe see if it should be improved or at least to direct efforts of improving quality to these spots.
It would be nice if the heatmap showed addition, deletion, and possibly modification separately. I can't think of suitable visualization methods, but if I can see them individually, it gives more insights.
This is really cool. Some other things might want to be able to visualize are this + a recency filter, or this + treesitter support, in order to be able to query eg which classes or functions have dominated
I want to do some whitespace refactoring on the files in our project. File activity is useful for that since then I can change files that no one is likely to be touching.
I experimented earlier to push the limits of SQLite inserts and wrote a blog post[0] about it. We can apply some of the learnings here.
I reviewed the OP's code and did some benchmarks; SQLite is not the bottleneck here. The code first generates the commit info from the git log, prints that to stdin [1] and the python script reads from it one by one in a loop [2]. Each of the commit info is written to SQLite. So, with or without WAL, the time is almost the same.
To confirm my hypothesis, I ran the project without insert calls. On my machine, for cpython, it took 160 seconds and without sqlite inserts 159 ish.
I believe the git log will be fast anyway, so other ways to make it faster would be to read a bunch of commits at once and then do batch inserts. We can also make it run in parallel since each commit info is independent, and we don't need to care about ordering while inserting.
After cloning, I had to change the perms of the .sh files, install flask via pip and hope for the best. I wasn't sure what Python version to use, I used some 3.x. It would be really nice if this was fully containerized and I could launch in docker/podman.
Thanks for the feedback, I'll try to add instructions for the sh files and the python version. This is still very much a WIP so I'm not going to look into docker just yet.
For the highlighting, currently only a single email pattern input is supported. Type in your pattern into the box at the bottom, then click submit.
The pattern gets fed into a LIKE statement in SQLite, so just plaintext, with % representing a wildcard eg torvalds% would highlight files modified by an email starting with torvalds.
I'm not sure the colouring is working as expected; I tried submitting with many email addresses I know are in the repo it ran over (with/out the wildcard) and the highlighting behaviour was difficult to predict.
https://codeatlas.dev