Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: git-bug – Distributed bug tracker embedded in git (github.com)
447 points by michaelmure on Aug 17, 2018 | hide | past | web | favorite | 94 comments

This is interesting, a few disconnected points:

- You should post this to the Git mailing list. We don't bite, and a lot of people would be interested in the details of how it works there.

- If you haven't you should check out git-annex. It solves an entirely different problem, but much of the distributed custom merging of histories, using git for ad-hoc metadata etc. is the same problem you're solving.

- You should be aware if you aren't that the design of having a ref per bug has some drawbacks, some operations in git are O(n) the number of refs.

- It would be really interesting to see some write-up on how the format you're using compares to the format Fossil uses for its bugs. I think some of what Fossil does wouldn't be possible to do in git due to its simpler data model, but still, such a comparison would be educational.

I have always wanted something like this without having to move away from Git (to Fossil).

A bug tracker like this allows the bug history to be tracked and remain with the source code which is much better than keeping the bugs in a web based service because every 5 years you need to migrate it (Bugzilla to Trac to JIRA) and in every migration, some context or formatting is lost.

For something like this to be successful in enterprises, the tool should be management-friendly. Right now JIRA leads because it allows management to pull pretty reports out of it. Is there anything planned in git-bug to appease these management type people so that it has a good chance of taking off in the enterprise world too?

Fossil's fantastic. The next release includes (in addition to their extant version-controlled tickets, wiki, and blog) discussion forums. https://www.fossil-scm.org/index.html/doc/trunk/www/forum.wi...

Are there any active Fossil instances available for free/open source projects? I found Chisel, but am not sure whether many of the projects are active.

Also, the design of Fossil and Chisel is s bit difficult to read on mobile.

If you click through Chisel you can find a few active projects. You're right about discoverability though; It's not great.

Fossil's neat because it really tries hard to be decentralized. Because of that the repo is really all you need. Hosting isn't so important when code, bugs, forums, and wiki can be moved by copying one file. The downside is that there's no FossilHub with thousands of projects using similar UIs and conventions.

As far as Fossil hosts go, I know of at least one more: https://secure.hydra.ecd.space/

I think one issue is that it's really easy to set up a public Fossil instance. It takes about two minutes to get it running if you've got a web server with CGI support.

If you mean active Fossil repos accessible over the web, there's Fossil itself (https://www.fossil-scm.org/index.html/doc/trunk/www/index.wi...), and SQLite (https://www.sqlite.org/cgi/src/doc/trunk/README.md). If you mean Fossil repo hosting, I know of Chisel but haven't used it and don't know anything about it. I don't think it requires much in the way of hosting though.

Edit: corrected SQLite URL.

or the mORMot ORM/REST framework: https://synopse.info/fossil/timeline

That is some phenomenal documentation.

> Also, the design of Fossil and Chisel is s bit difficult to read on mobile.

That's just the default. You can easily change the CSS, though it would be nice if you could select from themes created by others, particularly to get one that's friendlier for mobile users.

Looks like it exposes a GraphQL API - should be able to pull all the data you need out of it. Sounds like an excellent contribution opportunity :)

Upvoted for Fossil reference.

Now we just need to be able to also put a wiki in the same repo.

I do this with vimwiki: https://github.com/vimwiki/vimwiki

Can you describe your setup? I also use vimwiki but separately so interested to learn from your experience.

It's really just:

  mkdir Notes/
  vim Notes/index.wiki
  git add Notes/index.wiki
See ":help vimwiki-temporary-wiki" for details, but basically, it recognizes the Notes directory as the root of a wiki upon opening the file.

Since it's just tracked as a normal part of the repo, it gets branched and merged along with the code. So for example if I make an experiments branch, any wiki changes regarding the experiment won't sit around polluting the wiki if I decide to drop the branch. Similarly, any changes relevant to the branch I am working on get merged together along with the code merge.

If you missed it on the readme, I wrote a document that explain how it works behind the scene: https://github.com/MichaelMure/git-bug/blob/master/doc/model...

This is a nice idea, let me ask you a question:

One of the more interesting projects now a days is ForgeFed (https://github.com/forgefed/forgefed) which would allow for projects to use ActivityPub for pull requests, forking, etc.

This is important because so much of the value add of Github isn't in the git hosting (which one can do with Gitea or Gitlab) but in the collaboration.

This is also important because many Github extensions work through their API to glue functionality such as test suites and linting through monitoring a PR.

So where might your project fit in there?

The problem with this approach is that usually your work computer is not reachable from internet, so ActivityPub won't work here. It might be used to create a federated GitHub though, but it's outside of the scope of git-bug for now. The project is just one month old ;)

The way you could bridge the two is that you'd have something to monitor the bugs in both directions (gitbug -> AP and AP -> gitbug).

The argument that connectivity is an impediment is valid, but increasingly less so.

The problem with storing bugs in git is that you are forcing the users to download the entire project, or to make a PR for each comment, etc.

So a bridge would make things much easier.

I love this because I started building a small utility to help me consistently track issues inside my project files like tickets/{folders}/{file }.md inside my projects, because I do not need an enterprise level management system, and I also really like the idea of tickets moving and closing as a part of merges, IMO it provides a better context when looking through history.

This is clearly more elegant, though it adds a new dependency, which is OK.

>I also really like the idea of tickets moving and closing as a part of merges

I really like that, too, I wish we'd track our requirements in git instead of having an IBM enterprise solution which is slow, has a terrible web interface and no ability to diff requirements properly.

The more information is versioned in the same system, the better the coherence of the information.

> I also really like the idea of tickets moving and closing as a part of merges

So does that mean that bug reports are tied to branches? (I.e. depending on which branch you've checked out, you see a different list of bugs?) I wouldn't like that idea at all because how would you keep track of bugs (and related discussions) across branches? (For instance, having 10 branches would imply that you suddenly have 10 versions of the same bug discussion, too.)

It does pose a significant problem.

I see more value in it for clear technical tasks with technical descriptions, and not so much for discussions. Then proposals can be peer-reviewed before accepted as work that needs to be done (as a merge request to master, for example).

Then another problem that comes along with that, especially for golang users, is that your master branch is getting bombarded with issue updates, causing what looks like releases, when there's no effective change.

Not necessarily. All 10 branches could refer to the same discussion.

With the rise of static sites I'm not sure why something like this isn't much more common. Would love to see more options like this. Maybe eventually (not holding my breath) different git hosting providers will allow exporting to formats and importing to ticketing formats, so you can go from cloud to local instance or vice versa, or store tickets for mirror purposes.

Maybe someday git projects will have a /tickets folder or branch? Tickets per branch... hmmm... Maybe ignore the ones not in Master

There is also Distributed code review system for Git repos: https://github.com/google/git-appraise

That’s super rad. Thanks. Now something I can use with Keybase.

Keybase already has git integration: https://keybase.io/blog/encrypted-git-for-everyone (now sure why would you want Keybase but okay).

I imagine they meant it's a way to do code review for projects they currently host on Keybase.

Correct, thanks.

And http://mrzv.org/software/artemis (which I use)

BugsEverywhere was broken last time I tried it (one of its git submodules wasn't resolvable IIRC)

You might be interested in SIT[1] then. SIT has an issue-tracking[2] module which is comparable to git-bug in its scope.



Bugs are "retroactive" - you discover a bug, and then go on to determine the earliest point it was present.

Given that, I am looking for an extremely niche but vital feature: the ability to keep track of which branches a bug has been fixed in. Is this something git-bug can or does enable?

If you have the commit hash of the commit(s) introducing the bug, the it's a simple matter of seeing if that commit is an ancestor of any commit (or branch) you're interested in. Similarly, once fixed, it's just a matter of seeing if the fix is an ancestor of the interested branch/tag/commit. (Not saying that this tool couldn't or shouldn't, just simply that this should be very doable within git, I think.)

I imagine the most difficult part of this is just getting the data collected. Even with git-bisect, I very rarely find myself looking for the point of introduction, and instead just fixing the bug.

Not so niche: Bugs Everywhere does that, and when you fix a bug in a branch and then merge the branch in master the bug is closed in master as well.

Not yet, but that might be a good feature. Also, the goal is to have an extensible data model where you can store whatever metadata you want to support arbitrary external bug tracker.

Please open an issue so we can discuss that.

It would be nice to have a way to automatically update a bug's status when a commit claims to fix it. Presumably that could be done with a (server-side) hook.

A similar but much more minimal tool in shell: https://claudius-hubig.eu/note.sh

It uses a dedicated "bugs" branch checked out to a .bugs directory in the current working directory. To use, copy it to a git repository which has a master branch and a remote for this master branch an run it.

Very bare-bones, but so far sufficient and without external dependencies (but probably lots of security problems…).

This looks great! I'll very much consider using it for my next project, if only for the warm feeling in the stomach that my bugs are close to the project itself. Huge kudos to the author.

The one thing that I'm not sure about is the separate `git bug push/pull` commands. I guess it makes sense, but I'd prefer ordinary `git pull/push/fetch/merge` to seamlessly handle syncing bugs around.

git-bug store the data in separate references that are not in `refs/head`, so they are not recognized as branches. This is meant to not pollute the normal code workflow.

git push/pull doesn't touch references outside of branches and git-bug has special code in the push/pull commands to merge bugs when needed.

> git-bug store the data in separate references that are not in `refs/head`, so they are not recognized as branches

How does that work with garbage collection?

`git gc` will garbage collect what is not referenced by a ref anymore, including arbitrary references. So it's fine !

As each bug reference the file blob needed, once your delete a bug, `git gc` will clean these blob as well.

`git gc` will also garbage collect any ref that isn't used by a current branch/tag (under refs/heads and refs/tags), the index (staging), originals from filter-branch (refs/originals), and reflogs.

The question isn't whether the data will be garbage collected, but how the active data AVOIDS being garbage collected - git gc only targets a pretty small amount of refs to preserve.

Bugs are referenced at refs/bugs/<id> and refs/remotes/<remote>/bugs/<id> and those are not garbage collected.

I'm not the author (he'll probably comment here in a sec), but if this is a problem, perhaps this project could become a patch to git itself.

I'd be fine running a patched git that knows about bugs. Of course, then I'd want to make it a little more generic, and not care whether I was dealing with an issue tracker, a wiki, etc.

You are wrong about the semantics of git-gc. It will consider for garbage collection all objects that are not referenced, and a reference is everything under refs/*, in addition to the other things you mentioned (index, reflogs etc.).

The documentation on the man page does not make that clear, if explicitly calls out specific ref types - so that makes more sense.

This is really nice. I think this would work really well as a GitLab integration i.e. Use GitLab's UI when online, but store the issues in your repository so it's available offline etc.

That's my main issue with GitLab, there is so much interesting innovation that they could do w.r.t. tooling but instead they play it safe by copying Github or providing integrated services that are already possible in Github.

I genuinely feel this is not the case. GitLab is so much more - it encompasses the full DevOps Lifecycle in a single application. If you haven't checked out our homepage recently, it might be interesting for you to rediscover what GitLab offers - https://about.gitlab.com/

Full disclosure: I work for GitLab, but what I stated above has nothing to do with the fact, it's how I personally see GitLab as a product.

> GitLab is so much more - it encompasses the full DevOps Lifecycle in a single application

That's what I meant by "integrated services already possible with Github". I guess you are familiar with Github Marketplace.

I don't mean to offend GitLab, it's a great service, I just mean that it could be much more if it worked on low level things too, like storage of issues as git objects, or federated instances. These are hard problems that won't make GitLab easy sell to a manager but definitely interesting things that would get developers attention.

We have some low-level stuff on our roadmap (e.g. federated Merge Requests [1], federated GitLab instances [2], cross-platform mentions [3] - i.e. mentioning GitHub users). These didn't get prioritized yet, but it's only a matter of time. We have limited resources so we have to prioritize and scope out releases in a such a way that it advances both our vision and satisfies customer requests.

A really interesting low-level component that we created and use in production is gitaly [4] (which is a git RPC component that aims to completely remove GitLab's dependency on NFS for repository access - and replace it with gitaly itself).

[1] - https://gitlab.com/gitlab-org/gitlab-ce/issues/4013

[2] - https://gitlab.com/gitlab-org/gitlab-ee/issues/4517

[3] - https://gitlab.com/gitlab-org/gitlab-ce/issues/42230

[4] - https://gitlab.com/gitlab-org/gitaly

Glad to hear that, I was not aware of most of these. How about starting GitLab Engineering blog? :)

There is already an "engineering" category:


I work for GitLab on low level things in Git, for example:


which is requested by Drupal (among others):


GitHub hosts git repositories, GitLab does that AND EVERYTHING ELSE TOO.

Honest question: how does it compare to bugs everywhere?

http://bugseverywhere.org/ has been around for a long time but services like GitHub and GitLab refuse to support its format, effectively blocking the adoption of the tool.

Bugs Everywhere: - bug are stored as files alongside the normal code, which is different than the vast majorities of bug tracker operate, and also rely on git to merge properly the bug's files in a correct state. In git-bug, merge is simply rebasing commit, there is no conflict. - last commit was in 2013

As far as I understand, the data model of git-bug is much closer than what github does internally. If you look at githug's API, you will find the same timeline of edit operation, so it might help them to pick up the idea.

Have you tried BugsEverywhere recently? Some off its dependencies seemed to be broken last time I tried it (I use Artemis now)

There's also https://github.com/dspinellis/git-issue, but I don't have any experience with it yet. Maybe it can give you some inspiration

Awesome for personal projects or those startups with teams composed primarily of developers. But I wonder if the nature of embedding the bug reports in the repo makes it largely inaccessible to your typical QA and support personnel who tend to open the most tickets at larger organizations?

There are two sets of people who want to track programmer tasks, and they're quite different already.

The most successful company I worked for used two systems. There was an "external" user-friendly one, where customers could file issues and track changes and tie them back to contractual requirements and such, and there was an "internal" developer-friendly one, where programmers could talk about what line of the source code forgot to add one and which changeset hash introduced the bug.

Our partners wanted to file issues at the level of "I need to have some way to be able to do XYZ", and they shouldn't have visibility into source code and programmer assignments and internal roadmaps. Our programmers broke these into "Write a function to do X()" and "Add a database table for Y" and "Z.c crashes when you leave the title field blank", and they shouldn't be exposed to customer data that was part of the request.

It worked tremendously well, and I'm surprised this split approach isn't more popular. git-bug looks like it could be a great solution for the internal one.

IBM's old PMR system was their front-end, and they used CMVC as the back-end until it got replaced [1]. The two combined did what you described, and was the earliest example I know of as a commercial offering of version control tightly bound to bug reports. Very satisfying to see this concept get more traction with git and various bug trackers. I wish a public open interchange standard would evolve so the different integration combinations wouldn't be so brittle, though.

[1] https://en.wikipedia.org/wiki/IBM_Configuration_Management_V...

The web UI is designed to, in the future, be used as a public interface as well. It would need to have some kind of authentication to identify users though.

The permission model will be a problem.

The population you desire to be able to update tickets is... shall we say, "larger" ? ;) than the population you desire to be able to update code.

For any project where the bug-updaters and the code-updaters are plausibly the same set, this could be a useful thing.

But even for modest projects I'm doing for customers, it'd be awkward.

I also think the type of data your bug-updaters or bug-reporters would generate would be very different. Do you really want 23 gargantuan screenshots as blobs retained in perpetuity?

My kneejerk is that the bug metadata is a different _kind_ of data than the source, and trying to shoehorn them into the same container will be stuffy in most cases.

>Do you really want 23 gargantuan screenshots as blobs retained in perpetuity?

Yes! It's much better than having them linked from some image hoster that inevitably goes belly-up at some point.

The problem of not necessarily wanting to pull them after they've been lying around for 10 years can also be solved, see git-lfs.

Each bug is stored independently and in particular independently of the source code, so older bugs could be archived or simply left on the git remote.

I don't see your point about the permission model.

You will have access permission for the code as you normally do for your git remote as well as an extra user for the web UI that will store bug edits as one of the "public" user with whatever external authentication system you want.

>Do you really want 23 gargantuan screenshots as blobs retained in perpetuity?

A really good point. I'd say you should use something like this for internal bug tracking, and link to external bug reports instead.

Whether you accept an external bug report as an actual bug is a question in itself, which should be dealt with beforehand. I'd say this is good for notetaking for the team and for seeing which bugs are resolved in which version.

> Do you really want 23 gargantuan screenshots as blobs retained in perpetuity?

There's a whole bunch of ways to handle large files in git nowadays without bloating the size of the repo. Hopefully the distributed bug tracker would work with those...

IMO It should be feasible to build data bridges for integrating or syncing with related systems.

That said, as a developer I'd love it if I could track my more technical thoughts & needs without having to go to my enterprise tools where BA's run wild & confused. The trade-off of course being more dissonance and potentially a failure to advocate for this work in planning, unless you synchronize data somehow.

Would be nice if there was a tracker done in all shell, or just plain text, so that we don't need to rely on go, or another program.

Maybe it could work to develop a core protocol and data structures that are usable by others, with your own go implementation to work with those structures and protocol.

I really like this. However, I'd like to know if there is something like git-bug status to see what new bugs/comments were added and git-bug reset (or checkout) to revert recent changes that were not pushed.

That make sense. I'm waiting to have a clear design for the CLI UX to improve that. Feel free to raise your voice: https://github.com/MichaelMure/git-bug/issues/2

In the past I've worked in a lab where dev laptops could be disconnected from the network for days without notice.

A tool like this would have reduced the productivity hit during those periods.

Stumbled upon this similar project: https://github.com/sit-fyi/issue-tracking yesterday. I thought it was an interesting way of going about it. It's build upon this idea of serverless information tracking. https://github.com/sit-fyi/sit

Personally I've had great success with http://mrzv.org/software/artemis for the past few years. Issues are stored in a `.issues` directory as maildirs, so it does "leak out" into the file tree, unlike this which seems to build on Git's existing metadata storage.

I felt a need for this long time ago, but since i use emacs, I just created a "bugs.org" file at the root of the project directory

A pure python distributed bug tracker with a web interface. Also it has attachments. Very easily customizable:


Is there any reason to stop with a bug tracker? Why not a wiki, metadata about your files, task list, etc.?

I went through this thought process with SIT[1] -- it started out with issue tracking hard-coded into it but it has been later extracted into a separate model and the data & process models are being refined to make them much more generic and support other use cases.

[1] https://sit.fyi

The same idea for the storage and merge can certainly be applied to other use cases. A wiki would probably be kinda tricky because you would need a an edit operation for each change in the text but that's probably doable.

This reminds me of sit: https://sit.fyi

This is the second time this week I’ve seen, and been jealous of, a gocui interface.

huge fan of this model. i started my own proof of concept of it a while ago, tho didn't take it this far. i assume you can track the status of the bug independently per branch?

Bugs storage and management is entirely disconnected from the regular code and branches. That said, it probably make sense to be able to tag a bug with the branches that are affected.

ah, bummer. yeah, "closing" a bug in a branch / PR, and then having that status automatically tracked across all merges into master / other branches is i think a huge win for this kind of idea. (vs. manually tracking it per branch, which is a good feature but no one ever seems to use, i believe because it's just to labor intensive.)

That helps you track the location of one fix, but not of all the other places the bug still resides (other release trains, for instance.) I feel like maybe there are several release models that would benefit from different types of bug trackers.

agreed. to be clear i like this project a lot. we desperately need innovation in this area. this current github+jira duopoly on issue tracking is annoying.

Shameless plug: if you like this, I'm looking for a remote job. My email is on my Github profile and my resume is here: http://pellelatarte.fr/dl/resume.pdf .

Sorry for turning this comment thread into an AMA but I have a question for you as the author :)

Reading over this page https://github.com/MichaelMure/git-bug/blob/master/doc/model... gives me the impression that you’re reproducing some of Git’s functionality. Can you elaborate on why Git’s merge model is insufficient and why the “Operation/OperationPack” serves the distributed-edits problem better than Git itself? Or alternatively, explain what I’m missing about how this doesn’t step on Git’s toes? Congrats on building this and shipping it!

There is manual merging code in git-bug [0] only because git doesn't provide a low-level command to rebase a branch onto another without touching the index. I don't think it's an intended limitation of git but more likely no one had a use case for that so they simply used the normal index.

Apart from that, a normal git rebase would merge two versions of the same bug conflict free. Each commit is independent from the other. A merge is simply moving the new operations at the end of the chain of edit.

If for instance, that means that you closed a bug that is already closed, well the bug is still closed and your action will still be displayed in the timeline.

[0]: https://github.com/MichaelMure/git-bug/blob/c974cc022d3ecd2e...

Damn, your comment moved my shameless plug at the very end of the thread :(

I wouldn't be too worried, it made it here: https://news.ycombinator.com/bestcomments

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact