we use hatchet to orchestrate our long running backend jobs. it provided us with scalability, reliability, and observability into our tasks with a couple lines of code.
- processing is usually stuck at 99% because when we order the components of the repository by file-directory dependency and ast dependency, there are a lot, LOT more leaves than the internal nodes + the root. Since we have to have the results of the dependency before we move on, moving up the dependency chain with llm calls take a while. This is even more pronounced when nearing the root for completion. We are working on optimizing this flow as it is very annoying for us as well.
- "Error: Internal error while locating sources": this is embarrassing but we did experience a database outage today (I wonder why) some repositories have a faulty status. It should be back up now and we are working to recover/reprocess the repos that have failed during the outage (including dvx/lofi)
- "Onboard" that was our previous name, this has slipped through the cracks, thanks for pointing it out!
We are trying to parse most of the popular open source repos (we have processed repos like python, vscode, etc). We are hoping to fully process the linux kernel soon as well (a personal benchmark of mine).
Noted - I think Soohoon meant to use the word "often" and not usually. That said, you're right, counting completed nodes would be a more accurate way of measuring progress.
For now we are storing the embeddings of the generated docstrings, the intuition behind this is kinda like how HyDE works. I don't think the actual code itself is reversible from the embeddings, but yes we do want to store the actual code at some point and we will need the proper security measures for that. We are also actively developing an self-hostable version for enterprise.
Cofounder of Greptile here, good documentation is not going anywhere. What we do hate is going through bad documentation to find the tiny bit of information we need that is more often than not outdated to fit our needs. We are looking for ways to integrate information not explicitly written in code to understand codebases as well.
Hey looked into this error, Porter was not processed by us yet, it does take a while for a repo to be processed initially but this link should give you the progress report + you can chat with it when its done:
I followed the link because I was curious and it still said processing, so I navigated to the homepage and clicked "Try a popular repo" leading to <https://app.greptile.com/chat/3t4qpefuh9eqckcdt0b8i?repo=pos...> which then said "We couldn't access this repo." It seems the actual syntax is app.greptile.com/chat/github/postgres/postgres -> https://app.greptile.com/chat/lfbc034nkj7w03v2kb5zp?repo=git... so wherever that list of "popular repos" is coming from needs to be updated to avoid other people having bad first experiences
It also appears that the vote buttons do nothing if one is not logged in, so I'd recommend eliding them unless logged in since it's just frustrating to wonder if some JS error ate the vote or what
Unless you are batching them up and hiding them in the upteen bazillion posthog POST requests, no, they do not
I shudder to think how much you must be spending on compute asking for that /api/poll/batch once a second, each one saying it is a Miss from cloudfront. It's especially mysterious since there's nothing on the UI that the user would see about the .filesProcessed increasing that justifies DDoSing yourself like that, IMHO
We haven't built it for code gen in particular, but a lot of our users seem to be using it for that. We want to really nail down people getting to understand what is going on in the codebase itself.
There are about a hundred open source repos you can talk to for free over at https://app.greptile.com/repo, we are looking to add more repositories as well. The permissions we get on login are from the Github App flow, you will still have to grant Greptile access specific organization/repositories for it to work with your own private repos.
We don't do direct look up/indexing for authors (although the authors file is usually somewhere in the repo for larger projects), comments, issues, or PRs just yet but that is definitely something we are looking to add.
reply