Interesting approach! While converting docs to markdown works, I've found that technical documentation (especially for complex repos) needs more than just text conversion for effective LLM consumption.
The key challenge is preserving repository context - like code dependencies, architectural decisions, and evolution patterns. Have others experimented with knowledge graph approaches for maintaining these relationships when processing repos for LLMs?
Interesting approach! While URL-based extraction is convenient, I've been working on a more comprehensive solution for repository knowledge retrieval (llama-github). The key challenge isn't just extracting code, but understanding the semantic relationships and evolution patterns within repositories.
A few observations from building large-scale repo analysis systems:
1. Simple text extraction often misses critical context about code dependencies and architectural decisions
2. Repository structure varies significantly across languages and frameworks - what works for Python might fail for complex C++ projects
3. Caching strategies become crucial when dealing with enterprise-scale monorepos
The real challenge is building a universal knowledge graph that captures both explicit (code, dependencies) and implicit (architectural patterns, evolution history) relationships. We've found that combining static analysis with selective LLM augmentation provides better context than pure extraction approaches.
Curious about others' experiences with handling cross-repository knowledge transfer, especially in polyrepo environments?
Interesting discussion! While RAG is powerful for document retrieval, applying it to code repositories presents unique challenges that go beyond traditional RAG implementations. I've been working on a universal repository knowledge graph system, and found that the real complexity lies in handling cross-language semantic understanding and maintaining relationship context across different repo structures (mono/poly).
Has anyone successfully implemented a language-agnostic approach that can:
1. Capture implicit code relationships without heavy LLM dependency?
2. Scale efficiently for large monorepos while preserving fine-grained semantic links?
3. Handle cross-module dependencies and version evolution?
Current solutions like AST-based analysis + traditional embeddings seem to miss crucial semantic contexts. Curious about others' experiences with hybrid approaches combining static analysis and lightweight ML models.
I'd like to share our technical journey migrating our code review system from Mistral-Large-2407 to 2411, and the key challenges we overcame. Here are the most interesting findings:
1. Prompt Pattern Evolution
- Initial challenge: Direct model upgrade led to significant quality degradation
- Root cause: Changes in 2411's prompt processing architecture
# Previous prompt format for Mistral-Large-2407
<s>[INST] user message[/INST] assistant message</s>[INST] system prompt + "\n\n" + user message[/INST]
# New optimized prompt format for Mistral-Large-2411
<s>[SYSTEM_PROMPT] system prompt[/SYSTEM PROMPT][INST] user message[/INST] assistant message</s>[INST] user message[/INST]
- Solution: Implemented enhanced prompt patterns through LangChain
2. API Integration Insights
- Built custom HTTP client interceptor for debugging
- Discovered crucial differences in message formatting
- Leveraged LangChain's abstraction layer effectively
3. Key Technical Improvements
- Enhanced review focus through optimized prompts
- Improved output reliability and format compliance
- Eliminated response truncation issues
This is implemented in our AI Code Review Github APP LlamaPReview [https://jetxu-llm.github.io/LlamaPReview-site/]. Happy to discuss specific implementation details or share more technical insights about working with Mistral-Large-2411 in production.
I've been working on an AI-driven development vision, building on the "System of Agents" concept discussed in the article.
The key insight is that to achieve truly automated development, we need AI agents across the entire development lifecycle - not just in isolated tasks.
Our system implements a "Retrieval Augmented - Complex Coding Task accomplishment loop" with:
- Multiple LLM-based agents handling different aspects of development
- A Task and State Management System for agent coordination
- Dual knowledge retrieval combining self-repository learning and GitHub public knowledge
- Integrated development tools library (editing, testing, deployment) orchestrated through multi-agent collaboration
The key differentiator is the holistic integration - every step from initial development to deployment is agent-aware and interconnected. This creates a true "Service-as-Software" development environment where AI doesn't just assist, but actively drives the development process.
We're seeing promising results in automated code generation, self-healing test suites, and intelligent CI/CD pipelines that learn from deployment patterns.
Would love to hear thoughts from the community, especially from those working on similar full-lifecycle automation approaches.
Thank you for such detailed and thoughtful feedback! Really appreciate the time you took to analyze our claims and point out the areas needing more clarity.
You're absolutely right about the marketing copy - we should be more precise and transparent about what we actually do vs. what's aspirational.
Regarding "understanding code relationships and dependencies": We're building a knowledge graph of the entire repository that captures code relationships, function calls, and module dependencies. This graph is then used with GraphRAG to fetch relevant context for each PR, allowing the LLM to understand the broader impact of changes.
Important to note: We take privacy very seriously. All code analysis happens in-memory during PR reviews - we don't permanently store any source code or build persistent knowledge bases from customer code. The knowledge graph is generated and used on-the-fly for each review session.
This approach helps us work around context window limitations while providing meaningful insights. However, I should note that this feature is still under active development - we're continuously improving the graph construction and relevancy matching.
Would love to hear your thoughts on this approach. We're committed to building something genuinely useful for developers rather than just another LLM wrapper.
The service runs on secure cloud infrastructure and processes code in-memory during PR reviews - we don't permanently store any source code. We use enterprise-grade LLMs (can't disclose specific models due to licensing) and implement context-aware analysis without fine-tuning on customer code.
When we say "learning", we mean analyzing the codebase context during PR reviews to understand patterns and relationships, not training or building persistent knowledge bases. This ensures both privacy and effectiveness.
We're working on open-sourcing parts of the implementation - will share more soon!
The key challenge is preserving repository context - like code dependencies, architectural decisions, and evolution patterns. Have others experimented with knowledge graph approaches for maintaining these relationships when processing repos for LLMs?
reply