priompteng's comments

On: RAG Logger: An Open-Source Alternative to LangSmith

[priompteng]

Tbh, while their SDK openllmetry is good, traceloop the product hardly has their basic solution in order.

On: RAG Logger: An Open-Source Alternative to LangSmith

[priompteng]

Langsmith only shows the llm inferences. Also it hardly works with other frameworks like crewai for instance.

On: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps

[priompteng]

Felt the same way. While Langfuse could be great, it oddly looks like solicited “review”ish comments from existing Langfuse users. Just gotta be a careful.

On: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps

[priompteng]

Nice work but, Sorry but I don’t feel comfortable either proxying my llm calls through a 3rd party unless the 3rd party is a llm gateway like litellm or arch or storing my prompts in a SaaS. For tracing, I use OTEL libraries which is more than sufficient for my use case.

[marcklingen → priompteng]

If you use an OSS Gateway already, some (e.g. LiteLLM) can natively forward logs to Langfuse: https://docs.litellm.ai/docs/proxy/logging#langfuse

We are looking into adding an Otel Collector as OTel-semantics are maturing around LLMs. For now many features that are key to LLMOPs are difficult to make work with OTel instrumentation as the space is moving quickly. Main thread on this is here: https://github.com/orgs/langfuse/discussions/2509

On: LLM Stack Guide

[priompteng]

Good post! Seems like helicone excels at every feature. Surprise, surprise, helicone published this.