Here's the note I have on that: “For chatbot interfaces, emerging approach is to have another agent simulating the user (as opposed to a more classic approach based on token prediction probs on chat transcripts, what I think you're referencing). Then still use a model for grading. Only place I've seen this so far: https://github.com/Forethought-Technologies/AutoChain/blob/m... ” - AI Startup Founder
> For each predicted output token, I want to know exactly which source document(s) were utilized including indices from those documents and relevant statistics.
You don't do this for any other kind of decision or tool, why do you need it for LLMs?
i think in truth you need a source that convinces you (or the regulator) that your choice is acceptable, so that you can pass off the responsibility.
If the LLM were to give you an answer backed by relevant (cited) source documents of regulations and a good explanation it would make no difference compared to a human worker doing the same. -> this is already possible
SMART goals and KPIs are meant to improve work focus, not work ethos. Those two are not in opposition to each other. IME engineers who don't want to do measurably good work are inexperienced, disgruntled or unmotivated.
Forcing goals onto a disgruntled/unmotivated engineer will end badly.
Giving no goals to motivated engineers will also end badly.
Giving bad, unrelated or unachievable goals to motivated engineers will end in disgruntled or unmotivated engineers.
We have. This is acceptable for some clients, but not for others. Both groups, however, prefer maintaining complete control over their data, given the chance.
Edit: plus, my personal view is that local LLMs are the future. They've already caught up to GPT-3.5 (based on my testing); and they continue to evolve rapidly. Makes sense to focus our limited resources on riding that wave.
OpenAI won't go away, but neither will they remain the first choice (or only choice!) for most use-cases.
There's some . . entrepreneurs . . who have been promising NIST/ITAR-compliant LLM frameworks on Azure, but when you ask around, they have not done all the legwork (AG/AGS). They're working off Azure Public, with "waivers" that they won't show anyone. Also, the history of their leadership is . . questionable. It all feels just a little hinky. Until that's cleared up, I advise anyone fooling with LLMs to do it on-prem, at least for the moment. One thing I'm worried about: doing LLMs with something like GovCloud is going to be absolutely bananas in terms of price-per-compute.
This sounds like a perfect job for jina.ai -> sharding, redudancy, automatic up- and down scaling, security features and most importantly: flexibility to use and switch whatever (vector) database
https://jina.ai/
To see your page content, crawlers need to load your JS and wait for it to hydrate the page.
Google does this (but it's still not ideal). Most other crawlers don't. That means they won't see your rendered content.
This becomes an issue with sharing on social media, because your meta tags won't be read properly. If you share a blog post on Twitter for example, it won't be able to fetch your og:image, title etc so it won't look good.
To fix that, you have to use prerendering or server-side rendering. I'm doing hacky stuff with prerendering inside a Cloudflare worker where it detects that it's a crawler and prerenders the page if so.
SPAs also tend to have slow initial load times, which is becoming increasingly important for SEO. You can't avoid at least one additional network call after loading the initial HTML, and if you're not careful you can end up with chained requests for different components etc.
For sites like mine that rely on SEO and social media, it's a pretty big annoyance.
Thanks for your input!
server-side rendering should fix most of the first issue. You can get all the relevant data pre-rendered and it's pretty easy to do with the right framework. I've used quasar.dev for this in the past. It should also solve the initial load times (to an extent).
The chaining can be fixed with lazy loading/code splitting.
The only thing i haven't fixed yet is hiding the cookie banner for bots :/
reply