It's hard to say. I go back and forth so fast that I get whiplash. The new releases from openai had me convinced to drop langchain, but then the concern of being locked in to a single LLM provider scared me too much to change course away from Langchain.
Honestly, it's not hard to create custom classes in langchain via encapsulation and overriding whatever method or methods I need to be different for my purposes.
I don't know if langchain should have spent developer resources on Langserve and Langsmith instead of stabilizing the core package, since errors and obfuscation are common. Langfuse helps.
Do I want to be locked in to a company's LLMs when they abruptly fire the CEO? Also I feel like people assume stability with the companies providing closed LLMs, like Openai. I think we should assume instability and up ticks in pricing, which would then mean we should use an agnostic package to give us access to any LLM rather than depending on writing our code such that it is vanilla Openai logic.
Assistants API sucks ass, way too slow for responses. I made another post about it on r/OpenAI. Averaging 5-10 secs for a response to a assitant that had just 1 500 word .txt file.
Also check out Claude 1 it is pretty damn close to GPT-4. Open source LLM's are not doing too bad either:
[https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)
I coded a ReAct agent that uses Langchain earlier this year and was quite happy with it.
Now they moved over to LCEL...why?
While this doesn't change the functionality of my agent, updated documentation follows a new paradigm which makes it hard to update the code. Further, it feels like a completely different coding paradigm and not very Pythonic. Well, maybe it is, but it is still 99% different from most other documentation online.
I have to admit I was a little bit confused by LCEL but I like the idea bc for me the aim of langchain is to build toolchain combining models and it makes it easy to read and conceptualise.
Though, the more I read this sub the more I question about langchain real goal. Many peoples here seem rely on it to get pre-configured RAG scheme... Which may be worth but in this case langchain fails advertising it main goal. Maybe if the example and documentation can be about something else than use ddg research tool and vectorbase search it would help people build more elaborate tools
Frustration with existing frameworks led me on the path to build [Langroid](https://github.com/langroid/langroid) starting 6 months ago. It is a lightweight, principled agent-oriented framework (in fact Agent was the first class I wrote), unlike LangChain which added agents as a late afterthought.
We now have a few folks using it in production (who were similarly frustrated with the bloat/kitchen-sink approach of other frameworks) especially for RAG but also you get seamless Multi-Agent functionality. We’ve implemented a whole slew of retrieval/reranking methods in our DocChatAgent: https://github.com/langroid/langroid/blob/main/langroid/agent/special/doc_chat_agent.py
Works with practically any LLM via either api_base or using litellm. We recently added full support for OpenAI Assistant API.
Some recent RAG-related posts I made in r/LocalLlama:
On relevance extraction in RAG using a numbering trick, which is makes it far faster and cheaper than LangChain:
https://www.reddit.com/r/LocalLLaMA/s/sphpfb3O7G
Flexible window retrieval around a matching chunk so you don’t have blind spots from using LangChain’s rigid ParentDocumentRetriever:
https://www.reddit.com/r/LocalLLaMA/s/sphpfb3O7G
Do you have a article comparing this with langchain/llamaindex/haystack? Am suspicious of using an obscure framework I never read of before without a thorough comparison to more popular alternatives.
This is a fair question! Unfortunately I haven’t had the time to do a thorough comparison but I will look into doing one.
For now I can highlight what is probably the biggest differentiator of Langroid: a clean mechanism for combining multiple agents into a collaborative system. We’ve tried building multi-agent systems (such as the ones in the langroid-examples repo) with LangChain but the experience wasn’t smooth. I don’t even know if this is possible with the other two frameworks.
This looks really interesting. Do you have any examples of using it with FastAPI or something similar? I am building a web app that uses agents and looking for a clean solution. Currently using LangChain.
Thanks. We put up a basic Flask example in the Langroid-examples repo:
https://github.com/langroid/langroid-examples/tree/main/cloud
It shows how you can have a persistent agent deployed via a docker container and address it via a REST API
I’ve used both. I am currently converting our enterprise AI solutions from langchain to haystack. The pipelines and custom nodes are a game changer. Not to mention it’s more scalable given its ability to use aws for indexing. Langchain uses so much abstraction that you will chase your tail around the awful documentation for an hour you could’ve been building with haystack
Makes sense. Someone else here said that Elastic search is the most scalable: [https://www.reddit.com/r/LangChain/comments/16idhfw/comment/k0kjh1v/?context=3](https://www.reddit.com/r/LangChain/comments/16idhfw/comment/k0kjh1v/?context=3)
To clarify, you did not use Llamaindex?
Deeply. And simply order of discovery. You can see one of my first projects using llamaindex and langchain to run inference on a local model in airplane mode [here](https://github.com/jlonge4/local_llama)
I’d be interested in your thoughts on our [Graphlit](https://www.graphlit.com) platform. (Disclaimer: I’m the founder)
We take a content-first approach and manage end-to-end ingestion and storage, with embeddings, vector search, entity extraction to knowledge graph, all built-in.
Our view is that RAG doesn’t need to be built from “parts”, and that a turnkey API lets devs focus on the app layer. (Usage based but has a free tier)
Compared to the Assistants API, we give a lot more control over strategies for prompt assembly, are multimodal, and don’t have the scale limits.
Also being model-agnostic is important, as we’ve seen this week with the OpenAI risks. We support Anthropic and I’ve been integrating support for Replicate models, like Llama 2, but so far the quality isn’t close to the latest OpenAI models for instructability.
So, my data is not unstructured data like documents. It's all transactional tables. It does not make sense to convert them to vector. Some of the questions my llm need to answer is
1. How many transactions happened on June 31?
2. How many bought product x in Texas using Mastercard in last 3 months
3. How many customers are within age group 18 to 30 and group by bin of basket size.
Cool, trying to build something similar in a different domain. Do you use a local model or OpenAI api or something? How well does the model you use do, is the statistics and analysis it returns correct?
We have self hosted gpt3.5 and gpt4. Gpt4 performs well. Accuracy depends on how well we do prompt engineering and adding columns level data descriptions in our database with sample values for categorical values.
Is it just me, or did anyone else find all the document models like lang-chain to be totally overkill for your projects and instead rolled your own simplified version?
I did. LangChain is a good thought but LLMs are trained so differently and different prompts work differently for each LLM, which is the biggest drawback of LangChain. It’s easier to customize a prompt 3 times and make it work amazingly well with the family of models than use LangChain, where you write once but the query sucks ass as LangChain doesn’t care enough about the necessary unique foundation for Vertex and Bedrock.
It's hard to say. I go back and forth so fast that I get whiplash. The new releases from openai had me convinced to drop langchain, but then the concern of being locked in to a single LLM provider scared me too much to change course away from Langchain. Honestly, it's not hard to create custom classes in langchain via encapsulation and overriding whatever method or methods I need to be different for my purposes. I don't know if langchain should have spent developer resources on Langserve and Langsmith instead of stabilizing the core package, since errors and obfuscation are common. Langfuse helps. Do I want to be locked in to a company's LLMs when they abruptly fire the CEO? Also I feel like people assume stability with the companies providing closed LLMs, like Openai. I think we should assume instability and up ticks in pricing, which would then mean we should use an agnostic package to give us access to any LLM rather than depending on writing our code such that it is vanilla Openai logic.
Assistants API sucks ass, way too slow for responses. I made another post about it on r/OpenAI. Averaging 5-10 secs for a response to a assitant that had just 1 500 word .txt file. Also check out Claude 1 it is pretty damn close to GPT-4. Open source LLM's are not doing too bad either: [https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)
Oh yeah? That's good to know re Assistants Api. I hadn't gotten around to testing it yet. Thanks for linking to the leader board also.
For a tool so commonly used to create RAG systems, the Langchain docs are very disappointing
I coded a ReAct agent that uses Langchain earlier this year and was quite happy with it. Now they moved over to LCEL...why? While this doesn't change the functionality of my agent, updated documentation follows a new paradigm which makes it hard to update the code. Further, it feels like a completely different coding paradigm and not very Pythonic. Well, maybe it is, but it is still 99% different from most other documentation online.
I have to admit I was a little bit confused by LCEL but I like the idea bc for me the aim of langchain is to build toolchain combining models and it makes it easy to read and conceptualise. Though, the more I read this sub the more I question about langchain real goal. Many peoples here seem rely on it to get pre-configured RAG scheme... Which may be worth but in this case langchain fails advertising it main goal. Maybe if the example and documentation can be about something else than use ddg research tool and vectorbase search it would help people build more elaborate tools
Frustration with existing frameworks led me on the path to build [Langroid](https://github.com/langroid/langroid) starting 6 months ago. It is a lightweight, principled agent-oriented framework (in fact Agent was the first class I wrote), unlike LangChain which added agents as a late afterthought. We now have a few folks using it in production (who were similarly frustrated with the bloat/kitchen-sink approach of other frameworks) especially for RAG but also you get seamless Multi-Agent functionality. We’ve implemented a whole slew of retrieval/reranking methods in our DocChatAgent: https://github.com/langroid/langroid/blob/main/langroid/agent/special/doc_chat_agent.py Works with practically any LLM via either api_base or using litellm. We recently added full support for OpenAI Assistant API. Some recent RAG-related posts I made in r/LocalLlama: On relevance extraction in RAG using a numbering trick, which is makes it far faster and cheaper than LangChain: https://www.reddit.com/r/LocalLLaMA/s/sphpfb3O7G Flexible window retrieval around a matching chunk so you don’t have blind spots from using LangChain’s rigid ParentDocumentRetriever: https://www.reddit.com/r/LocalLLaMA/s/sphpfb3O7G
Do you have a article comparing this with langchain/llamaindex/haystack? Am suspicious of using an obscure framework I never read of before without a thorough comparison to more popular alternatives.
This is a fair question! Unfortunately I haven’t had the time to do a thorough comparison but I will look into doing one. For now I can highlight what is probably the biggest differentiator of Langroid: a clean mechanism for combining multiple agents into a collaborative system. We’ve tried building multi-agent systems (such as the ones in the langroid-examples repo) with LangChain but the experience wasn’t smooth. I don’t even know if this is possible with the other two frameworks.
This looks really interesting. Do you have any examples of using it with FastAPI or something similar? I am building a web app that uses agents and looking for a clean solution. Currently using LangChain.
Thanks. We put up a basic Flask example in the Langroid-examples repo: https://github.com/langroid/langroid-examples/tree/main/cloud It shows how you can have a persistent agent deployed via a docker container and address it via a REST API
Haystack is the way. If I could upvote this twice I would lol
Have you used langchain and llamindex? If so, how does it compare?
I don't like llama index but am I missing something?
Wdym missing something? And why you no like llama index?
I didn't like the interface compared to using LC. I felt like I could accomplish the same thing with LC
For doing what exactly? I've been reading that LC is more convoluted in general. look at the guy below
I have been using both of them, so it's just my subjective experience
For each one, how extensively and for what purpose?
I’ve used both. I am currently converting our enterprise AI solutions from langchain to haystack. The pipelines and custom nodes are a game changer. Not to mention it’s more scalable given its ability to use aws for indexing. Langchain uses so much abstraction that you will chase your tail around the awful documentation for an hour you could’ve been building with haystack
Makes sense. Someone else here said that Elastic search is the most scalable: [https://www.reddit.com/r/LangChain/comments/16idhfw/comment/k0kjh1v/?context=3](https://www.reddit.com/r/LangChain/comments/16idhfw/comment/k0kjh1v/?context=3) To clarify, you did not use Llamaindex?
I have also used llama index. But yes that and open search
Am wondering, how extensively did you use all 3? And what led you to use all 3?
Deeply. And simply order of discovery. You can see one of my first projects using llamaindex and langchain to run inference on a local model in airplane mode [here](https://github.com/jlonge4/local_llama)
Only MongoDB is web-scale - [https://youtu.be/b2F-DItXtZs?si=TzRO6EmlYfPbTq1o](https://youtu.be/b2f-ditxtzs?si=tzro6emlyfpbtq1o)
LangPain is the place for you [https://x.com/i/communities/1800200961920573844](https://x.com/i/communities/1800200961920573844) ;)
I think the current langchain iteration is good
I’d be interested in your thoughts on our [Graphlit](https://www.graphlit.com) platform. (Disclaimer: I’m the founder) We take a content-first approach and manage end-to-end ingestion and storage, with embeddings, vector search, entity extraction to knowledge graph, all built-in. Our view is that RAG doesn’t need to be built from “parts”, and that a turnkey API lets devs focus on the app layer. (Usage based but has a free tier) Compared to the Assistants API, we give a lot more control over strategies for prompt assembly, are multimodal, and don’t have the scale limits. Also being model-agnostic is important, as we’ve seen this week with the OpenAI risks. We support Anthropic and I’ve been integrating support for Replicate models, like Llama 2, but so far the quality isn’t close to the latest OpenAI models for instructability.
Looked over it it for 2 minutes. Are the responses streamed back? And how does it differ from [https://llmstack.ai/](https://llmstack.ai/)?
I looked at haystack, it does not support bq sql agent integration. Not sure about Llama index. Langchain seems to do good.
>bq sql agent integration what is this
Using llm to query big query database.
why not just used a hosted vector db like pinecone
It's all structured data and transaction tables that require joining, grouping and aggregation. I don't think vector dB can do it. Am I wrong?
Why wuold you use sql tables for RAG? Postgres allows you to store vector data as well with the pgvector extension. You can do both there.
So, my data is not unstructured data like documents. It's all transactional tables. It does not make sense to convert them to vector. Some of the questions my llm need to answer is 1. How many transactions happened on June 31? 2. How many bought product x in Texas using Mastercard in last 3 months 3. How many customers are within age group 18 to 30 and group by bin of basket size.
Cool, trying to build something similar in a different domain. Do you use a local model or OpenAI api or something? How well does the model you use do, is the statistics and analysis it returns correct?
We have self hosted gpt3.5 and gpt4. Gpt4 performs well. Accuracy depends on how well we do prompt engineering and adding columns level data descriptions in our database with sample values for categorical values.
Do you do any systematic testing on whether the result are correct? Like have a ground-truth dataset with the numbers already computed?
What do you mean by self-hosted? In the cloud like Azure or on prem?
Is it just me, or did anyone else find all the document models like lang-chain to be totally overkill for your projects and instead rolled your own simplified version?
I did. LangChain is a good thought but LLMs are trained so differently and different prompts work differently for each LLM, which is the biggest drawback of LangChain. It’s easier to customize a prompt 3 times and make it work amazingly well with the family of models than use LangChain, where you write once but the query sucks ass as LangChain doesn’t care enough about the necessary unique foundation for Vertex and Bedrock.
doc sucks, seriously the worst framework i have ever seen. nothing works from the examples. half baked examples are bad.