Local Langchain chatbot with chroma vector storage memory

[ossirytk]

I find that there is a woeful lack of more complex examples. For that purpose, I've made a character AI chatbot with chroma vector storage memory to serve as an example and a simple drop in platform for testing things.

The main chatbot is built using llama-cpp-python, langchain and chainlit. It supports json, yaml, V2 and Tavern character card formats.

It uses langchain llamacpp embeddings to parse documents into chroma vector storage collections. There is also a test script to query and test the collections.

Everything is local and in python. Runs in a virtual env with minimum installations. No api keys needed. Runs locally with cpu and optional gpu offloading

https://github.com/ossirytk/llama-cpp-chat-memory

[ossirytk]

I've added support for json lorebooks and metadata filtering. It's a bit hacky currently, but I'll see about improving the filtering when Chroma supports more complicated actions.

[eyurtsev]

Thanks for sharing!

There's some langchain documentation that could be useful:

1. Local model with ollama (https://python.langchain.com/docs/integrations/llms/ollama)
2. https://python.langchain.com/docs/use_cases/chatbots/

You can find various server implementations here (look at chat bots / persistence etc):

https://github.com/langchain-ai/langserve?tab=readme-ov-file#examples

For example: chat with persistence

https://github.com/langchain-ai/langserve/tree/main/examples/chat_with_persistence

And here's a version for multi-user (not to be used with ollama since ollama does not support single user):

https://github.com/langchain-ai/langserve/tree/main/examples/chat_with_persistence_and_user

[ossirytk]

Made another update

• Added basic csv parsing
• Improved ner parsing
• Improved web scraping
• Better document fetching

[ossirytk]

Updated Ner parsing to use multiprocess for improved performance.

[ossirytk]

Updated metadata key processing for text chunks to use multiprocess also. Checking all text chunks for relevant metadata keys was slow when there were many text chunks and a large amount of keys. The performance has been improved significantly.

[ossirytk]

Reworked collections. There is now a context collection and a message example collection available.
Run settings and settings panel. Collection and prompt template defaults are now in run settings.
Settings can be changed runtime with settings panel.