Mutable.ai logoAuto Wiki by Mutable.ai

langchain

Auto-generated from langchain-ai/langchain by Mutable.ai Auto Wiki

langchain
GitHub Repository
Developerlangchain-ai
Written inPython
Stars73k
Watchers 618
Created2022-10-17
Last updated2024-01-05
LicenseMIT
Homepagepython.langchain.com
Repositorylangchain-ai/langchain
Auto Wiki
Generated at2024-01-05
Generated fromCommit f73bf4
Version0.0.4

LangChain provides a modular framework for building natural language applications powered by large language models (LLMs). It enables developers to leverage state-of-the-art AI techniques through reusable components and templates.

At the core, LangChain offers base abstractions like the Chain class that allow composing modular building blocks into end-to-end systems. It also provides integrations with LLMs from partners like Anthropic, Cohere, and Google to unlock conversational and reasoning abilities.

Components for tasks like retrieval (MultiVectorRetriever), prompting (ChatPromptTemplate), and output parsing (StrOutputParser) can be combined into pipelines using LangChain's Runnable interfaces. This makes it easy to build complex workflows by reusing and chaining simple building blocks.

The langchain directory contains the core framework, while libs offers both low-level interfaces and high-level templates. docs provides comprehensive API references and guides.

Key functionality offered includes:

This modular design allows developers to build increasingly complex workflows by combining simple, well-defined components through common interfaces. Documentation and guides provide extensive learning resources.

LangChain relies heavily on vector search for efficient retrieval over documents, images, audio and more. This allows scaling to large datasets by avoiding expensive sequential scans. Composition of components is achieved through Python interfaces like chains and runnables.

In summary, LangChain offers an extensive toolbox of techniques and templates for developers to build applied NLP systems by combining reusable building blocks. Its modular architecture separates concerns into interchangeable components powered by LLMs. Documentation and guides provide extensive learning resources.

Core Functionality

References: langchain, libs

The libs directory contains Python libraries that define core abstractions and provide reusable components for building natural language applications with LangChain. These libraries power key functionality for tasks like question answering, information retrieval from documents, and developing agent-based systems.

The ChatModel class in …/__init__.py provides a common interface to Anthropic models.

Base Abstractions

References: libs/core

The LangChain core library defines several important base abstractions that provide reusable functionality across different components. The BaseLanguageModel class in …/base.py acts as the core interface that language model wrappers inherit from. It standardizes key methods like generate_prompt() and predict() to promote interoperability.

The BaseDocumentTransformer class in …/transformers.py defines the interface for transforming document representations. Subclasses implement transform_documents() to define custom transformation logic on Document objects.

The BaseExampleSelector class in …/base.py provides a common interface for selecting subsets of examples from a list. The LengthBasedExampleSelector subclass in …/length_based.py inherits from this to add example selection based on prompt length. Its select_examples() method iterates through examples, generates prompts with a PromptTemplate, and trims examples that exceed the maximum length.

The BasePromptTemplate class in …/base.py acts as the core prompt template interface. The PromptTemplate subclass in …/prompt.py represents reusable string templates that can be formatted with variables. Its format() method handles merging variables into the template string.

Together, these base classes provide common abstractions that various components in LangChain can inherit from. This promotes code reuse and standardized interfaces across different parts of the system.

Reusable Components

References: libs/community

The …/agent_toolkits directory provides implementations of reusable agent toolkits for integrating conversational agents with various external resources and services. It contains subdirectories that define toolkits for services such as AINetwork, providing Python classes that represent different agent toolkits and allow modular construction of complete agents by chaining components.

The BaseToolkit class defined in …/base.py provides a common interface for toolkits to return lists of tools via the abstract get_tools() method. Classes like AINetworkToolkit inherit from this and expose tools to interact with external resources like the AINetwork blockchain. Individual tools subclasses BaseTool and define the actual operations.

The BaseCallbackHandler abstract base class defined in …/__init__.py specifies the interface that all specific callback handlers must implement. Concrete subclasses then provide integrations with services like OpenAI. The BaseMetadataCallbackHandler class handles tracking useful metadata about callback execution.

Core Interfaces

References: libs/langchain

The …/__init__.py file imports core interfaces and base classes from various submodules. This provides a centralized access point for important LangChain abstractions.

The …/__init__.py file defines interfaces for core LangChain concepts. It imports interfaces like BaseCache, BaseDocumentTransformer, BasePromptTemplate, BaseLanguageModel, BaseMemory, and BaseStore from their respective implementation modules. These interfaces standardize how different components interact with each other.

The …/__init__.py file imports language model wrappers from …/base.py. This includes the BaseLLM interface, providing standardized access to models. Classes like LLMChain inherit from BaseLLM to build reusable chains.

The …/__init__.py file defines the Docstore interface that document store implementations inherit from. The Document class represents individual documents.

The …/__init__.py file imports prompt base classes like BasePromptTemplate. This establishes a common interface for prompt generation.

The …/__init__.py file imports reusable agent base classes. This includes interfaces like BaseSingleActionAgent and BaseMultiActionAgent that define agent behavior.

The …/__init__.py file re-exports general utilities from …/__init__.py. This provides core primitives for tasks like configuration, I/O, and math operations.

Integrations

References: libs/partners

The LangChain framework provides integrations with various AI partners through common interfaces defined in the …/partners directory. This allows applications built on LangChain to leverage capabilities from different AI platforms without needing specialized access for each one. The integrations handle authentication, normalizing request/response formats, batching and parallelizing requests to optimize throughput, and retrying failures transparently.

Some key partners integrated include Anthropic, Google AI services, Mistral AI, NVIDIA AI platforms, and Together for collaboration. Each partner has its own subdirectory under …/partners containing code to interface with their respective APIs or models. Common abstractions are defined to provide a consistent way to invoke different partner capabilities.

The …/langchain_anthropic subdirectory implements integration with Anthropic models. The main ChatModel class represents an Anthropic chatbot model and handles authentication, sending requests to Anthropic APIs, and returning predictions. Its predict() method accepts input either as a single example or batch and sends to Anthropic, applying optional parameters like temperature.

For Google integration, the …/langchain_google_genai subdirectory contains key classes like Embeddings and LLMs. Embeddings handles embedding inputs into vectors using Google's pretrained models via its _embed() method. LLMs defines functionality for generative models through methods like _generate() and _stream() that interface with Google APIs. These classes configure clients, validate parameters, and handle conversions between formats.

The …/langchain_mistralai subdirectory provides an interface for Mistral AI chatbots. The core ChatModel class represents a Mistral model and is initialized with a name. Its main methods like _generate(), _astream(), and associated helpers interface directly with the Mistral API for synchronous, asynchronous, and streaming responses. Messages are converted between formats and responses parsed.

Key functionality is also implemented for NVIDIA, Together, and other partners. Classes abstract access to different models through common interfaces, handling tasks like authentication, batching requests, retrying failures, and parsing responses. Partner-specific code reuses common patterns defined in files like /_VertexAICommon. Tests validate interfaces work across models, configurations, and edge cases.

The integrations provide a consistent way to leverage various AI capabilities from partners without needing specialized access for each platform. Common abstractions hide differences in underlying APIs.

Integrations

References: libs/partners, docs/docs/integrations

The LangChain framework provides integrations with external systems and services to enable capabilities like retrieval, generation, data loading and observability in conversational agents and language models. Key integrations include databases, search tools, and model hosting platforms.

Classes implemented across directories handle interactions with different external services. The LLMonitorCallbackHandler class in …/callbacks sends metrics, logs, and errors to the LLMonitor observability tool using callbacks. This allows tracking agent executions.

The AirtableLoader class in …/providers initializes from an Airtable base and table using credentials. Its load() method retrieves records asynchronously based on queries. This allows loading structured data from Airtable into agents.

The Baseten class in the same directory initializes models hosted on the Baseten platform. Its constructor takes a model ID to load a specific model version. The predict() method calls the Baseten API to generate predictions, handling authentication and requests.

The Vearch class provides access to the Vearch vector similarity search service. Its search() method takes a list of vectors and returns similar documents from the Vearch index along with scores. This enables retrieval based on embeddings.

The Remembrall integration in …/memory acts as a proxy for OpenAI API calls, augmenting responses with facts from a knowledge base using the ChatOpenAI class. It also supports retrieval from long-term memory stores via user IDs with the Memory class.

Integrations with AI Partners

References: libs/partners

The …/partners directory contains integrations with various AI partners to provide common interfaces to their models through the LangChain framework. This allows applications built with LangChain to leverage capabilities from different AI platforms.

Key partners integrated include Anthropic, Google AI services, Mistral AI, NVIDIA AI platforms, and Together for collaboration. Each partner is implemented in its own subdirectory under …/partners, following a consistent structure.

Classes define the core functionality for each partner. For example, the ChatModel class in …/__init__.py represents Anthropic models and handles authentication, request formatting, and response parsing. The LLMs class in …/llms.py interfaces with Google language models.

Common patterns are reused, like retry logic defined in _VertexAICommon. Tests validate interfaces across different models and configurations. Scripts enforce coding standards by checking imports and dependencies.

The ChatModel class initializes with a model name and API key to connect to a specific Anthropic model. Its main method is predict(), which accepts input as a single example dictionary or batch list, sends to Anthropic's prediction API, handles responses, and returns predictions.

For Google integration, the LLMs class defines core functionality. Its _generate() method interfaces directly with Google APIs to invoke models. Methods like predict() provide a higher-level interface.

The Project class in …/__init__.py represents a collaborative project. Its join() and leave() methods handle partners joining/leaving a project, while the backend would manage sharing files.

Observability and Monitoring

References: docs/docs/integrations/callbacks

This section covers integrations for monitoring, logging, and tracking executions of LangChain applications. There are two main integration options available - LLMonitor and Streamlit.

The LLMonitorCallbackHandler class is used to send event data from LangChain to the LLMonitor observability platform dashboard. It is initialized with a LLMonitor app ID, which associates the data with your LLMonitor application. This class packages event data into the expected format for the LLMonitor API. It can be passed as a callback to LangChain components via the callbacks argument to ensure all relevant events are tracked. Examples shown in the documentation demonstrate how to use it with LLMs, ChatOpenAI, AgentExecutor and during agent runs by passing the handler to the .run() method.

The identify() method of LLMonitorCallbackHandler provides user tracking functionality. It tags subsequent events with a user_id, allowing segmentation of metrics by individual users in the LLMonitor dashboard. Additional user_props can also be passed to further identify users.

The StreamlitCallbackHandler class is used to display an agent's responses within a Streamlit app in real-time for interactive visualization. It takes a Streamlit container as a parameter and handles rendering output from an agent to the Streamlit app. The example app demonstrates initializing an agent via initialize_agent() and passing the StreamlitCallbackHandler to the agent's .run() method to display the agent's thoughts live as it runs.

Data and Document Loading

References: docs/docs/integrations/document_loaders, docs/docs/integrations/providers

The core functionality for loading data and documents from external sources in LangChain is provided through document loaders. Document loaders abstract the process of connecting to external data sources and importing documents that can then be processed by LangChain pipelines.

The …/example_data directory contains example data that can be loaded by document loaders. This includes sample files like JSON, CSV, HTML and text that simulate real data sources. Loading this example data allows testing and debugging document loading pipelines without connecting to live external systems.

The AirtableLoader class defined in the airtable module provides the main interface for loading documents from an Airtable table into LangChain. Its constructor takes the Airtable credentials and table details needed to connect to a specific Airtable database. The AirtableLoader handles authentication, importing records from the configured table, and returning them as documents that can be processed downstream.

The PubMedLoader class defined in another module similarly handles loading documents retrieved from PubMed. After searching PubMed with the PubMedRetriever, the loaded records can be passed to the PubMedLoader to import them into LangChain's document framework.

The …/fake_discord_data subdirectory simulates Discord chat data through files structured like real user, server, channel and message payloads. Loading this sample data allows testing end-to-end functionality of a potential Discord integration without a live connection.

Programming interfaces for document loading aim to abstract away differences in data sources through common loader classes. The AirtableLoader and PubMedLoader provide standardized ways of accessing Airtable and PubMed within LangChain workflows. This simplifies integration of new data sources.

Example Datasets

References: docs/docs/integrations/document_loaders/example_data

The …/example_data directory contains sample datasets that demonstrate loading different data types. It aims to provide realistic sample input that could be used to test document loading and processing functionality.

The …/fake_discord_data subdirectory contains files structured like data from the Discord platform, including user profiles, server metadata, text channels, and message payloads. This allows end-to-end testing of a Discord integration.

The …/source_code subdirectory contains simple JavaScript and Python source code examples that demonstrate basic object-oriented principles.

The file …/fake-content.html provides an HTML document for testing loading and parsing of web content.

The file …/whatsapp_chat.txt simulates a conversational WhatsApp chat about buying an item, and could support testing NLP models.

Memory and Context

References: docs/docs/integrations/memory

The LangChain integration for memory and context augmentation provides functionality for long-term memory storage and retrieval augmented generation. When making predictions through the ChatOpenAI class, a unique user ID can be specified as a header like "x-gp-remember". This enables the Remembrall integration to track the conversation history for that user. Any facts or statements made during the conversation are then saved to the Remembrall database and can be retrieved on later turns by asking related questions.

For retrieval augmented generation, developers first create a document context in Remembrall using relevant texts or documents. The document ID is specified as a header when making predictions with ChatOpenAI. This allows the language model to incorporate information from the documents when generating responses, leading to more informative answers.

The core components that power these features are:

  • The Remembrall integration, which acts as a proxy for API calls to models like ChatGP and handles conversation state tracking and database operations.

  • The ChatOpenAI class, used for making predictions through the Remembrall proxy. Specifying headers with values like the user ID or document ID configures the desired memory and context features.

  • The Remembrall database that stores stated facts and documents. By saving this contextual information, relevant facts can be retrieved on later turns to enhance responses.

Model Hosting and Management

References: docs/docs/integrations/providers

This section covers integrations with platforms for hosting and managing models. Several classes and integrations are available for deploying models to external services.

The Baseten class from the llms module allows initializing LLMs that are hosted on the Baseten platform. It takes the model ID and deployment as arguments, such as a custom deployment. This provides a standardized interface to query models on Baseten. Once initialized, the Baseten object's () method can be used to query the model through a simple interface directly in LangChain code.

The Fireworks class acts as a client to interface with models deployed on the Fireworks platform. It requires authentication via an API key, allows specifying a model ID, and has a method to generate responses from that model. This provides an easy way to leverage Fireworks models within LangChain applications by authenticating, selecting a model, and generating responses with a single method call.

The Predibase class similarly provides access to models hosted on the Predibase platform. It initializes with a model and predibase_api_key, allowing queries to be sent to a specific model tied to the user's authentication. The class exposes a method that takes an input string to generate responses from the model.

The Portkey integration allows associating requests from an entire LangChain agent or chain together for analysis on the Portkey dashboard. This is done by passing a trace_id to the Portkey Config object. The config object controls Portkey's features like request logging. Features like caching and retries could help improve performance and reliability of a production LangChain deployment.

Conversational Agents

References: templates, cookbook

This section provides examples of building conversational agents, chatbots, and question answering systems using LangChain. The templates directory contains many reusable templates that implement different types of conversational applications.

The …/cohere-librarian subdirectory defines a modular librarian chatbot. It uses the Cohere model for basic chatting capabilities. The core chain class handles the overall workflow by composing reusable components like router for classification and routing of conversations. The router class leverages RunnableBranch to implement routing logic via condition checking. This allows building flexible dialogue systems.

The …/retrieval-agent subdirectory implements a modular conversational agent for document retrieval. The main Agent class processes full requests by validating input, maintaining conversation history, interfacing with retrieval tools via the GmailToolkit class, and parsing outputs. The template demonstrates how to initialize an agent with a model and tools to enable natural language interactions for tasks like information retrieval.

The …/openai-functions-agent-gmail subdirectory contains an agent that executes Gmail functions through conversations. The agent_executor function forms the core logic by parsing requests and delegating to an Agent class. The Agent class initializes with ChatGPT and the GmailToolkit to match requests to tools and return responses. This provides an example of how to build an agent interface for any API or application.

The …/research-assistant subdirectory implements a modular research assistant. It contains reusable search and report generation chains. The multi_search function defines the core search workflow. The DuckDuckGoSearchAPIWrapper handles actual web searching via its search method. The writer module generates reports by selecting templates and querying language models. This demonstrates composable pipelines for complex tasks.

RAG Templates

References: templates/rag-conversation, templates/rag-multi-index-fusion, templates/rag-multi-index-router, templates/rag-vectara, templates/rag-vectara-multiquery

The …/rag-conversation directory implements a conversational question answering template using retrieval augmented generation (RAG) techniques. At the core is the chain class defined in …/chain.py. This class handles the end-to-end workflow of taking a chat history and question as input.

It first normalizes the chat history into HumanMessage and AIMessage objects using the _format_chat_history method. These message objects define the expected format for inputs. The question is then preprocessed using conditional logic - if there is a chat history, the _search_query branch will condense the question into a standalone prompt using the CONDENSE_QUESTION_PROMPT template defined in the file. Otherwise, it will pass the original question directly.

The condensed or original question is used to select relevant context documents from the chat history. This is done by running the question through the ChatOpenAI model initialized in the file, which handles interactions with OpenAI APIs. The top documents returned become the context.

These context documents along with the formatted chat history and question are then passed to the _inputs method to generate prompts in the expected format for the generative model. The main response generation is handled by the ANSWER_PROMPT template defined in the file, which is populated with the prompts and passed to ChatOpenAI to synthesize a coherent response.

The chain orchestrates the overall workflow by initializing ChatOpenAI, running the question through the preprocessing branches, selecting context, populating templates, and generating the final response in a chained manner before returning it. This allows building conversational assistants with RAG techniques that can understand context and answer follow-ups.

Multi-modal Assistants

References: templates/rag-chroma-multi-modal, templates/rag-chroma-multi-modal-multi-vector, templates/rag-multi-modal-mv-local

The templates in this section implement conversational agents capable of reasoning over multiple modalities like images, audio, video and text. Templates like …/rag-chroma-multi-modal and …/rag-multi-modal-mv-local support building assistants that can understand and answer questions involving different media types.

The …/rag-chroma-multi-modal directory contains a template for a visual assistant that reasons over slide decks. It uses the OpenCLIPEmbeddings function defined in …/chain.py to embed images into a joint embedding space with text. When a question is received, the Chroma vector store's as_retriever() method is used in …/chain.py to retrieve relevant slide embeddings based on similarity to the question. The retrieved images and question are then passed to the ChatOpenAI model initialized in …/chain.py to generate a response.

The …/rag-multi-modal-mv-local directory implements a similar agent using a MultiVectorRetriever initialized in …/chain.py to retrieve context over multiple modalities. The ingest.py script in …/rag-multi-modal-mv-local handles ingesting images, using ChatGPT's summarize() function to generate text descriptions, and creating the MultiVectorRetriever. When a question comes in, the retrieve() method uses the retriever to find related image and text embeddings. These are passed to ChatOllama initialized in the same file to generate a response.

SQL and Database Integration

References: templates/rag-astradb, templates/rag-redis, templates/sql-llama2, templates/sql-ollama, templates/sql-pgvector

The templates provide examples of building conversational interfaces to databases. Key templates demonstrate techniques like:

The templates demonstrate techniques for building conversational interfaces that integrate databases and language models. They provide examples of connecting to different database systems, defining prompts and chains to structure multi-turn conversations, and handling tasks like query generation and response formatting.

Multi-stage Pipelines

References: templates/rag-codellama-fireworks, templates/rag-matching-engine, templates/rag-multi-index-fusion, templates/rag-multi-index-router, templates/rag-pinecone-multi-query, templates/rag-pinecone-rerank

This section covers templates that demonstrate complex multi-stage pipelines for question answering. Many of the templates listed here implement sophisticated workflows by chaining together multiple components.

The …/rag-codellama-fireworks template runs a complex multi-stage pipeline for code question answering. It loads a Python codebase from files using GenericLoader and splits the code into overlapped character chunks with RecursiveCharacterTextSplitter. GPT4AllEmbeddings encodes the chunks into vectors, which are stored in the Chroma vector database. When a question is received, Chroma retrieves the most similar code chunks. A ChatPromptTemplate formats the question and context, which is passed to the codellama-34b model via Fireworks for generation. In parallel, retrieval and generation are pipelined using RunnableParallel to maximize efficiency.

The …/rag-matching-engine template implements a complete natural language question answering pipeline. It uses MatchingEngine to perform dense vector searches for context retrieval from a document index. The question, context, and language models are used via VertexAI to generate a response via a multi-stage runnable approach. PromptTemplate and StrOutputParser format the inputs and outputs.

The …/rag-multi-index-fusion template retrieves documents from multiple sources in parallel using RunnableParallel. An embeddings model encodes texts for similarity calculations in fuse_retrieved_docs(). The top documents are selected and passed to answer_chain, which generates a prompt for a GPT-3 response. RunnableParallel and RunnablePassthrough allow different components to run concurrently.

The …/rag-pinecone-multi-query template generates multiple queries from input using an OpenAI model. For each query, MultiQueryRetriever retrieves contexts from Pinecone. Taking the union of all retrieved contexts provides multiple perspectives for answer synthesis.

Modular Agents

References: templates/cohere-librarian, templates/csv-agent, templates/openai-functions-agent-gmail, templates/pirate-speak-configurable, templates/plate-chain, templates/retrieval-agent, templates/research-assistant, templates/xml-agent

The templates demonstrate how to build modular and extensible conversational agents using LangChain. Many of the templates implement agents that can be configured through external files or code to customize their functionality. This allows agents to be adapted for different use cases while keeping the core architecture reusable.

Some key templates that demonstrate modular agent architectures include …/cohere-librarian, …/csv-agent, …/openai-functions-agent-gmail, and …/pirate-speak-configurable. These templates implement agents using configurable components and external files to define aspects like the conversation flow, available tools/skills, prompts and responses.

The …/cohere-librarian template builds a conversational librarian agent. It implements different capabilities like book recommendations, library information and general question answering as independent RunnableBranch components in …/router.py. The router handles classifying the user's topic and routing the conversation to the appropriate branch. This allows mixing and matching different skills without changing code.

The …/csv-agent implements a question answering agent over CSV data. It defines the core processing logic as a list of configurable steps on the Agent class. New steps can be added by defining new methods, making the agent easily extensible for any CSV transformations needed. The agent_executor handles executing all steps in sequence.

The …/openai-functions-agent-gmail builds an agent using Gmail tools exposed to an LLMs API. It initializes a ChatGPT model and binds Gmail tools to it using format_tool_to_openai_function. The Agent class processes full requests by chaining input handling, history formatting, tool response handling and output parsing. This allows adding new tools without changing code.

The …/pirate-speak-configurable implements a text transformation pipeline. The _model field uses ConfigurableField to select different LLMs at initialization via the id property. This makes the core generation component easily configurable. Prompts and responses are defined in external template files.

Self-Supervised Learning

References: templates/self-query-qdrant, templates/self-query-supabase

The templates …/self-query-qdrant and …/self-query-supabase implement conversational agents capable of self-supervised learning through interaction. Both templates utilize a SelfQueryRetriever class to handle querying a backend data store.

The SelfQueryRetriever takes an initial query and uses a language model like OpenAI to generate additional queries. It then queries the backend - either Qdrant for semantic search or Supabase with embedded vectors - to retrieve relevant results. By interacting with users and obtaining implicit feedback on results, the language model is able to gradually improve its ability to understand and respond to the kind of information needs expressed in natural language.

In self-query-qdrant, the SelfQueryRetriever class runs the language model interaction and Qdrant querying in parallel using RunnableParallel. It takes the initial query, prompts the language model to generate additional queries with LLM_CONTEXT_PROMPT, and queries the Qdrant collection populated with example document vectors.

In self-query-supabase, the SelfQueryRetriever first uses the SupabaseVectorStore class to retrieve the top matching document IDs based on embedding similarity to the query from Supabase. It then filters these results using provided metadata before prompting the language model for a response. The SupabaseVectorStore handles all interactions with Supabase like storing and retrieving embedding vectors indexed by document ID.

Both templates implement conversational agents that can improve their ability to understand and respond to user needs over multiple interactions, demonstrating self-supervised learning through interaction with a backend semantic search or embedding space.

Data Extraction and Transformation

References: templates/basic-critique-revise, templates/extraction-anthropic-functions, templates/llama2-functions, templates/mongo-parent-document-retrieval, templates/neo4j-parent, templates/propositional-retrieval

The LangChain templates provide functionality for extracting structured data from unstructured text. Templates like …/extraction-anthropic-functions and …/llama2-functions allow extracting entities, relationships, and other semantic information from documents using large language models.

These templates implement modular pipelines for data extraction. Key components include:

  • Defining input/output schemas using Pydantic models for type checking extracted data
  • Specifying extraction prompts with ChatPromptTemplate classes to guide models
  • Representing model interfaces like AnthropicFunctions and Replicate for querying LLM APIs
  • Parsing model outputs according to schemas with classes like JsonKeyOutputFunctionsParser
  • Chaining extraction steps into reusable pipelines with functions like chain

For example, the …/extraction-anthropic-functions template extracts entities from text using Anthropic's Constitutional AI model. It defines Paper and Info Pydantic schemas, initializes a prompt with ChatPromptTemplate, and calls the model via AnthropicFunctions to extract structured Info objects.

The …/llama2-functions template generates question variations from inputs for search using the LLAMa-2 model. It leverages Replicate to initialize LLAMa-2 and a ChatPromptTemplate to specify the JSON output format. The chain function enables composing these transformations.

These templates provide modular, reusable pipelines for tasks like named entity recognition, relation extraction, and semantic parsing using LLMs. Defining input/output formats and model interfaces allows replacing components while reusing common extraction logic.

Search and Recommendation

References: templates/elastic-query-generator, templates/hybrid-search-weaviate

This section focuses on specialized search and recommendation templates provided by LangChain. Key functionality includes:

The …/elastic-query-generator directory contains a template for building search queries against Elasticsearch databases from natural language. The …/chain.py file defines the chain class, which is used to programmatically construct Elasticsearch queries by chaining individual components together. The chain class represents each query component as an object and provides methods like add() to assemble components into a full query string.

The …/hybrid-search-weaviate template demonstrates hybrid search functionality. The …/chain.py file implements the chain class, which coordinates the overall hybrid search workflow. The chain first initializes a WeaviateHybridSearchRetriever object to query both a Weaviate vector index and an OpenAI model. Relevant documents are retrieved from Weaviate and used as context in a prompt for the OpenAI model to answer questions.

The WeaviateHybridSearchRetriever class handles the core search functionality. It initializes a Weaviate Python client and issues search queries that combine BM25 and vector similarity scores to surface the most relevant documents. A ChatPromptTemplate is used to format questions and context into a prompt for the conversational model. This allows leveraging both Weaviate's search and an AI model's question answering abilities for a more engaging search experience.

Knowledge Bases and Graphs

References: templates/cassandra-entomology-rag, templates/cassandra-synonym-caching, templates/neo4j-cypher-ft, templates/neo4j-parent, templates/rag-astradb

This section covers templates for building knowledge graphs and performing question answering over graphs. The templates provide functionality for modeling data as graphs using databases like Neo4j and Cassandra, and answering natural language questions by querying the graphs.

Some templates focus on ingesting text documents into a graph database and splitting the documents into parent and child nodes connected by relationships. For example, the …/neo4j-parent template loads documents from files and splits them into parent and child chunks with overlap using the TextLoader and TokenTextSplitter classes. The parent and child nodes are connected in Neo4j with relationships. Embeddings are also computed for each child node using OpenAIEmbeddings and stored as properties on the nodes, indexed by Neo4jVector for retrieval.

Other templates demonstrate question answering directly over graphs. The …/neo4j-cypher-ft template maps natural language questions to Cypher queries that can be executed against a Neo4j graph. It uses the Neo4jGraph class to interface with Neo4j and execute queries via query(). Entity extraction is performed with qa_llm, entities are mapped to the graph, Cypher queries are generated with cypher_llm, and responses are produced with qa_llm chained together with RunnablePassthrough. The Chain class defines the full pipeline.

The …/cassandra-entomology-rag template models insect taxonomic data in Cassandra and answers questions by retrieving similar context vectors. The Cassandra class interfaces with Cassandra, populate() loads data, and a retriever retrieves from the vector_store. ChatPromptTemplate generates prompts from context and questions for ChatOpenAI. The chain connects the components.

The templates demonstrate modeling a variety of data as graphs, querying the graphs to answer natural language questions, and chaining together modular components for end-to-end question answering. Templates provide reusable pipelines and abstractions for building knowledge graphs and conversational systems over graph-structured data.

Model Hosting and Management

References: templates/rag-aws-bedrock, templates/rag-codellama-fireworks

The …/chain.py module initializes a Fireworks class to interface with the codellama-34b model. The Fireworks class takes a Fireworks API key as a parameter to authenticate requests. When a prompt string is passed to the Fireworks class' generate() method, it forwards the request to the Fireworks API and returns the response. This allows leveraging the powerful codellama-34b model for code question answering tasks through a simple API.

Documentation

References: docs

The Documentation section covers documentation source files, scripts, tools and static assets used for generating API references, guides, and the overall documentation site. Key functionality includes:

The …/docs directory contains Markdown files that provide documentation on important topics like guides, integrations, modules, and community participation. Files like debugging.md and pydantic_compatibility.md give guidance on debugging and compatibility with Pydantic.

The …/scripts directory contains Python scripts that programmatically generate documentation content. The model_feat_table.py script inspects LLM and ChatModel classes to detect supported features and generate Markdown tables documenting these capabilities. It uses the LLM, BaseLLM, SimpleChatModel, and BaseChatModel classes defined in langchain_core.

The …/api_reference directory contains files for generating API reference documentation with Sphinx. The create_api_rst.py script parses Python modules using the ClassInfo, FunctionInfo, and ModuleMembers classes to collect metadata and generate .rst files. It recursively loads modules using _load_package_modules() and documents members with _construct_doc().

The …/src directory contains the source code for the documentation site built with React. The CodeBlockWrapper component in …/CodeBlock dynamically parses imports in code blocks for documentation pages.

Static assets like CSS stylesheets are contained in …/static. The google_analytics.js file in …/js initializes Google Analytics tracking on the site using the gtag() function.

API References

References: docs/api_reference

The …/api_reference directory contains documentation for the Langchain API reference generated using Sphinx. Sphinx is used to automatically generate API reference documentation by parsing Python modules and documenting classes, functions, and enums.

The core generation is handled by the …/create_api_rst.py script. This script uses the ClassInfo, FunctionInfo, and ModuleMembers classes to collect metadata on modules, classes, functions, and enums through introspection. It recursively loads all modules in packages and subpackages using the _load_package_modules() function. This builds a dictionary of module members organized by namespace.

The _construct_doc() function then generates reStructuredText files by iterating through members and selectively documenting them based on access and type. Template files like class.rst are used to format different member types. This modular approach separates concerns for maintainability.

Configuration is provided in …/conf.py, including a custom ExampleLinksDirective class. This class generates example links for API items by parsing a JSON file. The ExampleLinksDirective takes a class or function name and outputs a bullet list of links.

Styling is handled by CSS in …/custom.css. This file implements styles for preformatted text, responsive layouts, element positioning, and code formatting within tables through CSS selectors and rules.

Themes are defined under …/themes, including templates used to generate pages. Templates like COPYRIGHT.txt are also used here and in …/templates.

Documentation Source

References: docs/docs

The …/docs directory contains Markdown files that provide documentation on key aspects of developing applications with LangChain. This includes guides, integration information, and community documentation.

The …/guides subdirectory contains files such as debugging.md and pydantic_compatibility.md that provide guidance on debugging LangChain apps and ensuring compatibility with Pydantic. The debugging.md file describes tools like the set_debug() and set_verbose() global functions for controlling log verbosity. It also mentions the LangSmith and WandB tracing platforms.

The …/integrations subdirectory documents integrating external systems. For example, the …/callbacks directory contains documentation on callback handlers for sending event data, such as the LLMonitorCallbackHandler and StreamlitCallbackHandler classes. The …/document_loaders subdirectory demonstrates loading different document types with sample data files and code examples.

The …/modules subdirectory contains documentation on loading, preprocessing, and working with data. For example, the Dataset class in …/model_io defines a base class for representing datasets.

The …/community.md file provides information on getting involved in the LangChain community through contributing, events, blogging, and social media.

Documentation Generation

References: docs/scripts

The Python scripts in …/scripts handle automatically generating documentation from Langchain code and models. This ensures documentation remains accurate and in sync as the codebase evolves. The main scripts are:

  • …/copy_templates.py copies README files from templates to the docs folder, processing image links.

  • …/generate_api_reference_links.py searches documentation for code blocks, extracts import statements, resolves imported class names to module paths, and generates links to documentation pages for the classes.

  • …/model_feat_table.py programmatically generates Markdown tables documenting features of LLMs and ChatModels.

The generate_api_reference_links.py script handles linking imported classes to documentation. It uses the find_files() function to search documentation, then processes each file:

All import mappings are stored in a JSON file and replace_imports() replaces imports in code blocks with the generated links. This allows linking imported classes across the codebase.

The model_feat_table.py script generates model feature tables by inspecting the LLM, BaseLLM, SimpleChatModel, BaseChatModel classes to detect supported features. Features can be overridden via correction dictionaries. The get_llm_table() and get_chat_model_table() functions do the main feature detection work.

Documentation Site

References: docs/src

The source code for the Langchain documentation site is contained in the …/src directory. This directory implements all of the content, styling, routing, and component logic for the documentation site using React.

The key subdirectories that power the documentation site include:

  • …/css: Contains Cascading Style Sheets (CSS) that define the visual styling and layout of documentation pages. The main stylesheet is …/custom.css which overrides default styles and defines custom colors, dark mode styling, and component appearances.

  • …/pages: Implements the main documentation pages as React components. Routing is handled by the function defined in …/index.js which redirects users to the introduction.

  • …/theme: Contains components and utilities that customize the documentation theme. The CodeBlockWrapper component in …/index.js dynamically parses code examples to display imports.

The business logic implemented in key files includes:

  • …/index.js: The function returns a component which handles redirecting users from the root page / to the introduction documentation at /docs/get_started/introduction on initial page load.

  • …/index.js: The CodeBlockWrapper component checks if the children prop is a string, uses a regex to search for imports in a comment, populates an imports array, renders the original component and code, and conditionally displays the component passing the imports array.

Styling and Assets

References: docs/static

The …/static directory contains static assets used by the LangChain documentation site. The key subdirectories are:

  • …/js holds JavaScript files that add interactivity. The main file is …/google_analytics.js which initializes Google Analytics tracking. It defines the gtag() function to asynchronously log tracking events without blocking loads.

By separating static assets into subdirectories based on type (JS), it keeps the file structure clean and easy to navigate. Dependencies between files are managed through relative paths. This structure provides a clear separation of concerns for building the frontend.

Experimentation

References: libs/experimental, docs/docs/langsmith

The …/autonomous_agents directory contains code and architectures for implementing different types of autonomous intelligent agents with language models. It defines common interfaces and base classes that agent implementations can inherit from to gain core functionality.

The …/langsmith section describes tools provided for debugging, tracing, and analyzing language model applications. These tools are intended to help users better understand how their models are behaving and identify areas for improvement.

The …/baby_agi directory implements a basic autonomous agent called BabyAGI for experimentation. The BabyAGI class coordinates task execution, creation and prioritization using three reusable chains.

The …/hugginggpt directory contains an agent implementation called HuggingGPT for experimenting with task planning and execution techniques. The HuggingGPT agent class initializes a language model, TaskPlanner, TaskExecutor, and ResponseGenerator to handle taking inputs, planning tasks, executing the plan, and generating responses. The TaskPlanner interacts with the language model to generate a Plan object representing the task sequence. The TaskExecutor then handles running each individual Task based on the plan.

Autonomous Agents

References: libs/experimental/langchain_experimental/autonomous_agents

The core autonomous agent code is contained within the …/autonomous_agents directory. This directory provides implementations of various types of autonomous agents based on language models through modular and reusable components.

Three main agent classes are defined - BabyAGI, AutoGPT, and HuggingGPT. BabyAGI in /baby_agi/baby_agi.py implements a basic artificial general intelligence agent. It coordinates task execution, creation and prioritization using the TaskCreationChain, TaskExecutionChain, and TaskPrioritizationChain defined in /baby_agi.

AutoGPT in /autogpt/agent.py controls conversation with GPT-3 using a prompt, memory store, and output parser. The AutoGPTPrompt class generates prompts containing goals and context, while AutoGPTMemory in /autogpt/memory.py manages relevant context and goals retrieved from a vector store using its retriever field.

HuggingGPT in /hugginggpt/hugginggpt.py implements task planning with a TaskPlanner and execution with a TaskExecutor defined in /hugginggpt/task_executor.py to handle user requests. The TaskExecutor initializes Task objects for each step in a plan returned by the planner. It checks dependencies before executing each task's run() method, which uses BaseTool to execute different task types.

Common interfaces are defined in /autonomous_agents/__init__.py and chains are initialized from language models. The chains separate concerns into modular components like task creation, execution and prioritization, while the agent classes coordinate workflow. This provides a framework for implementing different types of autonomous agents based on language models.

Debugging and Analysis Tools

References: docs/docs/langsmith

The …/langsmith directory contains documentation and code related to LangSmith, a tool for debugging, tracing, and analyzing language model applications. LangSmith helps users evaluate their models and agents to improve them.

The main index.md file introduces LangSmith and provides links to resources like an interactive walkthrough and full documentation. The LangSmith Cookbook contains examples demonstrating how to integrate LangSmith into different workflows. This includes leveraging user feedback in a JS application, building an automated feedback pipeline, evaluating and auditing RAG workflows, fine-tuning an LLM on real usage data, and using the LangChain Hub to version prompts.

The autonomous_agents subdirectory contains code and architectures for implementing autonomous intelligent agents. This code allows programmers to build agents that can learn and improve over time from interactions and data collection. The autonomous_agents code implements various machine learning techniques to allow agents to become more helpful, harmless, and honest through self-supervised learning.

The langsmith subdirectory contains the core LangSmith Python package. The langsmith package provides the LangSmith class for initializing LangSmith and collecting telemetry data on language models. It also contains the Trace class which can be used to trace the execution of language models and capture runtime information. The Trace class implements a decorator pattern to trace function calls. Programmers can leverage the LangSmith and Trace classes to debug their agents and better understand model behavior.

Development Environment

References: .devcontainer

The .devcontainer directory contains configuration files that allow developers to set up a consistent Docker development environment for contributing to LangChain. This avoids needing to install tools locally and provides a reproducible environment.

The main configuration files are …/README.md. The README.md file documents how to use the development container with either GitHub Codespaces or the VS Code Dev Containers extension. It provides instructions for initializing Codespaces directly on GitHub or opening a local container in VS Code. The file also includes tips for resolving common issues that may occur like git conflicts.

Some key functionality available in the container includes classes like LangChain for interacting with the blockchain and LangChain for managing accounts. These abstract away details and provide a consistent way for developers to work with the codebase.