LangStream Documentation
Langstream.aiLangStream GitHub RepoChangelog
  • LangStream Documentation
  • ❤️Langstream.ai
  • ⭐LangStream GitHub Repo
  • 📜Changelog
  • about
    • What is LangStream?
    • License
  • Get Started
  • installation
    • LangStream CLI
    • Docker
    • Minikube (mini-langstream)
    • Kubernetes
    • Build and install from source
  • Building Applications
    • Vector Databases
    • Application structure
      • Pipelines
      • Instances
      • Configuration
      • Topics
      • Assets
      • Secrets
      • YAML templating
      • Error Handling
      • Stateful agents
      • .langstreamignore
    • Sample App
    • Develop, test and deploy
    • Application Lifecycle
    • Expression Language
    • API Gateways
      • Websocket
      • HTTP
      • Message filtering
      • Gateway authentication
    • API Reference
      • Agents
      • Resources
      • Assets
  • LangStream CLI
    • CLI Commands
    • CLI Configuration
    • Web interface
  • Integrations
    • Large Language Models (LLMs)
      • OpenAI
      • Hugging Face
      • Google Vertex AI
      • Amazon Bedrock
      • Ollama
    • Data storage
      • Astra Vector DB
      • Astra
      • Cassandra
      • Pinecone
      • Milvus
      • Solr
      • JDBC
      • OpenSearch
    • Integrations
      • Apache Kafka Connect
      • Apache Camel
    • LangServe
  • Pipeline Agents
    • Agent Messaging
    • Builtin agents
      • Input & Output
        • webcrawler-source
        • s3-source
        • azure-blob-storage-source
        • sink
        • vector-db-sink
        • camel-source
      • AI Agents
        • ai-chat-completions
        • ai-text-completions
        • compute-ai-embeddings
        • flare-controller
      • Text Processors
        • document-to-json
        • language-detector
        • query
        • query-vector-db
        • re-rank
        • text-normaliser
        • text-extractor
        • text-splitter
        • http-request
      • Data Transform
        • cast
        • compute
        • drop
        • drop-fields
        • merge-key-value
        • unwrap-key-value
      • Flow control
        • dispatch
        • timer-source
        • trigger-event
    • Custom Agents
      • Python sink
      • Python source
      • Python processor
      • Python service
    • Agent Developer Guide
      • Agent Types
      • Agent Creation
      • Configuration and Testing
      • Environment variables
  • Messaging
    • Messaging
      • Apache Pulsar
      • Apache Kafka
      • Pravega.io
  • Patterns
    • RAG pattern
    • FLARE pattern
  • Examples
    • LangServe chatbot
    • LlamaIndex Cassandra sink
Powered by GitBook
On this page
  • Source
  • Sink
  • Processor
  • Service
  • What's next?
Edit on GitHub
  1. Pipeline Agents
  2. Agent Developer Guide

Agent Types

Part 1 of the Agent Developer Guide

PreviousAgent Developer GuideNextAgent Creation

Last updated 1 year ago

LangStream offers 3 different types of agents. A agent is the start of a pipeline, a agent writes pipeline results to another service, and a agent is a step in the pipeline. The gist of developing an agent is to implement one of its APIs (, , or ). You provide the application’s source (the .py) files when deploying to the Control Plane. All of Python’s constructs and can be used as you build the agent.

It’s typical to run the Control Plane as an internal service within an Organization. An agent’s outgoing access might be limited to the internal network, or might have internet access. This may create incompatibility with LangStream’s pre-built agents, or you may want to include domain knowledge specific to processing within the agent.

Source

A source agent is responsible for bringing data into the application’s pipeline.

A source agent is the first step of a pipeline and replaces the use of an input topic. A source can be anything. The agent is responsible for retrieving data from a service and uses the LangStream API to create a Record that encapsulates that data. LangStream will produce a message on the specified topic for other agents to consume.

Sink

A sink agent is responsible for writing data to a service outside the application pipeline.

The agent is responsible for accepting a list of Records as input and writing the value to the given service. A sink agent may be the last step in a pipeline or may capture data and continue to the next steps in the pipeline.

Processor

Processor agents are typically placed throughout an application’s pipeline.

A processor agent might manipulate data as it flows through the pipeline, or could add in context to help downstream agents make decisions. A processor agent is responsible for accepting a list of Records as input, doing some processing as necessary, and returning a Record (or many Records) as a result.

Service

Service agents are generic applications that usually do not process streaming data in the scope of a pipeline.

Typically a Service exposes an API service that can be consumed by external applications. For instance you can build your ChatBot UI using a Service

What's next?

Continue on to Part 2 of the Agent Developer Guide,

source
sink
processor
Source
Sink
Processor
duck typing
Agent Creation.