vector-db-sink
Last updated
Last updated
This agent writes vector data to vector databases. LangStream currently supports and .
Astra DB and Pinecone are both of the type "vector-db-sink" in a LangStream pipeline, but the databases require different configuration values to map the vector data from the sink into the database.
The Astra DB vector database connection is defined in configuration.yaml:
The "Write to Astra DB" pipeline step takes embeddings as input from "input-topic" and writes them to the configured datasource "AstraDatasource":
Input
Output
None, it’s a sink.
datasource
String
The datasource is defined in the Resources section of configuration.yaml.
table
String
The `keyspace.table-name` the vector data will be written to
mapping
String
How the data from the input records will be mapped to the corresponding columns in the database table. "id=value.id" maps the "id" value in the input record to the "id" value of the database.
The "Write to Pinecone" pipeline step takes embeddings as input from "vectors-topic" and writes them to a Pinecone datasource.
The Pinecone vector database connection is defined in configuration.yaml:
The "Write to Pinecone" pipeline step takes embeddings as input from "input-topic" and writes them to the configured datasource "PineconeDatasource":
Input
Output
None, it’s a sink.
datasource
String
The datasource is defined in the Resources section of configuration.yaml.
vector.id
String
Maps id to vector.id
vector.vector
String
Maps the input value "vector" to "vector.vector" in the database.
vector.namespace
String
Maps the input value "namespace" to "vector.namespace" in the database.
vector.metadata.{metadataField}
String
Maps the input value "metadata.{metadataField}" to "vector.metadata.{metadataField}"
Structured and unstructured text
Implicit topic
Templating
Structured and unstructured text
Implicit topic
Templating