API Gateways

API gateways in LangStream are HTTP/WebSocket endpoints that allow applications to interact with agents via message topics.

Messages contain a key, a value, and headers for metadata. This metadata is used to connect clients to topics and can be used to filter messages.

LangStream applications can have three different gateway types:

Producer gateways are clients that create a message and write it to a topic.

Consumer gateways are clients that watch a topic for messages from a given position. This gateway can only be accessed via WebSocket.

Chat gateways create a producer client (questions-topic) and a consumer client (answers-topic) across one connection, optimized for "chat" style interactions between client and server. This gateway can only be accessed via WebSocket.

Service (topics) gateways create a producer client (input-topic) and a consumer client (output-topic), writing a single message and awaiting for the receivement of another message. This gateway can only be accessed via HTTP.

Service (agents) gateways send requests directly to a service agent and return the response to the client. This gateway can only be accessed via HTTP.

Example gateways

Given an application in tenant “my-super-cool-tenant” with an id of “some-application” and the following gateway manifest, 1 gateway of each type is created:

gateways:
  - id: "user-input"
    type: produce
    topic: "questions-topic"

  - id: "bot-output"
    type: consume
    topic: "answers-topic"
    
  - id: "chat"
    type: chat
    chat-options:
      questions-topic: questions-topic
      answers-topic: answers-topic

  - id: "service"
    type: service
    service-options:
      input-topic: questions-topic
      output-topic: answers-topic

The corresponding URLs to the gateways would be:

Produce gateway: ws://localhost:8091/produce/my-super-cool-tenant/some-application/user-input

Consume gateway: ws://localhost:8091/consume/my-super-cool-tenant/some-application/bot-output

Chat gateway: ws://localhost:8091/consume/my-super-cool-tenant/some-application/chat

Service gateway: http://localhost:8091/api/gateways/service/my-super-cool-tenant/some-application/service

The URL structure is ws://<control-plane-domain>:<api-gateway-port>/<gateway-type>/<tenant-name>/<application-id>/<gateway-id> for WebSocket.

The URL structure is http://<control-plane-domain>:<api-gateway-port>/api/gateways/<gateway-type>/<tenant-name>/<application-id>/<gateway-id> for HTTP.

Gateways configuration

Root	Node	Type	Description
gateways			The base node in the yaml. Holds the collection of gateways.
	id	string (required)	An authentic name for the given gateway. Allowed characters are a-zA-Z0-9._-
	type	string (required)	The type of gateway to create. Supported values are: produce consume chat Set to "produce" to use the gateway to create messages in the topic. Set to "consume" to use the gateway to consume messages in the topic. Set to "chat" to create a producer topic (answers-topic) and a consumer topic (questions-topic) and connect them through the gateway.
	topic	string (required)	The mapping of a topic declared in pipeline.yaml manifest. This is where a gateway produces and consumes messages. This value must match the name of a topic in the pipeline manifest.
	parameters	string[]	A string collection of labels to look for in the querystring. See Message filtering.
	headers	object	The values to use as headers on produced messages, that hold the given parameters.
	authentication	object	Configuration of an identity provider. See Gateway authentication.
	produce-options	object	See headers.
	consume-options	object	See headers.
	chat-options	object	See chat-options.
	headers	map<String, String>	See headers.

chat-options

Label	Type	Description
answers-topic	string	The name of the chat consumer topic, ex "input-topic".
questions-topic	string	The name of the chat producer topic, ex "output-topic".
headers	object	The values to use as headers on produced messages, that hold the given parameters. See headers.

headers

Label	Type	Description
key	string	An identifier of the value. This can be thought of as the name. Allowed characters are a-zA-Z0-9._-
value-from-parameters	string	The mapped name to connect a querystring value in the Gateway URL to message headers. This must match a value set in the parameters collection.
value-from-authentication	string	The mapped name to connect a provided querysting value with the identity provider’s handshake value. See Gateway authentication.

Example gateways manifest

https://github.com/LangStream/langstream/blob/main/examples/applications/gateway-authentication/gateways.yaml

#
#
# Copyright DataStax, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

gateways:
  - id: produce-input-no-auth
    type: produce
    topic: input-topic
    parameters:
      - sessionId
    produce-options:
      headers:
        - key: langstream-client-session-id
          value-from-parameters: sessionId
  - id: consume-output-no-auth
    type: consume
    topic: output-topic
    parameters:
      - sessionId
    consume-options:
      filters:
        headers:
          - key: langstream-client-session-id
            value-from-parameters: sessionId
  - id: chat-no-auth
    type: chat
    chat-options:
      headers:
      - value-from-parameters: session-id
      questions-topic: input-topic
      answers-topic: output-topic

  - id: produce-input-auth-google
    type: produce
    topic: input-topic
    parameters:
      - sessionId
    authentication:
      provider: google
      allow-test-mode: true
      configuration:
        clientId: "${secrets.google.client-id}"
    produce-options:
      headers:
        - key: langstream-client-user-id
          value-from-authentication: subject
        - key: langstream-client-session-id
          value-from-parameters: sessionId

  - id: consume-output-auth-google
    type: consume
    topic: output-topic
    parameters:
      - sessionId
    authentication:
      allow-test-mode: true
      provider: google
      configuration:
        clientId: "${secrets.google.client-id}"
    consume-options:
      filters:
        headers:
          - key: langstream-client-user-id
            value-from-authentication: subject
          - key: langstream-client-session-id
            value-from-parameters: sessionId

  - id: produce-input-auth-github
    type: produce
    topic: input-topic
    parameters:
      - sessionId
    authentication:
      provider: github
      configuration:
        clientId: "${secrets.github.client-id}"
    produce-options:
      headers:
        - key: langstream-client-user-id
          value-from-authentication: login
        - key: langstream-client-session-id
          value-from-parameters: sessionId

  - id: consume-output-auth-github
    type: consume
    topic: output-topic
    parameters:
      - sessionId
    authentication:
      provider: github
      configuration:
        clientId: "${secrets.github.client-id}"
    consume-options:
      filters:
        headers:
          - key: langstream-client-user-id
            value-from-authentication: login
          - key: langstream-client-session-id
            value-from-parameters: sessionId