Skip to main content

LLM Provider

LLM capability is injected into a workflow through one provider interface. Public workflows use their own provider configuration and API key; the Server does not store or proxy third-party model credentials for a steamharness.

Create a provider ref​

Create a provider ref so that nodes resolve configuration when they run:

const llmProvider = steamharness.createLLMProviderRef("default", () => {
  return steamharness.createLLMProviderFromConfig();
});

The node resolves the provider at runtime instead of reading business configuration while the module is loading.

LLM nodes​

Non-streaming node:

const llmNode = steamharness.createLLMNode<Input>({
  name: "answer-llm",
  provider: llmProvider,
  mapInput(input) {
    return {
      system: "You are a concise assistant.",
      prompt: input.message,
      maxOutputTokens: 300,
    };
  },
});

Streaming nodes use the same provider:

const llmStreamNode = steamharness.createLLMStreamNode<Input>({
  name: "answer-stream",
  provider: llmProvider,
  mapInput(input) {
    return { messages: [{ role: "user", content: input.message }] };
  },
});

Project configuration fields​

steamharness.createLLMProviderFromConfig() reads:

  • LLM_TYPE
  • LLM_API_KEY
  • LLM_BASE_URL
  • LLM_MODEL

Declare these fields in configuration.json and enter their values in Project configurations. Values come from the immutable run snapshot and never fall back to .env or process variables. LLM_BASE_URL must be an HTTP(S) URL without credentials, query parameters, or a fragment.

Third-party services use their own fields through steamharness.getConfig(). Host account credentials and transport settings remain host managed.

Provider types and errors​

The provider factory supports the types in LLM_PROVIDER_TYPES. When no model is configured, it selects a default by provider type; tests can use a mock provider.

A provider failure becomes a workflow error in the execution report. Unsupported optional settings may produce a compatibility warning; authentication failures, unavailable models, and upstream request failures remain errors.