A cache for storing LLM generations that stores data in memory.

Type Parameters

Hierarchy (view full)

Constructors

Methods

  • Retrieves data from the cache using a prompt and an LLM key. If the data is not found, it returns null.

    Parameters

    • prompt: string

      The prompt used to find the data.

    • llmKey: string

      The LLM key used to find the data.

    Returns Promise<null | T>

    The data corresponding to the prompt and LLM key, or null if not found.

  • Updates the cache with new data using a prompt and an LLM key.

    Parameters

    • prompt: string

      The prompt used to store the data.

    • llmKey: string

      The LLM key used to store the data.

    • value: T

      The data to be stored.

    Returns Promise<void>

""