Securing LLM API Keys with AWS Secrets Manager in Node.js

Protecting API keys is a key concern, especially with large language models (LLMs) like OpenAI or Claude. Many developers slip up by embedding these keys directly into their codebase, a setup ripe for accidental leaks and tedious rotation periods. We're going to see how AWS Secrets Manager can provide a more secure method to store, rotate, and retrieve these keys efficiently at runtime in a Node.js setup.

Why Consider AWS Secrets Manager Instead of Environment Variables?

Environment variables may be easy to use, but they come with their fair share of risks. They're stored as plain text on the host machine, might be logged, and, oops, sometimes checked into version control by mistake. AWS Secrets Manager, however, is like a digital fortress for your keys. It encrypts secrets at rest with AWS KMS keys, offers fine-grained access control using IAM policies, and supports automatic rotation of secrets. This keeps your LLM credentials safe and sound.

Code Comparison

// Bad Practice: Hard-coded environment variable
const apiKey = process.env.CLAUDE_API_KEY;

// Good Practice: Fetch from AWS Secrets Manager
import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager";
const client = new SecretsManagerClient({ region: "us-east-1" });
async function getKeyFromSM(name: string) {
  const cmd = new GetSecretValueCommand({ SecretId: name });
  const resp = await client.send(cmd);
  return resp.SecretString ?? "";
}

Using AWS Secrets Manager means your key isn't stored in your source code, and AWS logs each access, creating an auditable trail for you to review if needed.

Automatic Rotation Setup for LLM API Keys

Think of API keys as being like passwords. If a key is exposed, unauthorized use can lead to unexpected costs and damage your reputation. Rotating keys regularly limits the window of opportunity for attackers.

Rotation Workflow

• Create a secret: Store your initial key as a JSON payload, like {"key":"sk-abc123"}.
• Enable rotation: Connect the secret to a Lambda function that requests a new key from the provider.
• Schedule: Have AWS run the Lambda function at your chosen intervals, such as every 30 days.
• Versioning: Each rotation produces a new version. The AWSCURRENT label indicates the latest version.

If the format changes during rotation, you might run into JSON parse errors. Handling these will require implementing some retry logic.

Lambda Function for Rotation

// rotate-claude-key.ts
import {
  SecretsManagerClient,
  GetSecretValueCommand,
  PutSecretValueCommand,
} from "@aws-sdk/client-secrets-manager";
import fetch from "node-fetch";
const sm = new SecretsManagerClient({ region: "us-east-1" });
export const handler = async (event: any) => {
  const getCmd = new GetSecretValueCommand({ SecretId: event.SecretId });
  const current = await sm.send(getCmd);
  const oldPayload = JSON.parse(current.SecretString ?? "{}");
  const resp = await fetch("https://api.anthropic.com/v1/keys/rotate", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${oldPayload.key}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ reason: "scheduled rotation" }),
  });
  const newData = await resp.json();
  const putCmd = new PutSecretValueCommand({
    SecretId: event.SecretId,
    SecretString: JSON.stringify({ key: newData.key }),
    VersionStages: ["AWSCURRENT"],
  });
  await sm.send(putCmd);
};

Make sure your Lambda function is efficient to keep costs down and avoid leaving your secret in an inconsistent state.

Ensuring Type-Safe Retrieval in Node.js

The secrets you fetch are strings, which you then parse into objects. This can result in runtime errors if the JSON is off. TypeScript’s satisfies operator comes in handy here, providing type safety without the need for explicit casting.

// secret-types.ts
export interface ClaudeSecret {
  key: string;
}

// getClaudeKey.ts
import {
  SecretsManagerClient,
  GetSecretValueCommand,
  ResourceNotFoundException,
} from "@aws-sdk/client-secrets-manager";
import { ClaudeSecret } from "./secret-types";
const client = new SecretsManagerClient({ region: "us-east-1" });
export async function fetchClaudeKey(secretName: string): Promise {
  try {
    const cmd = new GetSecretValueCommand({ SecretId: secretName });
    const resp = await client.send(cmd);
    const raw = resp.SecretString ?? "{}";
    const parsed = JSON.parse(raw) as unknown;
    if ((parsed as ClaudeSecret).key === undefined) {
      throw new Error("Secret does not contain a 'key' field");
    }
    const secret = parsed satisfies ClaudeSecret;
    return secret.key;
  } catch (err) {
    if (err instanceof ResourceNotFoundException) {
      console.warn("Secret not found – possibly during rotation. Retrying later.");
      throw err;
    }
    throw err;
  }
}

This way, you're sure at compile time that the secret JSON is correctly shaped, keeping surprise runtime errors at bay.

Caching Secrets Locally Without Stale Data

Caching secrets locally cuts down on both costs and latency from API calls. An in-memory cache can handle the same secret for multiple requests, just ensure it doesn’t serve stale data following a rotation.

Cache Design

• Singleton: One instance per Node process.
• TTL (time-to-live): Refresh secrets after a short interval, say 5 minutes.
• Version check: If the VersionId changes, immediately update the cached value.

// secretCache.ts
import {
  SecretsManagerClient,
  GetSecretValueCommand,
  GetSecretValueResponse,
} from "@aws-sdk/client-secrets-manager";
import { ClaudeSecret } from "./secret-types";
type CacheEntry = {
  secret: ClaudeSecret;
  versionId: string;
  expiresAt: number;
};
class SecretCache {
  private client = new SecretsManagerClient({ region: "us-east-1" });
  private cache: Map = new Map();
  private readonly ttlMs = 5 * 60 * 1000; // 5 minutes
  async get(secretName: string): Promise {
    const now = Date.now();
    const entry = this.cache.get(secretName);
    if (entry && entry.expiresAt > now) {
      return entry.secret;
    }
    const fresh = await this.fetchFromSM(secretName);
    this.cache.set(secretName, {
      secret: fresh.secret,
      versionId: fresh.versionId,
      expiresAt: now + this.ttlMs,
    });
    return fresh.secret;
  }
  private async fetchFromSM(name: string): Promise<{ secret: ClaudeSecret; versionId: string }> {
    const cmd = new GetSecretValueCommand({ SecretId: name });
    const resp: GetSecretValueResponse = await this.client.send(cmd);
    const raw = resp.SecretString ?? "{}";
    const parsed = JSON.parse(raw) as unknown;
    const secret = parsed satisfies ClaudeSecret;
    const versionId = resp.VersionId ?? "";
    return { secret, versionId };
  }
}
export const secretCache = new SecretCache();

With the cache expiring every 5 minutes, you minimize lag while ensuring fresh data is fetched after each rotation.

Putting It All Together: A Minimal AI Agent

Let's build a simple function, askClaude, that retrieves the Claude API key from Secrets Manager, calls Claude’s /v1/complete endpoint, and returns the generated text.

// aiAgent.ts
import { secretCache } from "./secretCache";
import fetch from "node-fetch";
const CLAUDE_SECRET_NAME = "prod/claude/api-key";
export async function askClaude(prompt: string): Promise {
  const secret = await secretCache.get(CLAUDE_SECRET_NAME);
  const apiKey = secret.key;
  const body = {
    model: "claude-3-sonnet-20240229",
    prompt,
    max_tokens_to_sample: 256,
  };
  const resp = await fetch("https://api.anthropic.com/v1/complete", {
    method: "POST",
    headers: {
      "x-api-key": apiKey,
      "content-type": "application/json",
    },
    body: JSON.stringify(body),
  });
  if (!resp.ok) {
    if (resp.status === 401) {
      console.warn("Claude returned 401 – key may be stale. Flushing cache and retrying.");
      secretCache["cache"].delete(CLAUDE_SECRET_NAME);
      throw new Error("Authentication failed – retry later");
    }
    const errBody = await resp.text();
    throw new Error(`Claude API error ${resp.status}: ${errBody}`);
  }
  const result = await resp.json();
  return result.completion?.trim() ?? "";
}
export const handler = async (event: any) => {
  const userPrompt = event.body?.prompt ?? "Tell me a joke.";
  try {
    const answer = await askClaude(userPrompt);
    return { statusCode: 200, body: JSON.stringify({ answer }) };
  } catch (e) {
    console.error("Failed to get Claude response:", e);
    return { statusCode: 500, body: "Internal server error" };
  }
};

This configuration uses your API keys efficiently while ensuring they stay secure and ready for action, even as you rotate keys.