Creating a Proxy Debugging Index: Practical Steps for Developers

Debugging proxy issues in complex web applications can feel like you're searching for a needle in a haystack. A typical browser trace might contain hundreds of requests, yet analyzing just a few key ones can unravel the problem. The real challenge? Pinpointing those crucial requests amidst all the noise.

Today's applications often spread a single user action across various hosts and services. One action could touch multiple APIs, identity endpoints, or asset hosts, potentially on different ports. If something goes wrong during this journey, a broad network snapshot might skip over the first sign of trouble. On the flip side, a narrow hostname filter might miss important elements such as redirects or token refreshes. The answer is a host-and-port evidence index. This tool organizes requests logically, labels each upstream role, and distinguishes browser observations from actual route proof.

Start with a Request Contract

Before jumping into the DevTools, plan out what requests you expect to see. Think of this plan as a contract. It helps you spot requests that should have occurred but didn't, and ensures that successful asset requests don't falsely suggest API routes are functioning. Here's a straightforward example:

const expectedRoles = {
  document: ["app.example.test:443"],
  identity: ["auth.example.test:443"],
  api: ["api.example.test:443", "api.example.test:8443"],
  assets: ["static.example.test:443"]
};

Always use placeholders or internal values in shared examples to keep sensitive information out of tickets.

Capture the Complete Journey Once

Accuracy is key. Open DevTools before you navigate, enable 'Preserve log', clear out any old entries, and then undertake a single, authorized, non-destructive journey. Avoid refreshing multiple times to prevent cache changes or destination throttling, which can skew the event sequence.

Make sure these fields are visible: request sequence, hostname, port, method, status class, initiator, protocol, and timing specifics like connection, TLS, TTFB, and total time. Remember, the request number is the browser's sequence key, not a universal event ID. Resource loading in parallel can change this numbering with each run.

Normalize Without Retaining Secrets

When you normalize data, exclude sensitive information such as query strings, cookies, and authorization headers. Here's how you can shape a sanitized row with JavaScript:

function evidenceRow(flow, caseId) {
  const url = new URL(flow.url);
  return {
    case_id: caseId,
    request_number: flow.sequence,
    hostname: url.hostname,
    port: url.port || (url.protocol === "https:" ? "443" : "80"),
    method: flow.method,
    status_class: flow.status ? `${Math.floor(flow.status / 100)}xx` : "none",
    initiator_class: classifyInitiator(flow.initiator),
    failure_layer: "unclassified",
    route_verified: false,
    ttfb_ms: flow.timing?.ttfb ?? null,
    total_ms: flow.timing?.total ?? null
  };
}

By sticking to the essentials, you avoid exposing sensitive data and keep the focus on comparison elements.

Use Two Axes: Site Relation and Upstream Identity

Grasping the browser's security context and identifying the actual host and port to examine can be simplified using site relation and upstream identity. Here's a format to use:

{
  site_relation: "same-site", // or cross-site
  upstream_key: "api.example.test:8443"
}

This separation clarifies the difference between perceived user experience and the technical behavior.

Classify the First Divergence

To find the root issue, compare a failing trace with a successful one by matching requests by role, method, hostname, port, and route template. Here’s a function to identify the first significant difference:

function firstDivergence(control, failure) {
  const byKey = rows => new Map(rows.map(row => [
    [row.role, row.method, row.hostname, row.port, row.route_template].join("|"),
    row
  ]));
  const baseline = byKey(control);
  return failure.find(row => {
    const peer = baseline.get([
      row.role,
      row.method,
      row.hostname,
      row.port,
      row.route_template
    ].join("|"));
    return !peer ||
      peer.status_class !== row.status_class ||
      peer.failure_layer !== row.failure_layer;
  });
}

In practice, this strategy aids in pinpointing issues, be it an authentication hiccup, a failed preflight, or a request the browser never sent.

Explicit Failure Layers

Label failure types carefully to avoid mixing up scenarios like timeouts with proxy failures. Here's a list for reference:

const failureLayers = [
  "browser",
  "service_worker",
  "dns",
  "proxy_gateway",
  "proxy_exit",
  "tls",
  "destination",
  "application",
  "policy",
  "unclassified"
];

Link browser trace data with a sanitized session ID and conduct an authorized route or exit check to keep things clear.

Port Differences Require Their Own Comparison

Different ports on the same hostname can yield varying results due to listeners, certificates, or proxy policies. Compare standard and alternate-port rows explicitly:

function groupByUpstream(rows) {
  return Object.groupBy(rows, row => `${row.hostname}:${row.port}`);
}

This methodical approach ensures a full understanding of network behavior.

Export the Minimum Useful Artifact

When handing off, ensure you include:

• The evidence index
• A screenshot showing sequence and host context
• A brief reproduction contract
• A sanitized correlation identifier
• The first-divergence conclusion with any noted uncertainties

Offer a full HAR file only if really needed, and only after you've made sure no sensitive data is included.

A Practical Acceptance Checklist

Check off these steps to ensure your debugging process is sound:

• Make sure the test scope and action are authorized.
• Prevent direct fallback behaviors.
• Record browser and DevTools versions.
• List expected hosts and ports before capturing data.
• Enable 'Preserve log' before navigation.
• Review the full site context before narrowing it down.
• Use a stable request-role key for rows.
• Mark the first significant divergence.
• Separate browser observation and route proof fields.
• Avoid retaining credentials, tokens, cookies, personal data, or unnecessary URLs.
• Use a safe, idempotent action for reproduction.
• Be aware of rate limits and destination restrictions to stop the test if needed.

The aim isn’t to collect more data but to produce a succinct artifact allowing engineers to reach conclusions independently. If you're into proxy testing, you might want to check out more guides from 98IP.

You could also find my talk on Building High-Performance Web Apps informative, especially if debugging ties into performance issues. Delving into security implications could be crucial for maintaining robust systems too.