Intercepting Network Traffic in Browser Automation: Playwright route, CDP Fetch, and the Proxy Layer

Introduction

Driving a page is where browser automation starts; controlling its data flow is where you actually take charge.

Requirements like these cannot be solved by element locators alone:

• The backend API isn't finished yet, but the frontend needs integration testing now — you need controlled data to keep the tests moving;
• You want to verify how the page behaves when an API returns 500 / times out / delivers dirty data — scenarios that are nearly impossible to produce with real data;
• The page drags in images, fonts, and analytics scripts that slow your tests down, and you want to cut them off entirely;
• A request carries a device-fingerprint parameter, and you want to see how the page reacts when you change its value.

The common solution to all of these is network interception: catch a request before it goes out or a response before it arrives, then pass, tamper with, or fake it as you see fit.

There are three layers where you can do this, and they are the three threads of this article:

Test code
   │
   ├── ① Playwright route API   —— built into the framework, page/context scope, easiest
   │
   ├── ② CDP Fetch domain       —— the browser protocol layer, hands-on, finest control
   │
   └── ③ Proxy layer (mitmproxy) —— the OS network layer, covers traffic beyond the browser

Prerequisite reading: the two posts on this site, Introduction to Chrome Automation Protocol CDP (CDP protocol basics) and Playwright Installation Acceleration Guide for China (environment setup). This article does not repeat them and goes straight to interception.

---

1. Three Layers at a Glance

The essential difference between the three layers is where you plug in:

Layer	Insertion point	One-line summary
Playwright route()	Inside the browser's network stack (page/context scope)	The interception API your test framework hands you — declarative, fastest to start
CDP Fetch domain	Inside the browser's network stack (protocol session scope)	Skip the framework wrapper and talk to the browser directly — the finest control
Proxy layer	The OS network layer	Catches traffic the browser cannot govern — desktop apps, system-level requests

A rule of thumb: start with route, reach for CDP when it's not enough, go to the proxy only when traffic leaves the browser. Section 5 has the full comparison.

---

2. Playwright route(): The Built-in Layer

2.1 The Three Verbs: abort, continue, fulfill

Every matched request hands you a Route object, and you have exactly three dispositions, each mapping to a testing intent:

import json
import re
from playwright.sync_api import sync_playwright

MOCK_ITEMS = {"code": 0, "data": {"items": [{"id": 1, "name": "mocked-by-route"}]}}

def mock_items(route):
    if route.request.method != "GET":
        route.continue_()          # not a GET: pass it through untouched
        return
    route.fulfill(                 # GET: serve a forged response — the request never leaves the machine
        status=200,
        content_type="application/json",
        headers={"x-from": "mock"},
        body=json.dumps(MOCK_ITEMS),
    )

with sync_playwright() as p:
    browser = p.chromium.launch()
    context = browser.new_context(service_workers="block")   # see 2.5
    page = context.new_page()

    page.route(re.compile(r"/api/items(\?.*)?$"), mock_items)        # mock the items API
    page.route("**/*.{png,jpg,woff2}", lambda r: r.abort())          # strip static assets to speed up loads

    page.goto("https://mall.example.com/items")
    browser.close()

• route.abort(): kill the request. Use it to strip images and fonts, simulate being offline, or simulate a failed request;
• route.continue_(): pass the request through — and modify it on the way: swap headers, rewrite the POST body, redirect the URL, e.g. route.continue_(headers={**route.request.headers, "x-test": "1"});
• route.fulfill(): forge a response. Status, headers, and body are all yours; what the page receives is indistinguishable from a real server reply.

2.2 Matching: Globs, Regexes, Predicates

The first argument of page.route takes three forms, in increasing order of precision:

page.route("**/api/items", handler)                          # glob: /api/items on any host
page.route(re.compile(r"/api/items\?page=\d+$"), handler)    # regex: with a page parameter
page.route(lambda url: url.endswith("/api/items"), handler)  # predicate: arbitrary logic

The full glob semantics (** across path segments, ? modifiers, and friends) are documented officially; practical advice: prefer a regex over an intricate glob — it reads better and ages better.

2.3 Scope and Handler Chaining

Two things people trip on:

Scope: page.route governs only the current page; context.route governs every page in the context, which suits "global" rules (say, injecting a test-marker header everywhere). Prefer context level for global rules and page level for case-specific ones — cleaner isolation, and parallel tests stay out of each other's way.

Registration order: the rule registered last matches first. If a handler finishes without calling fallback(), the chain stops; if it does call fallback(), the request moves on to the next (earlier-registered) handler. This enables a "bypass observer + main handler" split:

# global bypass: log only, then hand the request down the chain
context.route("**/api/**", lambda r: (print("→", r.request.url), r.fallback()))
# main handler: do the actual mocking
page.route("**/api/items*", mock_items)

2.4 A Bonus: HAR Replay

If writing mock data by hand sounds tedious, Playwright can record real traffic into a HAR file and replay it:

# recording: pass record_har_path to new_context
context = browser.new_context(record_har_path="demo.har")

# replay: every request matching **/api/** is answered from the recording
page.route_from_har("demo.har", url="**/api/**")

Ideal for stability suites: record once against the real environment, replay a hundred times offline.

2.5 Two Boundaries of route

• Service Workers: requests issued by a page's service worker do not pass through page.route. The sturdiest fix is to disable them outright: new_context(service_workers="block") (already in the example above). Testing the SW itself is a different exercise;
• WebSocket: route never sees frames after the WS handshake. To observe WS traffic, subscribe to page.on("websocket"); frame-level data currently requires the CDP Network.webSocketFrame* events or a proxy.

---

3. The CDP Fetch Domain: Hands-on at the Protocol Layer

However convenient route is, there are cases where it doesn't get the call: the project runs Selenium or raw CDP; you need control by request stage or resource type; or you simply want to know what the framework wrapper is doing underneath. That's when you drive the CDP Fetch domain yourself.

3.1 Enabling Interception: Fetch.enable and requestStage

The setup matches the site's CDP post: start Chrome with --remote-debugging-port=9222, read /json/list for the target tab's WebSocket URL, and connect to it directly. Then:

Fetch.enable({
    "patterns": [{"urlPattern": "*", "requestStage": "Response"}]
})

requestStage is the essence of the Fetch domain: pausing at the Request stage means the request hasn't gone out yet (rewrite headers, body); pausing at the Response stage means the response is back but hasn't reached the page yet (rewrite the body, the status code). You can declare patterns for both stages at once, each governing its own segment.

3.2 Rewriting a Response, End to End

The core mechanism is an event loop: every matched request raises one Fetch.requestPaused event, and stays suspended until you answer it. There are exactly three answers: continueRequest (pass through), fulfillRequest (forge), failRequest (kill).

Below is a complete, runnable response rewriter that intercepts a JSON API and swaps its payload (dependencies: websocket-client, requests):

import base64
import json

import requests
import websocket


class CdpInterceptor:
    """A response interceptor built on the raw CDP Fetch domain (single-tab edition)"""

    def __init__(self, host='127.0.0.1', port=9222):
        self.host, self.port = host, port
        self.ws = None
        self.msg_id = 0

    def connect(self):
        targets = requests.get(f'http://{self.host}:{self.port}/json/list').json()
        page = next(t for t in targets if t['type'] == 'page')
        self.ws = websocket.create_connection(page['webSocketDebuggerUrl'])
        self._cmd('Fetch.enable', {
            'patterns': [{'urlPattern': '*', 'requestStage': 'Response'}]
        })

    def _send(self, method, params=None):
        self.msg_id += 1
        self.ws.send(json.dumps({'id': self.msg_id, 'method': method,
                                 'params': params or {}}))

    def _cmd(self, method, params=None):
        """Send a command and block for its reply; events arriving meanwhile are dropped (kept simple)"""
        mid = self.msg_id + 1
        self._send(method, params)
        while True:
            msg = json.loads(self.ws.recv())
            if msg.get('id') == mid:
                return msg

    def serve_forever(self):
        while True:
            msg = json.loads(self.ws.recv())
            if msg.get('method') == 'Fetch.requestPaused':
                params = msg['params']
                try:
                    self._on_paused(params)
                except Exception:
                    # whatever the rewrite did, the request MUST be released,
                    # or the page hangs forever (see pitfall 1 in section 6)
                    self._send('Fetch.continueRequest',
                               {'requestId': params['requestId']})

    def _on_paused(self, params):
        # only rewrite JSON APIs; everything else passes through
        headers = {h['name'].lower(): h['value'] for h in params['responseHeaders']}
        if 'application/json' not in headers.get('content-type', ''):
            self._send('Fetch.continueRequest', {'requestId': params['requestId']})
            return

        # 1. fetch the original body (returned base64-encoded)
        resp = self._cmd('Fetch.getResponseBody', {'requestId': params['requestId']})
        body = resp['result']['body']
        if resp['result']['base64Encoded']:
            body = base64.b64decode(body).decode('utf-8')

        # 2. rewrite: replace the items payload with controlled data
        data = json.loads(body)
        data['data']['items'] = [{'id': 1, 'name': 'mocked-by-cdp'}]

        # 3. carry the original headers back (minus length/encoding, plus no-cache)
        #    and fill in the rewritten body
        out_headers = [{'name': h['name'], 'value': h['value']}
                       for h in params['responseHeaders']
                       if h['name'].lower() not in ('content-length', 'content-encoding')]
        out_headers.append({'name': 'Cache-Control', 'value': 'no-store'})

        self._cmd('Fetch.fulfillRequest', {
            'requestId': params['requestId'],
            'responseCode': params['responseStatusCode'],
            'body': base64.b64encode(json.dumps(data).encode()).decode(),
            'responseHeaders': out_headers,
        })


if __name__ == '__main__':
    interceptor = CdpInterceptor()
    interceptor.connect()
    interceptor.serve_forever()

Three implementation details worth your attention:

1. Response headers must be carried back yourself. fulfillRequest does not inherit the original headers: lose Content-Type and the page fails to parse the body; lose Set-Cookie and the session dies (pitfall 3). Meanwhile Content-Length/Content-Encoding must be removed — you have replaced the body;
2. getResponseBody always returns the body base64-encoded. Decode according to the base64Encoded flag; the body you pass back to fulfillRequest must be base64-encoded as well;
3. Release the request in the error path. After the try/except in serve_forever, continueRequest runs regardless, so any rewrite failure degrades to "no interception" instead of "a frozen page".

3.3 Request-stage Rewrites and a Protocol Gotcha

Switch the pattern's requestStage to "Request" and you act before the request leaves; continueRequest accepts a new method, URL, postData, and headers:

self._cmd('Fetch.continueRequest', {
    'requestId': request_id,
    'headers': [{'name': 'x-device-id', 'value': base64.b64encode(b'fake-device').decode()}],
})

Note this counter-intuitive protocol detail: the value field of Fetch.HeaderEntry must be base64-encoded (it is a binary type in the protocol). Send plaintext and the browser throws a protocol error — the most common first trip-up in CDP interception.

There is also a Fetch.authRequired event paired with continueWithAuth, handy for auto-filling proxy credentials — a clean way to inject proxy auth in automated runs.

3.4 What About Multiple Tabs

For brevity, the example above connects straight to the first page-type target. In real projects you attach every tab to one WebSocket via Target.attachToTarget (with flatten: true) and dispatch messages by their sessionId field — each tab is its own session, and the Fetch domain must be enabled per session. That session plumbing is the main engineering step between this demo and production; the interception logic itself does not change.

---

4. The Proxy Layer: Traffic the Browser Can't Govern

The first two layers operate inside the browser. Some traffic is out of their reach: the client is a desktop app; the traffic never passes through a browser at all (a local service calling another local service); or you want to rewrite packets without touching any code. That is when a proxy earns its keep — mitmproxy, for example.

4.1 A mitmproxy Addon

mitmproxy's addon model revolves around two hooks: request (a request arrives) and response (a response is on its way back). The same "replace the items payload" as in 3.2:

# mitm_addon.py
import json
from mitmproxy import http

API_HOST = "mall.example.com"

def response(flow: http.HTTPFlow) -> None:
    if flow.request.host != API_HOST or not flow.request.path.startswith("/api/items"):
        return
    data = json.loads(flow.response.get_text())
    data["data"]["items"] = [{"id": 1, "name": "mocked-by-proxy"}]
    flow.response.set_text(json.dumps(data))

Start the proxy and point the browser (or the whole system) at it:

mitmdump -s mitm_addon.py -p 8080

chrome --proxy-server=http://127.0.0.1:8080 --ignore-certificate-errors

--ignore-certificate-errors is for local test environments only; the proper setup installs mitmproxy's CA certificate into the system/browser trust store, otherwise every HTTPS request comes with a red warning.

4.2 When the Proxy Is Actually Worth It

Of the three layers the proxy has the highest setup cost (certificates, port management, performance), and the only "indiscriminate" coverage:

• The target is a desktop application (say, a PyWebView-packaged client — in the architecture from the site's earlier post, requests from the embedded WebView are just as interceptable);
• The traffic never passes through a browser instance you can inject into;
• You need global capture-and-log across all processes — for security audits or API inventory.

If your target is a page inside a browser, route and Fetch will always be the lighter choice.

---

5. Comparison and Selection

Dimension	Playwright route	CDP Fetch domain	Proxy (mitmproxy)
Insertion point	In-browser (page/context)	In-browser (protocol session)	OS network layer
Coverage	Requests of the current page/context	Requests of the session's tab (multi-tab = multi-session)	All traffic of the process/machine, beyond the browser
Match granularity	glob / regex / predicate	urlPattern + requestStage + resourceType	Arbitrary logic in a script
Rewrite requests	continue_ (headers/body/URL)	continueRequest	request hook
Rewrite responses	fulfill	getResponseBody + fulfillRequest	response hook
HTTPS handling	transparent	transparent	requires trusting the mitm CA
WebSocket frames	no (observation events only)	not via Fetch (observable via Network)	supported
Service Worker requests	bypassed by default (block them)	attach to the SW session separately	covered indiscriminately
Parallel-run friendliness	high (context-level isolation)	medium (session management on you)	low (a shared proxy is a bottleneck)
Learning curve	low	medium	medium-high

One line of advice: in Playwright suites, always route, and prefer HAR replay for mock data; on Selenium or raw CDP, or for request-stage precision, take the Fetch domain; when the target leaves the browser, take the proxy.

---

6. Pitfall Checklist

1. An unanswered requestPaused freezes the page. A paused request stays suspended; miss the answer on any branch (especially the error path) and the page spins forever. Put all handling inside try/finally whose fallback is continueRequest;
2. CDP header values must be base64. Fetch.HeaderEntry.value is a protocol-level binary; passing plaintext throws a protocol error;
3. fulfillRequest does not inherit the original response headers. Without Content-Type the page cannot parse the body; without Set-Cookie the session dies. Always carry the original headers and drop Content-Length/Content-Encoding;
4. Mock not taking effect? Check the cache first. The browser may be reusing a cached response. Disable caching with Network.setCacheDisabled on the CDP side, and add Cache-Control: no-store to fulfill responses;
5. Service Workers bypass interception. In Playwright, disable them with service_workers="block"; in CDP, attach to the SW session separately;
6. Nobody intercepts WebSocket frames. Both route and the Fetch domain stop at the handshake; for frame-level needs, observe with Network.webSocketFrame* events or use a proxy (mitmproxy supports WS);
7. Don't share one proxy across parallel tests. Squeezing every case through a single mitmproxy instance is slow and a single point of failure; in parallel runs prefer route (and give each Fetch-domain instance its own session).

---

7. Conclusion

The point of network interception is turning "the test depends on a real backend" into "the test depends on controlled data". The three layers are not ranked — they have different radii:

• route is the default answer for Playwright users: declarative, certificate-free, context-isolated;
• the Fetch domain is the protocol layer's scalpel: the finest control, at the price of managing sessions and protocol details yourself;
• the proxy layer is the wide-area fallback: the only answer for traffic beyond the browser, at the price of certificates and performance.

A natural follow-up: a generic fault-injection harness built on these mechanisms (500s, timeouts, dirty payloads), and recording/replaying raw traffic with the Network domain.

References

1. Playwright Python docs – page.route, Route class, route_from_har, Network mocking guide
2. Chrome DevTools Protocol – Fetch Domain
3. mitmproxy – Addons
4. Prerequisite reading on this site: Introduction to Chrome Automation Protocol CDP, Playwright Installation Acceleration Guide for China