Web Driving

Drive web UIs repeatably using OpenClaw’s browser tool (Playwright-style), with a bias toward robustness over brittle selector hacks.

Workflow

0) Choose the right browser profile

• If the user mentions Chrome extension / Browser Relay / attach tab: use browser(profile="chrome").
• Otherwise default to browser(profile="openclaw").

If automation keeps failing (consent walls, CAPTCHAs, logged-out state), switch to Chrome relay.

1) Establish the target + constraints

Write down (in your scratchpad / message) the concrete parameters before clicking anything:

• Origin(s), destination(s)
• Date flexibility (fixed dates vs window)
• Hard constraints (e.g., nonstop only)
• Allowed alternates (e.g., FLL primary, MIA ok, PBI no)

2) Navigate + stabilize

• browser.open or browser.navigate
• browser.snapshot(refs="aria") early and often.
• Prefer aria refs over CSS/XPath.
• Avoid long wait loops; use short wait only when UI is clearly transitioning.

3) Apply filters in a robust way

General pattern for flaky dialogs:

1. Click the button that opens the dialog.
2. If click-by-ref is flaky, fall back to evaluate:
   • Find controls by text (innerText) or accessible labels (aria-label).
   • Click the underlying <input type=radio> when present (more reliable than clicking styled wrappers).
3. Close the dialog (Escape or close button).
4. Confirm the filter “stuck” by reading the visible label/state.

4) Extract results

Use a two-pass approach:

• Pass A (structured): parse the DOM for prices/times/airlines if accessible.
• Pass B (visual): if the UI renders prices in a canvas/virtualized grid, take screenshots and read them (OCR or manual inspection) and report the best candidates.

5) Report back (minimal + actionable)

Return:

• Top 3–5 options: date pair, price, airline, nonstop confirmation, any baggage gotchas.
• A shareable URL if the site supports it.

Do not spam the user with intermediate screenshots; only send them if the user asks or if ambiguity blocks progress.

Site playbook: Google Flights

Read: references/google-flights.md

Key tactics:

• There is no stable free API; treat this as web driving.
• Expect dynamic DOM; use aria refs + evaluate fallbacks.
• For Stops filter, the reliable approach is often: find the underlying input[type=radio] whose label text is “Nonstop only” and click it.
• Use Date grid / Price graph to find cheaper date pairs; grid may require screenshots.

Resources

references/

• references/google-flights.md — robust steps for SEA↔FLL/MIA searches, nonstop filter, date grid scanning, and screenshot-based extraction.