Shared browser interaction, login popups, and session persistence

The VPS-hosted shared browser requires a manual Take Control / Release Control handoff, and typing, selection, copy/paste, and shortcuts feel rigid. Login buttons can create hidden popup tabs while the visible page stays unchanged. Fix the interaction itself and verify login persistence with disposable fixtures.

Related investigation: #1. This issue tracks the implementation requested on 2026-10-28, separately from the earlier browser executable and viewport work.

Reported behavior and reproduction evidence

• In the Default profile, Facebook Ads Manager redirected to Meta Business Suite with a Continue with Facebook button.
• Repeated clicks created four Facebook login popup tabs, each with an openerTabId pointing to the original page and visible: false. The clicks were delivered; the popup reveal/focus path failed.
• Explicit preview_open({ tabId: <popup>, open: true }) still reported the popup as invisible. Navigating the original tab to the observed login URL displayed the actual login form.
• The user subsequently reported Facebook Ads Manager and GoHighLevel signed in. Those accounts must remain undisturbed.
• The exact causes of every typing/copy/paste symptom have not been established. Treat ownership, focus, input delivery, clipboard behavior, and popup visibility as separate paths.

Required result

• Ordinary clicks, scrolling, typing, and pasting work without a required handoff button.
• Active human input takes priority automatically; queued agent mutations cannot overwrite it, and automation can resume after interaction pauses.
• Focus, keyboard input, text selection, copy/cut/paste, shortcuts, IME input, and input ordering are verified on the remote browser UI. Review the desktop paths where applicable.
• A human-opened login popup becomes visible and focused, preserving the actual popup and its opener relationship.
• Persistent-profile fixture login survives refresh, another tab, closing/reopening tabs, and browser-context/host restart where the website's session permits it.
• Profile/host isolation, incognito, isolated agent tabs, session expiry, session-only cookies, and explicit storage clearing are explained.
• Targeted tests and vp run check:changed pass; the branch is pushed and a PR is linked here.
• Installation/restart requirements are reported without deploying or restarting the user's active service.

Confirmed code findings and unresolved symptoms

• apps/server/src/preview/SessionControl.ts, ServerBrowser.ts, and the shared stream client all enforce manual ownership. Removing only the UI buttons would leave input blocked.
• ServerBrowser.adoptPopup previously suppressed reveal and popup notification whenever the opener had an automation owner, including a popup opened by a person interacting with that tab.
• ServerBrowserSurface disabled its keyboard textarea whenever the controller was not you, and cleared focus on ownership changes. Editing shortcuts could also perform their default action against the local hidden textarea or propagate outside the streamed page.
• ServerBrowserContexts uses launchPersistentContext for regular profiles and ephemeral contexts for incognito/isolation keys. Desktop profiles use persistent Electron partitions. The later disposable Chromium fixture passed persistence and isolation checks; this does not establish that every site's session survives a restart.
• Blur/input ordering: releasing ownership before accepted human input drained could discard queued typing. A second regression covered a pending navigation blocking the drain while a person tries to correct the URL. Both tests initially failed because finishInteraction did not exist, then passed in the later combined run. The local fix drains accepted input before release and keeps navigation waits outside the input queue.
• Rerender/focus loss: clearInput had a new identity on every render and was a dependency of the stream effect. A status rerender could reconnect the stream, clear input, and blur the keyboard target. The local code stabilizes the callback with useCallback. The last completed web test still failed its first-click focus assertion; the rerun after this fix was interrupted, so resolution is unverified.
• Explicit reveal: opening an existing tab without navigating previously went through the mutation/control path. The local patch permits revealing an authorized existing tab while observing active human input. End-to-end visibility remains unverified.
• Reported, incompletely verified: text selection, copy/cut/paste, shortcuts, IME input, mobile keyboard/paste and popup selection, dialog/file-picker release, and multi-viewer interaction still need integrated verification. No claim that all of these symptoms share one cause.
• Stuck-chat report (2026-10-28): the user reported this implementation chat stayed Working and did not finish after “log all the bugs and finish this chat” and “are you done?”, then stopped it and requested continuation in a new chat. Recovered command records show interrupted validation commands, including a browser-test rerun and check:changed; they do not establish the cause of the stuck conversation. This is a separate orchestration/UI symptom, not a proven browser-input defect.

Implementation checkpoint

Working branch: fix/shared-browser-interaction in the VPS checkout. Starting HEAD: 9169bd3e50 (feat(server): allow a configured Chromium executable), inherited from the existing checkout. The executable-override commit predates this task.

The following changes are currently local and uncommitted; they have not been deployed or published in a PR:

• Added automatic interaction reservation and inactivity release to SessionControl, with tests for agent drain/cancellation, renewal, held input, and cleanup.
• Updated streamed server input to reserve automatically, keep read-only access restricted, release held input on blur/disconnect, and allow observation while a person interacts.
• Changed human-opened popup adoption to reveal and notify the interacting viewer even when the opener belongs to an agent. Added a popup regression fixture.
• Removed the required handoff buttons in web and mobile clients. A shared previewStreamCanInteract predicate keeps page input and browser controls available when no other viewer is actively interacting.
• Kept the streamed keyboard focused across automatic release, prevented editing-shortcut defaults against the hidden textarea, and reused the application's clipboard fallback for HTTP/clipboard-API-limited clients.
• Added mobile paste handling and a popup message path to select the new tab.
• Updated browser-control errors, floating-preview focus detection, and the remote-browser user guide.
• Added a real Chromium/local HTTP login fixture using a temporary profile directory. It covers refresh, new tabs, all tabs closed/reopened, browser-pool restart, localStorage continuity, another profile, isolated agent context, and incognito recreation.
• Added browser component regressions for first-click interaction, focus after automatic release, editing shortcuts, paste delivery, and blur.
• Added finishInteraction to preserve queued input on blur and let new input correct a pending navigation while release waits for it.
• Stabilized the web stream's clearInput callback to avoid reconnecting on status rerenders. The subsequent web regression rerun did not complete.

Touched source areas: apps/server/src/preview, apps/web/src/browser, apps/web/src/components/preview, apps/mobile/src/features/browser, packages/client-runtime/src/preview, packages/contracts/src/preview*, and docs/user/remote-access.md.

Verification log

Recovered from the original chat's completed command records on 2026-10-28. No new tests or browser sessions were started during the logging-only takeover.

• Initial checkout was clean; AGENTS.md was inspected. vp i completed because no checkout setup marker existed.
• Automatic-reservation and shared-client regressions failed before their implementations and passed afterward. An early attach-viewer test expectation was updated to require actual human input before ownership is asserted.
• At 20:47 UTC, the larger targeted run had 58 passing tests across five files (ServerBrowser, SessionControl, shared stream client, desktop GuestInput, and desktop BrowserSession) and one persistence-fixture launch failure because Chromium revision 1223 was missing.
• Browser installation initially failed with Playwright does not support chromium on ubuntu26.04-x64. The test dependency was retried with PLAYWRIGHT_HOST_PLATFORM_OVERRIDE=ubuntu24.04-x64. The later browser tests executed, so the missing executable was no longer blocking that run.
• At 21:08 UTC, the two new blur/navigation tests failed with control.finishInteraction is not a function, establishing their pre-fix failure. Focused lint commands completed with exit code 0 at 20:54 and 21:08 UTC; later edits still need final validation.
• At 21:09 UTC, the last completed combined run had 51 passes and one failure across five files: SessionControl, ServerBrowserContexts.integration, ServerBrowser, shared stream client, and the web browser component suite. All four non-web files passed. The editing-shortcut/paste web test passed; first-click/focus failed because the active element was the body rather than the hidden keyboard textarea.
• The passing real Chromium fixture used a local fake login service and a temporary profile. It verified refresh, another tab, closing all tabs and reopening, browser-pool restart, localStorage continuity, another profile, an isolated agent context, and incognito recreation. It did not exercise private accounts or a VPS reboot.
• The focus callback fix was then applied, but its browser-test rerun produced no completed result and ended with exit code 143 when the stopped run was cleaned up. Do not report that regression as passing.
• vp run check:changed reported formatting, GraphQL generation, and lint pass before interruption (exit code 143). There is no completed passing result for the entire check, including typecheck, knip, and tests. Formatting was rerun later but also interrupted.

Handoff state and remaining work

The user's latest instruction in the original chat was “log all the bugs and finish this chat.” Implementation is stopped. This issue remains open because the fixes are unfinished; pushing or opening a PR was deferred at that handoff.

At takeover, HEAD is still 9169bd3e50 on fix/shared-browser-interaction. The working tree contains 16 modified tracked files and two untracked test files: apps/server/src/preview/ServerBrowserContexts.integration.test.ts and apps/web/src/browser/ServerBrowserSurface.browser.test.tsx. Those files are preserved in place. No implementation commit or PR was created by this task.

When implementation is resumed:

1. Rerun and resolve the first-click/focus browser regression after the stable-callback change. Finish final formatting, targeted checks, and vp run check:changed.
2. Review automatic reservation, accepted-input ordering, held key/button cleanup, navigation drains, dialog/file-picker completion, passive viewers/resizes, and multi-viewer priority.
3. Verify actual popup reveal and selection in the panel, floating preview, and mobile; review desktop-specific paths. Exercise typing, selection, clipboard, shortcuts, and IME against disposable fixtures.
4. Preserve profile directories and Electron partition identities. Keep account/session limitations clear: another host/profile, incognito/isolated contexts, session-only cookies, expiry, and explicit clearing can change login continuity.
5. If the implementation/PR request is resumed, review the final diff, capture UI evidence with authorization, then commit/push and open the PR. Record its URL, head SHA, and completed checks here.
6. Investigate the stuck-conversation symptom separately before attributing it to a provider, tool, server, or client fault. Follow-up read-only inspection found the first final-turn interrupt effect available at 22:01:32 UTC and completed at 22:13:40 UTC (about 12 minutes). The provider thread is now idle; all seven runs/attempts are interrupted or cancelled, no message is streaming, and no outbox effect remains pending. Two historical todo-list nodes still have running status, but their runs are terminal; their role in any stale client indicator is unproven. The user still reported the chat would not turn off after handoff. No service restart, direct database modification, or process kill was used.

Operational constraints

Do not log out, clear profiles, read or export private credentials/cookies, or reproduce against the active authenticated accounts. Do not start a test server against the live T3 home, restart the installed service, or deploy the changes without separate authorization. Preserve existing persistent profile paths and Electron partition identities.

No live service restart, deployment, logout, credential extraction, profile clearing, or infrastructure change was performed during this implementation. Tests use mocks or disposable fixture state. Updating source alone does not update the installed application; eventual rollout needs a rebuilt/updated server and matching client assets, plus an intentionally scheduled restart if required.

Prepared with GPT-6.1 Sol in the Codex harness, running through T3 Code. Handoff reconciled on 2026-10-28 from the original conversation, completed command records, and the current working tree.