[depfu]

Here is everything you need to know about this update. Please take a good look at what changed and the test results before merging this pull request.

What changed?

✳️ ferrum (0.18.0 → 0.19.1) · Repo · Changelog

Release Notes

0.19.1

Changed

• --disable-web-security is no longer among the default Chrome flags, so pages keep the same-origin policy and an
  untrusted page can't read local files with <iframe src="file:///etc/passwd">. Frame#frame_element returns nil
  for a cross-origin frame now. Pass browser_options: { "disable-web-security" => nil } for the old behavior [#472]

Fixed

• Mouse#move with a button held didn't send which button, so Chrome never started a drag. Native HTML5 drag and drop
  now works with down, move and up, and Input.dragIntercepted fires when drags are intercepted [#635]
• A paused request with a binary body, e.g. a Blob upload with blacklist=/whitelist= set, hung forever: Chrome
  sends the body with lone UTF-16 surrogates that the json gem rejects, and the fallback that unescaped them broke
  the message further, so Fetch.requestPaused was dropped and the request was never continued. Lone surrogates are
  now replaced with U+FFFD, so broken text from the page reads as � instead of ??? [rubycdp/cuprite#308]
• Passing a node resolved on the previous page to evaluate, e.g. when the page navigates while Cuprite searches
  within an element, raised Ferrum::BrowserError: Argument should belong to the same JavaScript world as target object. It's Ferrum::NodeNotFoundError now, so callers can treat the node as stale and find it again
  [rubycdp/cuprite#233]

0.19.0

Added

• Ferrum::Frame#wait_for_selector (also reachable from Page/Browser), which waits for an element to show up
  in the DOM. It retries at_css/at_xpath every interval until a match is found or time runs out, returning nil
  on timeout [#82]
• Ferrum::Page#on(:response)/Ferrum::Worker#on(:response), a callback fired once a request's response has fully
  loaded (on CDP's Network.loadingFinished, not Network.responseReceived, so exchange.response.body is always
  available), yielding the request's Network::Exchange. Doesn't require network.intercept to be set up, and is
  never fired for requests that fail to load [#294]
• Ferrum::Browser#quit/Ferrum::Browser::Process#stop accept wait: false to return immediately and run process
  killing and user-data-directory cleanup on a background thread instead of blocking; the call returns the Thread
  so callers can #join it if they need cleanup to have finished, e.g. before process exit or before reusing a
  fixed port. Default (wait: true) keeps the previous synchronous behavior; #restart always waits.
• Ferrum::Page::Stream#close_stream, a public method that releases a CDP IO stream handle (IO.close).
• Ferrum::Page#set_viewport's mobile: option now also enables touch emulation
  (Emulation.setTouchEmulationEnabled), alongside the existing device metrics override, so it fully emulates a
  mobile device's viewport and touch support together. [#94]

Changed

• Ferrum::TimeoutError names the CDP command and session it was waiting on, and exposes them as #command and
  #session_id. Waits that aren't for a command keep the old message [#470]
• Ferrum::Page::Stream#stream now closes the CDP stream handle (IO.close) once it's been fully read, so streams
  opened for Ferrum::Browser#pdf and Ferrum::Page::Tracing#record no longer keep their backing storage alive in
  the browser; a failed IO.close is raised to the caller.

Fixed

• An out-of-process iframe, e.g. Chrome's PDF viewer for an embedded PDF, appearing while a web worker was connected
  raised 'Page.enable' wasn't found in the event thread. Only pages and iframes are now searched for its parent [#640]
• A browser that stopped reading its websocket froze the whole client: a large command blocked in a socket write
  forever while holding the send lock. The write now gives up after the command's timeout and raises
  Ferrum::DeadBrowserError [#644]
• A browser context whose Target.disposeBrowserContext timed out stayed registered, so every later #reset asked
  again and raised Disposal of browser context ... is already pending for the rest of the process. It's now
  forgotten either way, and #reset disposes the remaining contexts before raising the first error [#641]
• An error in the client or websocket thread was re-raised in the main thread wherever it happened to be, or took
  the process down with it, since Utils::Thread.spawn defaulted to abort_on_exception: true and neither thread
  rescued. Ferrum's threads no longer abort, they report to stderr and mark the connection dead [#470]
• Commands in flight when the browser died waited out :protocol_timeout each before raising, so a dead browser
  turned into a long parade of slow failures. They're released as soon as the socket closes, and a command sent
  after that raises Ferrum::DeadBrowserError right away [#470]
• A raising callback killed Client::Subscriber's dispatch thread and with it every event for the rest of the
  process: pages created later raised NoSuchTargetError, open ones lost their execution contexts. Callbacks are
  now rescued one by one and reported to stderr [#470]
• create_page raised Failed to find browser context with id against a browser that came up with its own startup
  window (browserless). That window lives in the browser's implicit context, which Chrome below 145
  won't address by id. Ferrum now detects it and exposes as Ferrum::Context#implicit?. It's never
  disposed, so #reset won't clear such a browser. [#578] [#627]
• Browser.setDownloadBehavior raised Failed to find browser context for id for a page in the browser's implicit
  context, so connecting to a browser that came up with its own startup window and passing :save_path died on the
  first page. Ferrum::Target#context_id, and with it Ferrum::Page#context_id, is now nil for that context,
  which is how Chrome wants it addressed, by omitting browserContextId [#546]
• Connecting to a remote browser with :url drops query string when probing /json/version, endpoint might answer
  401 and Ferrum died with undefined method 'host' for nil. in case of parse issues Ferrum::NoWebSocketUrlError is raised.
• Targets of a type Ferrum::Contexts doesn't track were left attached and paused by auto-attach for the lifetime
  of the browser; Chrome opens two browser_ui ones per browser context. They're now resumed and detached from
• Ferrum::Browser::Process registered two ObjectSpace finalizers on the same object, a directory remover and
  a process killer. Ruby runs them FIFO so directory could have been deleted before process dead, and browser could
  have restored its content on exit.
• Ferrum::Browser::Process::Killer.kill waited on the browser's leader pid with a blocking, unbounded
  Process.wait after escalating to KILL. That method is installed as an ObjectSpace finalizer, so it can run
  while the interpreter is shutting down, where a blocking wait risks hanging the process instead of letting it
  exit.
• --no-crashpad, added to the default Chrome flags in 0.18.0 to stop chrome_crashpad_handler from spawning, is
  not a Chromium switch at all, so Chrome silently ignored it and two handler processes kept starting per browser.
  Removed. There is no replacement: the only switch that does stop them, --disable-crashpad-for-testing, is meant
  for Chromium's own test harness and breaks a normally launched browser — child processes die at startup with
  Crashing due to FD ownership violation and the network service crash-loops, while the browser process itself
  survives and answers CDP, so every navigation returns net::ERR_ABORTED against an about:blank document.
  Reap the handlers with an init in the container instead (docker run --init, init: true, or tini); Chrome's
  crashpad behaviour is now documented under "The crashpad handler" in the customization docs [#610]
• #evaluate/#evaluate_on/etc. resolved an object/array result by making two CDP round trips
  (Runtime.callFunctionOn to check for a cyclic reference, then Runtime.getProperties) per nested
  object or array, recursively — even though the cyclic check already walks the entire reachable
  object graph from wherever it's invoked in a single round trip, making every check below the top one
  provably redundant. An expression returning an array of 1,000 small objects (each with one nested
  object, as in [#528]) therefore issued about 4,000 CDP round trips instead of 2,000, each one a
  chance to trip the per-command timeout while the browser was still working through the backlog;
  once the websocket happened to already be closed at that point, the timeout surfaced as
  Ferrum::DeadBrowserError instead of Ferrum::TimeoutError. The cyclic check now runs once, at the
  top level of the result, and that answer is reused for every nested object/array instead of
  rechecking each one.
• Ferrum::Contexts#connect_worker/#detach_unless_manually_attached only rescued Ferrum::BrowserError, not Ferrum::TimeoutError.
  A worker's attach or service-worker detach that timed out (e.g. on a loaded CI runner) would escape unrescued
  into Client::Subscriber's dispatch thread and kill it, silently breaking further Target.* event delivery
  for the rest of that browser's life, raising Ferrum::NoSuchTargetError
• Ferrum::Browser::Process killed the browser's leader pid alone with SIGUSR1, a signal Chromium has no shutdown
  handler for; it now sends SIGTERM, escalating to SIGKILL, to the whole process group, so renderer/GPU/zygote
  child processes are no longer orphaned after #quit. A leader process that exited promptly on TERM used to
  short-circuit escalation to KILL for the rest of its process group, so a child that ignored TERM
  (e.g. a stuck renderer) was left running forever; termination now keeps polling the group until it's actually empty
  or the timeout fires.
• Removing the user data directory after #quit silently gave up on any error, potentially leaking the temp
  directory forever with no indication; it now retries with exponential backoff on transient errors
  (Errno::ENOTEMPTY/EBUSY/EACCES/EPERM, since Chrome can briefly hold file locks right after being killed)
  and warns if it still can't be removed.
• :host option is respected again when Ferrum spawns Chrome itself. Chrome dropped support for
  --remote-debugging-address and always binds/logs 127.0.0.1, so Ferrum now substitutes the requested
  :host back into the address it uses to talk to Chrome (e.g. Ferrum::Browser.new(host: "ferrum.localhost")),
  instead of always hitting 127.0.0.1 [#552]
• Ferrum::Frame::Runtime#call no longer retries Ferrum::NodeNotFoundError/Ferrum::NoExecutionContextError when
  evaluating on an existing node (on:, e.g. #evaluate_on/#evaluate_func). Chrome reporting that a resolved
  node/object id is gone can't be fixed by retrying the same command, so the retry only added up to
  INTERMITTENT_ATTEMPTS * INTERMITTENT_SLEEP of dead time per stale node before raising anyway; this showed up as
  flaky/slow Capybara specs that queried collections of nodes shortly after the DOM replaced them, since Capybara's
  own retry loop (which re-finds nodes) already handles this case correctly one layer up [#360]

Removed

• The now no-op --remote-debugging-address Chrome flag, which Chrome has stopped honoring [#552]
• Experimental Firefox support (browser_name: :firefox). Ferrum is Chrome/CDP only now

Does any of this look wrong? Please let us know.

Commits

See the full diff on Github. The new version differs by 40 commits:

• chore(release): bump version to 0.19.1
• fix(browser): drop --disable-web-security from default Chrome flags (#649)
• fix(mouse): preserve held button during movement (#646)
• fix(client): replace lone UTF-16 surrogates instead of dropping the message (#648)
• fix(client): raise NodeNotFoundError for a node from another JavaScript world (#647)
• chore(release): bump version to 0.19.0
• docs: shrink and float logo on introduction page
• docs(readme): replace logo with new periodic table logo
• chore(deps): add irb to the development gems
• fix(client): give up on a websocket write the browser doesn't take (#644)
• fix(contexts): look up an iframe's parent among pages and iframes only (#640) (#643)
• fix(contexts): forget a context whose disposal fails (#642)
• fix(client): close the socket when the browser dies before the close handshake (#639)
• fix(browser): let quit clean up a browser whose contexts were never built (#634)
• chore: get rid of ff
• fix(client): keep Ferrum's threads from taking the host process down (#632)
• feat(errors): name the command a timeout was waiting on (#631)
• fix(client): release in-flight commands when the connection dies (#630)
• fix(subscriber): don't let a raising callback kill event dispatch (#629)
• fix(downloads): don't address the browser's implicit context by id (#628)
• fix: connect to a remote browser through the url (#625)
• fix(contexts): detect the browser's implicit context and create targets in it without id (#627)
• docs: add a Docker page (#626)
• fix: resume and detach from targets we don't track (#623)
• fix: kill the browser before removing its user data directory (#624)
• ci: cap test job runtime with timeout-minutes (#622)
• fix: don't block indefinitely waiting on the browser inside a finalizer (#621)
• fix: drop the no-op `--no-crashpad` default flag [#610] (#620)
• chore: logo light/dark mode
• feat: add touch emulation to `Page#set_viewport`'s `mobile:` option (#618)
• fix: changelog merge conflicts
• feat: add `#wait_for_selector` to support waiting for specific DOM elements [#82]
• fix: only check for cyclic references once per evaluate() result, not per nested object
• Add support for `browser.on(:response)`.
• fix: tests
• Do not retry -> rescue (and timeout in case of removal) on existing nodes (#360)
• feat: close stream
• feat: respect `:host` option when spawning Chrome
• feat: improve process termination and directory cleanup logic (#616)
• fix: handle `TimeoutError` in worker attach/detach to prevent subscriber crashes

✳️ cuprite (0.18 → 0.19) · Repo · Changelog

Release Notes

0.19

Added

• :emulate driver option to emulate a device listed in Chrome DevTools with its user agent, e.g.
  emulate: :iphone_14, along with :mobile and :scale_factor options it sets, kept across window resizes [#216]
• Node#drag_to accepts x: and y: to drop at an offset from the target, measured from its center or top left like
  click, so drop targets that tell the upper half from the lower one can be driven [#325]

Changed

• Ferrum is bumped to ~> 0.19.1
• --disable-web-security is added to the browser options, since Ferrum dropped it from its defaults. Tests keep
  reaching into cross-origin frames, e.g. current_url within a frame from another origin [rubycdp/ferrum#472]
• Change Node#send_keys to avoid triggering click on an already focused element. send_keys(:tab) without an
  element no longer clicks the middle of the page first, so Tab moves focus from the start of the page, and Capybara's
  #active_element and focused: specs run in the suite [#327]

Fixed

• Node#drag_to fired dragleave right before drop on an HTML5 drop target, which browsers never do, so libraries such
  as Pragmatic drag and drop dropped onto nothing. A drag now ends with drop if the target accepted it, otherwise
  with dragleave [#325]
• window_size: nil raised Ferrum::BrowserError: Invalid parameters on every page instead of leaving the window
  size to Chrome, so start-maximized couldn't be used [#278]
• Driver#reset! kept the page it had cached when the browser failed to reset, so every later example raised
  Session with given id not found or Browser is dead or given window is closed instead of starting on a fresh
  page [#274], [rubycdp/ferrum#641]
• Node#visible? reported an open popover or modal dialog as hidden when an ancestor had opacity: 0 [#328]
• Driver#window_handles listed the web workers a page spawned as windows, so window_opened_by raised
  opened 2 windows instead of 1 [#326]
• Node#select_option and Node#unselect_option didn't fire the input event on the select [#223]
• Node#trigger silently dispatched nothing for events other than mouse, focus and submit, such as input,
  change or custom events [#197]
• Node#set with an empty string cleared a text field without firing the input event [#164]
• Finders with exact_text: true raised InvalidSelector because Capybara appended an empty [] predicate to the
  XPath [#256]

Does any of this look wrong? Please let us know.

Commits

See the full diff on Github. The new version differs by 19 commits:

• chore(release): bump version to 0.19
• chore(deps): require ferrum ~> 0.19.1 for the held mouse button fix
• fix(drag): end HTML5 drags like a browser and support drop offsets in drag_to (#343)
• fix(driver): add --disable-web-security to the browser options (#342)
• `send_keys`: skip `click` if `Node` has focus (#327)
• test(ci): log test app requests to the CI artifacts (#340)
• fix(page): leave the window size to Chrome when window_size is nil (#339)
• feat(driver): emulate devices from Chrome DevTools with `emulate:` (#338)
• fix(browser): drop the empty xpath predicate Capybara adds for exact_text: true (#256) (#337)
• docs: add documentation pages for docs.rubycdp.com
• fix(node): fire input event when clearing a text field (#336)
• docs(readme): add logo
• fix(node): dispatch input, change and custom events from trigger (#335)
• fix(node): fire input event when selecting an option (#334)
• docs(readme): clarify proxy limitations for remote browsers
• fix(browser): list only page targets as window handles (#333)
• Follow-ups to #328: popover visibility (#331)
• Ignore ancestor opacity for open popovers and modal dialogs (#328)
• fix(browser): drop the cached page even when reset raises (#329)

---

Depfu will automatically keep this PR conflict-free, as long as you don't add any commits to this branch yourself. You can also trigger a rebase manually by commenting with @depfu rebase.

All Depfu comment commands

@​depfu rebase

Rebases against your default branch and redoes this update

@​depfu recreate

Recreates this PR, overwriting any edits that you've made to it

@​depfu merge

Merges this PR once your tests are passing and conflicts are resolved

@​depfu cancel merge

Cancels automatic merging of this PR

@​depfu close

Closes this PR and deletes the branch

@​depfu reopen

Restores the branch and reopens this PR (if it's closed)

@​depfu pause

Ignores all future updates for this dependency and closes this PR

@​depfu pause [minor|major]

Ignores all future minor/major updates for this dependency and closes this PR

@​depfu resume

Future versions of this dependency will create PRs again (leaves this PR as is)