Modernize the dashboard JavaScript and improve things #558

ivan · 2023-06-09T10:17:35Z

Notable changes:

convert the JavaScript from ES5 to ES2020
move dashboard JavaScript to its own file so that we can format it with rome check
reduce default historyLines from 1000 to 500 to compensate for many more jobs than before
optimize: remove animations for all browsers, add CSS contain and content-visibility on log windows
optimize: avoid reflows by not scrolling hidden log windows (was ~40% of our time in Firefox, ~30% in Chrome)
limit the size of the BatchingQueue to fix the issue that caused Chromium-based browsers to hang when returning to the tab
context menu: limit to 8 ignore suggestions until you click to show more
context menu: create ignores based on the original URL characters
(Dashboard escapes { and } as %7B and %7D when copying ignores from context menu #340)
start with all jobs collapsed by default and add &initialFilter= URL parameter
document more URL parameters in the help text, including &loadRecent=0 to skip loading recent data
show progress when loading the recent data
show WebSocket messages/sec and KB/s
if uBlock Origin cosmetic filtering is detected, tell user to read help and turn off ad blocker
CSS improvements: make font sizes, families, and weights more consistent

I had to move some dashboard3 things to avoid confusing the two dashboard JavaScript files.

I didn't test whether the Ruby still serves things correctly.

obsolete tests results for 9cf8f41

Test results with all job windows collapsed, Chrome 116.0.5817.5 (Official Build) dev (64-bit) on Windows 10 22H2 on an ancient laptop (Intel Core i7-2620M), no browser extensions:

9cf8f41 uses 55% of the CPU time of the current dashboard and ~25% as much memory.

Test results with all job windows collapsed, Firefox 114.0 on Windows 10 22H2 on an ancient laptop (Intel Core i7-2620M), no browser extensions:

9cf8f41 uses ~95% of the CPU time of the current dashboard and about the same amount of memory (it fluctuates as Firefox takes a while to do a full GC).

Unlike Chrome's Task Manager, Firefox's about:processes doesn't have a CPU time column, so the difference is less measurable.

Test results with all job windows collapsed, Chrome 116.0.5817.5 (Official Build) dev (64-bit) on Windows 10 22H2 on an ancient laptop (Intel Core i7-2620M), no browser extensions:

f7d5b1a uses 40% of the CPU time of the current dashboard and ~25% as much memory.

Test results with all job windows collapsed, Firefox 114.0 on Windows 10 22H2 on an ancient laptop (Intel Core i7-2620M), no browser extensions:

f7d5b1a uses ~57% of the CPU time of the current dashboard and about the same amount of memory (it fluctuates as Firefox takes a while to do a full GC).

Unlike Chrome's Task Manager, Firefox's about:processes doesn't have a CPU time column, so the difference is less measurable.

…erver now sends ~250 messages/sec

… with rome

The dashboard displays 2-3 times as many jobs as it used to, and we need to do something to mitigate the performance impact.

Their performance wasn't good enough in Chrome, even on a 7950X3D.

…pre-ES6 implementation

…mplementation

…) {`

…efault

We don't really need a dynamic !con; it's easy enough to backspace and change the number yourself.

…t Chrome Natively, browsers either make mousewheel close the context menu (Chrome on Windows), or swallow all mousewheel events while the context menu is open, doing nothing until the context menu is closed (any browser on macOS). We might want to implement that swallowing behavior later for some (browser, OS) combinations, but it's not very important.

…rease performance https://developer.mozilla.org/en-US/docs/Web/CSS/contain

This is the result of: rome check dashboard/dashboard.js --apply-unsafe --line-width 120 with my own manual changes to `getSummaryResponses` and the replacement of some `// comments` with `/* comments */`

More predictable, less annoying (doesn't break ctrl+a and other keys after focusing the input)

…e easier to test locally

…onds Limited to 31 max readings, to be fixed in an upcoming commit.

…rage the rate over 3 sec and remove the cap on the number of readings

…ntirely

…ers, not the escaped URL in the DOM Fixes #340

…at lines fit nicely (at least when there is no horizontal scrollbar)

…rformance

…batchTimeWhenVisible This reduces CPU usage.

…ents This might increase performance.

…very filter change `updateAlign` was seen using a lot of CPU time in the Chrome DevTools profiler.

… outside the viewport

Inspired by SEDA's batching, of course.

This reduces CPU use by ~43% in Firefox and ~30% in Chrome. Scroll log windows to the bottom only after they are unhidden again.

…g strings in the queue This doesn't make much of a difference, but it seems pointless to batch the parsing of JSON strings, given that JSON.parse is both fairly small and likely to be hot.

JustAnotherArchivist

So many changes and the code moves that it's too annoying to fully review. But the commits look reasonable, it works well, and it's a great improvement; that's good enough for me.

Thank you!

ivan added 30 commits June 7, 2023 22:43

dashboard: fix whitespace on control flow statements

ab96deb

dashboard: continue even without /logs/recent data

3d92ccd

dashboard: reduce batchTimeWhenHidden to 1000 because the WebSocket s…

2272154

…erver now sends ~250 messages/sec

dashboard: move JavaScript to its own file so that it can be analyzed…

7be1a8e

… with rome

dashboard: replace var with let or const

7946d7f

dashboard: use for-of to iterate arrays

6b62e8f

dashboard: reduce default historyLines to 500

c2cf784

The dashboard displays 2-3 times as many jobs as it used to, and we need to do something to mitigate the performance impact.

dashboard: remove animations for all browsers

cf8abc6

Their performance wasn't good enough in Chrome, even on a 7950X3D.

dashboard: replace all .bind(this) with arrow functions

2bea325

dashboard: use String.prototype.{startsWith,endsWith} instead of our …

e1d5905

…pre-ES6 implementation

dashboard: use Array.prototype.findIndex instead of our own pre-ES6 i…

4faa46b

…mplementation

dashboard: use function X(...) { instead of `const X = function(...…

7fc1752

…) {`

dashboard: remove overscroll-behavior: contain emulation

35308c9

dashboard: context menu: add suggested command to set delay back to d…

7373edd

…efault

dashboard: fix indentation

b1dadfb

dashboard: remove some old TODO items

5508ac6

We don't really need a dynamic !con; it's easy enough to backspace and change the number yourself.

dashboard: remove unused browser version checking functions

0dc4794

dashboard: use ES6 classes

438a75b

dashboard: use Array.from instead of our own pre-ES6 implementation

ec6aea2

dashboard: use arrow functions instead of anonymous functions

52ba003

dashboard: use CSS contain: strict on the log windows to try to inc…

66795b2

…rease performance https://developer.mozilla.org/en-US/docs/Web/CSS/contain

dashboard: remove long-obsolete X-UA-Compatible header

ccec511

dashboard: load dashboard.js async; don't document.write

f0202cc

dashboard: remove unnecessary semicolons

c2ceef4

dashboard: apply fixes and format with rome

0cf0913

This is the result of: rome check dashboard/dashboard.js --apply-unsafe --line-width 120 with my own manual changes to `getSummaryResponses` and the replacement of some `// comments` with `/* comments */`

dashboard: fix bug introduced by rome check's --apply-unsafe

eb3b27d

dashboard: mention that the file should be formatted with rome

d45eb79

dashboard: remove unused argument that should not have been here

7e0e930

dashboard: remove some commented-out console.log lines

69f133a

ivan added 22 commits June 9, 2023 11:30

dashboard: put the job ident in a span instead of an <input>

58e8c3f

More predictable, less annoying (doesn't break ctrl+a and other keys after focusing the input)

dashboard: help: not all the browser tips are important

719af9f

dashboard: use relative path assets/dashboard.js to make this a littl…

3c9dd02

…e easier to test locally

dashboard: average the msg/s and KB/s rates shown over the last 2 sec…

87a3c95

…onds Limited to 31 max readings, to be fixed in an upcoming commit.

dashboard: optimize RateTracker by not changing the array length; ave…

e5bf5a4

…rage the rate over 3 sec and remove the cap on the number of readings

dashboard: optimize RateTracker

c0f5389

dashboard: optimize RateTracker

40bd8f5

dashboard: keep updating the rates even when we stop receiving data e…

7f94238

…ntirely

dashboard: create suggested ignores based on the original URL charact…

7077606

…ers, not the escaped URL in the DOM Fixes #340

dashboard: make the log window heights a multiple of (13 + 4)px so th…

a80ca10

…at lines fit nicely (at least when there is no horizontal scrollbar)

dashboard: format with rome

fc969b3

dashboard: help: add note about other browser extensions impacting pe…

2c66a86

…rformance

dashboard: update the rate information every 1000ms instead of every …

4bea178

…batchTimeWhenVisible This reduces CPU usage.

dashboard: add contain: strict on the frequently-updated stats elem…

c2fab74

…ents This might increase performance.

dashboard: don't search the DOM for elements to align/unalign after e…

bf1e367

…very filter change `updateAlign` was seen using a lot of CPU time in the Chrome DevTools profiler.

dashboard: add a log-container class for testing and future use

bf94278

dashboard: use content-visibility to try to avoid rendering updates…

1c9ed0e

… outside the viewport

dashboard: parse all the JSON before doing DOM updates

9cf8f41

Inspired by SEDA's batching, of course.

dashboard: don't scroll hidden log windows to avoid reflows

f7d5b1a

This reduces CPU use by ~43% in Firefox and ~30% in Chrome. Scroll log windows to the bottom only after they are unhidden again.

dashboard: parse JSON messages as we receive them instead of retainin…

079e356

…g strings in the queue This doesn't make much of a difference, but it seems pointless to batch the parsing of JSON strings, given that JSON.parse is both fairly small and likely to be hot.

dashboard: fix the units: a megabyte is a million bytes

81d06de

dashboard: show "NaN MB" instead of "NaN.0 MB" when n is undefined

61af407

JustAnotherArchivist approved these changes Jun 30, 2023

View reviewed changes

JustAnotherArchivist linked an issue Jun 30, 2023 that may be closed by this pull request

Dashboard escapes { and } as %7B and %7D when copying ignores from context menu #340

Closed

JustAnotherArchivist added bug enhancement dashboard Client-side dashboard only; dashboard server things should use the backend label instead. labels Jun 30, 2023

JustAnotherArchivist merged commit 86e4009 into ArchiveTeam:master Jun 30, 2023

JustAnotherArchivist mentioned this pull request Jun 30, 2023

Poor dashboard performance scaling #384

Open

ivan mentioned this pull request Jul 5, 2023

Seeking new maintainer / project owner ArchiveTeam/grab-site#146

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modernize the dashboard JavaScript and improve things #558

Modernize the dashboard JavaScript and improve things #558

ivan commented Jun 9, 2023 •

edited

Loading

JustAnotherArchivist left a comment

Modernize the dashboard JavaScript and improve things #558

Modernize the dashboard JavaScript and improve things #558

Conversation

ivan commented Jun 9, 2023 • edited Loading

JustAnotherArchivist left a comment

Choose a reason for hiding this comment

ivan commented Jun 9, 2023 •

edited

Loading