Looking to capture some performance metrics for website from the Linux command line and eventually get it into Cacti (RRD).

Here are my scattered notes on this process. I’m not very familiar with NodeJS stuff, so I’m documenting from installation of NodeJS on Debian 11 to creating the project.

Install NodeJS, Puppeteer and Chromium headless on Debian 11

Install NodeJS

curl -fsSL https://deb.nodesource.com/setup_14.x | sudo -E bash -

apt install -y nodejs

Create Project

mkdir test_project
cd test_project
npm init

Install NodeJS Puppeteer

npm i puppeteer --save

Install Debian dependencies for Chromium

See: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#chrome-headless-doesnt-launch-on-unix

This is what I needed to grab:

apt install libatk-bridge2.0-0 libatk1.0-0 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxrandr2 libxrender1 libgbm1 libxkbcommon-x11-0

Write the Application

This is basic idea copied and modified from something I found online on SO. It takes an argument, the website, passed.

Create index.js:

const puppeteer = require('puppeteer');

(async () => {
        const browser = await puppeteer.launch({
        ignoreDefaultArgs: ['--no-sandbox'],
});
        const page = await browser.newPage();

        const t1 = Date.now();
        await page.goto(process.argv[2], { waitUntil: 'networkidle0'});
        const diff1 = Date.now() - t1;

        await browser.close();
        console.log(`Time: ${diff1}ms`);
})();

To run it:

node app.js https://google.com

Example output:

Time: 1201ms

Integrate with Cacti

TODO

The basic idea is to be able to call the node app.js https://website/ and have it return a metric (milliseconds) that can be stored into an RRD and then graphed upon. Concern would be ensuring that the poller allows for script completion — I’m not sure what would happen if node can’t complete the job before the poller times out.

Other Methods

Some things I scoured from the internet.

Curl

curl -s -w 'Testing Website Response Time for :%{url_effective}\n\nLookup Time:\t\t%{time_namelookup}\nConnect Time:\t\t%{time_connect}\nAppCon Time:\t\t%{time_appconnect}\nRedirect Time:\t\t%{time_redirect}\nPre-transfer Time:\t%{time_pretransfer}\nStart-transfer Time:\t%{time_starttransfer}\n\nTotal Time:\t\t%{time_total}\n' -o /dev/null https://jonesperformance.com/wp-json/wc/v3
Testing Website Response Time for :https://jonesperformance.com/wp-json/wc/v3

Lookup Time:            0.004972
Connect Time:           0.053358
AppCon Time:            0.112053
Redirect Time:          0.000000
Pre-transfer Time:      0.112155
Start-transfer Time:    0.746088

Total Time:             0.851602

Cacti: Using Cacti to monitor web page loading

The AskAboutPHP.com has a PHP script to grab some info (not rendering, but at least some of the connection timings) and walks through how to integrate with Cacti for graphing. There are 3 parts:

Part 1: http://www.askaboutphp.com/2008/09/17/cacti-using-cacti-to-monitor-web-page-loading-part-1/

Part 2: http://www.askaboutphp.com/2008/09/19/cacti-using-cacti-to-monitor-web-page-loading-part-2/

Part 3: http://www.askaboutphp.com/2008/09/19/cacti-using-cacti-to-monitor-web-page-loading-part-3/

For modern systems, you’ll need to fix up the pageload-agent.php file to fix the line and remove deprecated and removed function eregi to match the following (line 10):

if (!preg_match('/^https?:\/\//', $url_argv, $matches)) {
                $url_argv = "https://$url_argv";
        }