Merge pull request #16 from eidam/e/cron-status-details

feat: collect response metrics

.prettierignore

@@ -0,0 +1,3 @@
+# Ignore generated files
+out
+public

CODE_OF_CONDUCT.md

@@ -14,21 +14,21 @@ appearance, race, religion, or sexual identity and orientation.
 Examples of behavior that contributes to creating a positive environment
 include:
 
-* Using welcoming and inclusive language
+- Using welcoming and inclusive language
-* Being respectful of differing viewpoints and experiences
+- Being respectful of differing viewpoints and experiences
-* Gracefully accepting constructive criticism
+- Gracefully accepting constructive criticism
-* Focusing on what is best for the community
+- Focusing on what is best for the community
-* Showing empathy towards other community members
+- Showing empathy towards other community members
 
 Examples of unacceptable behavior by participants include:
 
-* The use of sexualized language or imagery and unwelcome sexual attention or
+- The use of sexualized language or imagery and unwelcome sexual attention or
   advances
-* Trolling, insulting/derogatory comments, and personal or political attacks
+- Trolling, insulting/derogatory comments, and personal or political attacks
-* Public or private harassment
+- Public or private harassment
-* Publishing others' private information, such as a physical or electronic
+- Publishing others' private information, such as a physical or electronic
   address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a
+- Other conduct which could reasonably be considered inappropriate in a
   professional setting
 
 ## Our Responsibilities

README.md

@@ -10,15 +10,15 @@ Monitor your websites, showcase status including daily history, and get Slack no
 
 You'll need a [Cloudflare Workers account](https://dash.cloudflare.com/sign-up/workers) with
 
-* A workers domain set up
+- A workers domain set up
-* The Workers Bundled subscription \($5/mo\)
+- The Workers Bundled subscription \($5/mo\)
-    * [It works with Workers Free!](https://blog.cloudflare.com/workers-kv-free-tier/) Check [more info](#workers-kv-free-tier) on how to run on Workers Free. 
+  - [It works with Workers Free!](https://blog.cloudflare.com/workers-kv-free-tier/) Check [more info](#workers-kv-free-tier) on how to run on Workers Free.
-* Some websites/APIs to watch 🙂
+- Some websites/APIs to watch 🙂
 
 Also, prepare the following secrets
 
-* Cloudflare API token with `Edit Cloudflare Workers` permissions
+- Cloudflare API token with `Edit Cloudflare Workers` permissions
-* Slack incoming webhook \(optional\)
+- Slack incoming webhook \(optional\)
 
 ## Getting started
 
@@ -39,6 +39,7 @@ You can either deploy with **Cloudflare Deploy Button** using GitHub Actions or
    - Name: SECRET_SLACK_WEBHOOK_URL (optional)
    - Value: your-slack-webhook-url
    ```
+
 3. Navigate to the **Actions** settings in your repository and enable them
 4. Edit [config.yaml](./config.yaml) to adjust configuration and list all of your websites/APIs you want to monitor
 
@@ -73,32 +74,35 @@ You can either deploy with **Cloudflare Deploy Button** using GitHub Actions or
 5. Push to `main` branch to trigger the deployment
 6. 🎉
 7. _\(optional\)_ Go to [Cloudflare Workers settings](https://dash.cloudflare.com/?to=/workers) and assign custom domain/route
-   * e.g. `status-page.eidam.dev/*` _\(make sure you include `/*` as the Worker also serve static files\)_
+   - e.g. `status-page.eidam.dev/*` _\(make sure you include `/*` as the Worker also serve static files\)_
 8. _\(optional\)_ Edit [wrangler.toml](./wrangler.toml) to adjust Worker settings or CRON Trigger schedule, especially if you are on [Workers Free plan](#workers-kv-free-tier)
 
 ### Deploy on your own
 
 You can clone the repository yourself and use Wrangler CLI to develop/deploy, extra list of things you need to take care of:
 
-* create KV namespace and add the `KV_STATUS_PAGE` binding to [wrangler.toml](./wrangler.toml)
+- create KV namespace and add the `KV_STATUS_PAGE` binding to [wrangler.toml](./wrangler.toml)
-* create Worker secrets _\(optional\)_
+- create Worker secrets _\(optional\)_
-  * `SECRET_SLACK_WEBHOOK_URL`
+  - `SECRET_SLACK_WEBHOOK_URL`
 
 ## Workers KV free tier
+
 The Workers Free plan includes limited KV usage, but the quota is sufficient for 2-minute checks only
-* Change the CRON trigger to 2 minutes interval (`crons = ["*/2 * * * *"]`) in [wrangler.toml](./wrangler.toml)
+
+- Change the CRON trigger to 2 minutes interval (`crons = ["*/2 * * * *"]`) in [wrangler.toml](./wrangler.toml)
 
 ## Known issues
 
-* **Max 25 monitors to watch in case you are using Slack notifications**, due to the limit of subrequests Cloudflare Worker can make \(50\).
+- **Max 25 monitors to watch in case you are using Slack notifications**, due to the limit of subrequests Cloudflare Worker can make \(50\).
 
   The plan is to support up to 49 by sending only one Slack notification per scheduled run.
 
-* **KV replication lag** - You might get Slack notification instantly, however it may take couple of more seconds to see the change on your status page as [Cron Triggers are usually running on underutilized quiet hours machines](https://blog.cloudflare.com/introducing-cron-triggers-for-cloudflare-workers/#how-are-you-able-to-offer-this-feature-at-no-additional-cost).
+- **KV replication lag** - You might get Slack notification instantly, however it may take couple of more seconds to see the change on your status page as [Cron Triggers are usually running on underutilized quiet hours machines](https://blog.cloudflare.com/introducing-cron-triggers-for-cloudflare-workers/#how-are-you-able-to-offer-this-feature-at-no-additional-cost).
 
-* **Initial delay (no data)** - It takes couple of minutes to schedule and run CRON Triggers for the first time
+- **Initial delay (no data)** - It takes couple of minutes to schedule and run CRON Triggers for the first time
 
 ## Future plans
+
 Stay tuned for more features coming in, like leveraging the fact that CRON instances are scheduled around the world during the day
 so we can monitor the response times. However, we will most probably wait for the [Durable Objects](https://blog.cloudflare.com/introducing-workers-durable-objects/) to be in open beta
 as they are better fit to reliably store such info.

SUMMARY.md

@@ -1,4 +1,3 @@
 # Table of contents
 
-* [Cloudflare Worker - Status Page](README.md)
+- [Cloudflare Worker - Status Page](README.md)
-

config.yaml

@@ -3,6 +3,7 @@ settings:
   url: 'https://status-page.eidam.dev' # used for Slack messages
   logo: logo-192x192.png # image in ./public/ folder
   daysInHistogram: 90 # number of days you want to display in histogram
+  collectResponseTimes: false # experimental feature, enable only for <5 monitors or on paid plans
 
   allmonitorsOperational: 'All Systems Operational'
   notAllmonitorsOperational: 'Not All Systems Operational'
@@ -11,7 +12,7 @@ settings:
   monitorLabelNoData: 'No data'
   dayInHistogramNoData: 'No data'
   dayInHistogramOperational: 'All good'
-  dayInHistogramNotOperational: 'Some checks failed'
+  dayInHistogramNotOperational: ' incident(s)' # xx incident(s) recorded
 
 monitors:
   - id: workers-cloudflare-com # unique identifier

pages/index.js

@@ -2,7 +2,7 @@ import { Store } from 'laco'
 import { useStore } from 'laco-react'
 import Head from 'flareact/head'
 
-import { getMonitors, useKeyPress, switchTheme } from '../src/functions/helpers'
+import { getKVMonitors, useKeyPress } from '../src/functions/helpers'
 import config from '../config.yaml'
 import MonitorCard from '../src/components/monitorCard'
 import MonitorFilter from '../src/components/monitorFilter'
@@ -24,23 +24,20 @@ const filterByTerm = (term) =>
 
 export async function getEdgeProps() {
   // get KV data
-  const {
+  const kvMonitors = await getKVMonitors()
-    value: kvMonitors,
-    metadata: kvMonitorsMetadata,
-  } = await getMonitors()
 
   return {
     props: {
       config,
-      kvMonitors: kvMonitors || {},
+      kvMonitors: kvMonitors ? kvMonitors.monitors : {},
-      kvMonitorsMetadata: kvMonitorsMetadata || {},
+      kvMonitorsLastUpdate: kvMonitors ? kvMonitors.lastUpdate : {},
     },
     // Revalidate these props once every x seconds
     revalidate: 5,
   }
 }
 
-export default function Index({ config, kvMonitors, kvMonitorsMetadata }) {
+export default function Index({ config, kvMonitors, kvMonitorsLastUpdate }) {
   const state = useStore(MonitorStore)
   const slash = useKeyPress('/')
 
@@ -81,7 +78,7 @@ export default function Index({ config, kvMonitors, kvMonitorsMetadata }) {
             <MonitorFilter active={slash} callback={filterByTerm} />
           </div>
         </div>
-        <MonitorStatusHeader kvMonitorsMetadata={kvMonitorsMetadata} />
+        <MonitorStatusHeader kvMonitorsLastUpdate={kvMonitorsLastUpdate} />
         {state.visible.map((monitor, key) => {
           return (
             <MonitorCard

public/tailwind.css

@@ -58,7 +58,7 @@
 
   .tooltip .content {
     @apply invisible absolute z-50 inline-block;
-    @apply rounded-full py-1 px-2 bg-gray-100 dark:bg-gray-800 shadow;
+    @apply rounded-lg py-1 px-2 bg-gray-100 dark:bg-gray-800 shadow;
     @apply opacity-0 transition-all duration-200 scale-50;
   }
 

src/components/monitorHistogram.js

@@ -16,14 +16,17 @@ export default function MonitorHistogram({ monitorId, kvMonitor }) {
         let bg = ''
         let dayInHistogramLabel = config.settings.dayInHistogramNoData
 
-        // filter all dates before first check, check the rest
+        // filter all dates before first check, then check the rest
         if (kvMonitor && kvMonitor.firstCheck <= dayInHistogram) {
-          if (!kvMonitor.failedDays.includes(dayInHistogram)) {
+          if (
+            kvMonitor.checks.hasOwnProperty(dayInHistogram) &&
+            kvMonitor.checks[dayInHistogram].fails > 0
+          ) {
+            bg = 'yellow'
+            dayInHistogramLabel = `${kvMonitor.checks[dayInHistogram].fails} ${config.settings.dayInHistogramNotOperational}`
+          } else {
             bg = 'green'
             dayInHistogramLabel = config.settings.dayInHistogramOperational
-          } else {
-            bg = 'yellow'
-            dayInHistogramLabel = config.settings.dayInHistogramNotOperational
           }
         }
 
@@ -36,6 +39,15 @@ export default function MonitorHistogram({ monitorId, kvMonitor }) {
               <span className="font-semibold text-sm">
                 {dayInHistogramLabel}
               </span>
+              {kvMonitor.checks.hasOwnProperty(dayInHistogram) &&
+                Object.keys(kvMonitor.checks[dayInHistogram].res).map((key) => {
+                  return (
+                    <>
+                      <br />
+                      {key}: {kvMonitor.checks[dayInHistogram].res[key].a}ms
+                    </>
+                  )
+                })}
             </div>
           </div>
         )

src/components/monitorStatusHeader.js

@@ -7,11 +7,11 @@ const classes = {
     'bg-yellow-200 text-yellow-700 dark:bg-yellow-700 dark:text-yellow-200 border-yellow-300 dark:border-yellow-600',
 }
 
-export default function MonitorStatusHeader({ kvMonitorsMetadata }) {
+export default function MonitorStatusHeader({ kvMonitorsLastUpdate }) {
   let color = 'green'
   let text = config.settings.allmonitorsOperational
 
-  if (!kvMonitorsMetadata.monitorsOperational) {
+  if (!kvMonitorsLastUpdate.allOperational) {
     color = 'yellow'
     text = config.settings.notAllmonitorsOperational
   }
@@ -20,13 +20,11 @@ export default function MonitorStatusHeader({ kvMonitorsMetadata }) {
     <div className={`card mb-4 font-semibold ${classes[color]}`}>
       <div className="flex flex-row justify-between items-center">
         <div>{text}</div>
-        {kvMonitorsMetadata.lastUpdate && typeof window !== 'undefined' && (
+        {kvMonitorsLastUpdate.time && typeof window !== 'undefined' && (
           <div className="text-xs font-light">
             checked{' '}
-            {Math.round(
+            {Math.round((Date.now() - kvMonitorsLastUpdate.time) / 1000)} sec
-              (Date.now() - kvMonitorsMetadata.lastUpdate.time) / 1000,
+            ago (from {kvMonitorsLastUpdate.loc})
-            )}{' '}
-            sec ago (from {kvMonitorsMetadata.lastUpdate.loc})
           </div>
         )}
       </div>

src/components/monitorStatusLabel.js

@@ -12,7 +12,7 @@ export default function MonitorStatusLabel({ kvMonitor }) {
   let text = 'No data'
 
   if (typeof kvMonitor !== 'undefined') {
-    if (kvMonitor.operational) {
+    if (kvMonitor.lastCheck.operational) {
       color = 'green'
       text = config.settings.monitorLabelOperational
     } else {

src/functions/cronTrigger.js

@@ -1,33 +1,40 @@
 import config from '../../config.yaml'
 
-import { setKV, getKVWithMetadata, notifySlack } from './helpers'
+import {
+  notifySlack,
+  getCheckLocation,
+  getKVMonitors,
+  setKVMonitors,
+} from './helpers'
 
 function getDate() {
   return new Date().toISOString().split('T')[0]
 }
 
 export async function processCronTrigger(event) {
+  // Get Worker PoP and save it to monitorsStateMetadata
+  const checkLocation = await getCheckLocation()
+  const checkDay = getDate()
+
   // Get monitors state from KV
-  let {
+  let monitorsState = await getKVMonitors()
-    value: monitorsState,
-    metadata: monitorsStateMetadata,
-  } = await getKVWithMetadata('monitors_data', 'json')
 
   // Create empty state objects if not exists in KV storage yet
   if (!monitorsState) {
-    monitorsState = {}
+    monitorsState = { lastUpdate: {}, monitors: {} }
-  }
-  if (!monitorsStateMetadata) {
-    monitorsStateMetadata = {}
   }
 
   // Reset default all monitors state to true
-  monitorsStateMetadata.monitorsOperational = true
+  monitorsState.lastUpdate.allOperational = true
 
   for (const monitor of config.monitors) {
     // Create default monitor state if does not exist yet
-    if (typeof monitorsState[monitor.id] === 'undefined') {
+    if (typeof monitorsState.monitors[monitor.id] === 'undefined') {
-      monitorsState[monitor.id] = { failedDays: [] }
+      monitorsState.monitors[monitor.id] = {
+        firstCheck: checkDay,
+        lastCheck: {},
+        checks: {},
+      }
     }
 
     console.log(`Checking ${monitor.name} ...`)
@@ -41,52 +48,90 @@ export async function processCronTrigger(event) {
       },
     }
 
+    // Perform a check and measure time
+    const requestStartTime = Date.now()
     const checkResponse = await fetch(monitor.url, init)
+    const requestTime = Math.round(Date.now() - requestStartTime)
+
+    // Determine whether operational and status changed
     const monitorOperational =
       checkResponse.status === (monitor.expectStatus || 200)
+    const monitorStatusChanged =
+      monitorsState.monitors[monitor.id].lastCheck.operational !==
+      monitorOperational
+
+    // Save monitor's last check response status
+    monitorsState.monitors[monitor.id].lastCheck = {
+      status: checkResponse.status,
+      statusText: checkResponse.statusText,
+      operational: monitorOperational,
+    }
 
     // Send Slack message on monitor change
     if (
-      monitorsState[monitor.id].operational !== monitorOperational &&
+      monitorStatusChanged &&
       typeof SECRET_SLACK_WEBHOOK_URL !== 'undefined' &&
       SECRET_SLACK_WEBHOOK_URL !== 'default-gh-action-secret'
     ) {
       event.waitUntil(notifySlack(monitor, monitorOperational))
     }
 
-    monitorsState[monitor.id].operational =
+    // make sure checkDay exists in checks in cases when needed
-      checkResponse.status === (monitor.expectStatus || 200)
+    if (
-    monitorsState[monitor.id].firstCheck =
+      (config.settings.collectResponseTimes || !monitorOperational) &&
-      monitorsState[monitor.id].firstCheck || getDate()
+      !monitorsState.monitors[monitor.id].checks.hasOwnProperty(checkDay)
-
+    ) {
-    // Set monitorsOperational and push current day to failedDays
+      monitorsState.monitors[monitor.id].checks[checkDay] = {
-    if (!monitorOperational) {
+        fails: 0,
-      monitorsStateMetadata.monitorsOperational = false
+        res: {},
-
-      const failedDay = getDate()
-      if (!monitorsState[monitor.id].failedDays.includes(failedDay)) {
-        console.log('Saving new failed daily status ...')
-        monitorsState[monitor.id].failedDays.push(failedDay)
-      }
       }
     }
 
-  // Get Worker PoP and save it to monitorsStateMetadata
+    if (config.settings.collectResponseTimes && monitorOperational) {
-  const res = await fetch('https://cloudflare-dns.com/dns-query', {
+      // make sure location exists in current checkDay
-    method: 'OPTIONS',
+      if (
-  })
+        !monitorsState.monitors[monitor.id].checks[checkDay].res.hasOwnProperty(
-  const loc = res.headers.get('cf-ray').split('-')[1]
+          checkLocation,
-  monitorsStateMetadata.lastUpdate = {
-    loc,
-    time: Date.now(),
-  }
-
-  // Save monitorsState and monitorsStateMetadata to KV storage
-  await setKV(
-    'monitors_data',
-    JSON.stringify(monitorsState),
-    monitorsStateMetadata,
         )
+      ) {
+        monitorsState.monitors[monitor.id].checks[checkDay].res[
+          checkLocation
+        ] = {
+          n: 0,
+          ms: 0,
+          a: 0,
+        }
+      }
+
+      // increment number of checks and sum of ms
+      const no = ++monitorsState.monitors[monitor.id].checks[checkDay].res[
+        checkLocation
+      ].n
+      const ms = (monitorsState.monitors[monitor.id].checks[checkDay].res[
+        checkLocation
+      ].ms += requestTime)
+
+      // save new average ms
+      monitorsState.monitors[monitor.id].checks[checkDay].res[
+        checkLocation
+      ].a = Math.round(ms / no)
+    } else if (!monitorOperational) {
+      // Save allOperational to false
+      monitorsState.lastUpdate.allOperational = false
+
+      // Increment failed checks, only on status change (maybe call it .incidents instead?)
+      if (monitorStatusChanged) {
+        monitorsState.monitors[monitor.id].checks[checkDay].fails++
+      }
+    }
+  }
+
+  // Save last update information
+  monitorsState.lastUpdate.time = Date.now()
+  monitorsState.lastUpdate.loc = checkLocation
+
+  // Save monitorsState to KV storage
+  await setKVMonitors(monitorsState)
 
   return new Response('OK')
 }

src/functions/helpers.js

@@ -1,18 +1,20 @@
 import config from '../../config.yaml'
 import { useEffect, useState } from 'react'
 
-export async function getMonitors() {
+export async function getKVMonitors() {
-  return await getKVWithMetadata('monitors_data', 'json')
+  // trying both to see performance difference
+  return KV_STATUS_PAGE.get('monitors_data', 'json')
+  //return JSON.parse(await KV_STATUS_PAGE.get('monitors_data', 'text'))
+}
+
+export async function setKVMonitors(data) {
+  return setKV('monitors_data', JSON.stringify(data))
 }
 
 export async function setKV(key, value, metadata, expirationTtl) {
   return KV_STATUS_PAGE.put(key, value, { metadata, expirationTtl })
 }
 
-export async function getKVWithMetadata(key, type = 'text') {
-  return KV_STATUS_PAGE.getWithMetadata(key, type)
-}
-
 export async function notifySlack(monitor, operational) {
   const payload = {
     attachments: [
@@ -23,7 +25,8 @@ export async function notifySlack(monitor, operational) {
             type: 'section',
             text: {
               type: 'mrkdwn',
-              text: `Monitor *${monitor.name}* changed status to *${operational
+              text: `Monitor *${monitor.name}* changed status to *${
+                operational
                   ? config.settings.monitorLabelOperational
                   : config.settings.monitorLabelNotOperational
               }*`,
@@ -34,8 +37,10 @@ export async function notifySlack(monitor, operational) {
             elements: [
               {
                 type: 'mrkdwn',
-                text: `${operational ? ':white_check_mark:' : ':x:'} \`${monitor.method ? monitor.method : 'GET'
+                text: `${operational ? ':white_check_mark:' : ':x:'} \`${
-                  } ${monitor.url}\` - :eyes: <${config.settings.url
+                  monitor.method ? monitor.method : 'GET'
+                } ${monitor.url}\` - :eyes: <${
+                  config.settings.url
                 }|Status Page>`,
               },
             ],
@@ -78,3 +83,10 @@ export function useKeyPress(targetKey) {
 
   return keyPressed
 }
+
+export async function getCheckLocation() {
+  const res = await fetch('https://cloudflare-dns.com/dns-query', {
+    method: 'OPTIONS',
+  })
+  return res.headers.get('cf-ray').split('-')[1]
+}