Tabulations

Views are accessed through the tabulations in the site's pages header.

Each view has a persistent filters bar and column selector (folded under the tabs), a column filters bar (top), and a tool bar (bottom).

Persistent filters bar

The orange thick line under the tabs can be unfolded by clicking on it or through the ctrl+: or ctrl+; shortcuts. The filters set there are stored in database and are applied on all collector views. They persist across navigation sessions. Filters can apply on data not presented in the current view or embed hardcoded complex queries, like 'nodes with services'.

Upon filter selection, a tag cloud appears to display all possible values of the filter. Clicking a tag will set the filter value box. The filter value follow the same syntax rules as the column filters.

A filter/value is added by clicking on the '+' blue icon and removed by clicking on the 'x' red icon.

Column selector

View columns can be toggled on and off from the column selector. Each view has a default set of column displayed. User column selection is stored in database, thus persistent across user sessions and different browsing devices.

Column filters bar

Each column of the table supports filtering. Filters are applied upon <enter> in any filter box. Filter box expressions are and'ed. Filter syntax supports :

• %: Wildcard for SQL expressions. Can be used as a header, trailer or in the middle of a pattern (%gie%01%)

• &: And multiple patterns (%gie%&%adomain.com)

• |: Or multiple patterns (%gie%|%adomain.com)

• !: Negate the pattern (!%gie%&%adomain.com)

• empty: Select only empty cells (!empty)

Service tabs

In all views, clicking on a service name will open the service tabs to display the most comprehensive data set the collector knows about this service.

properties

opensvc version

Pushed to database daily by the opensvc nodeware cronjob.

unackowledged errors

A count of the unacknowledged errors on this service found in the actions table.

type

Service type can be PRD, DEV, ...

comment

Pushed to database daily by the opensvc nodeware cronjob. This information resides in the service env file on the nodes.

last update

Timestamp updated upon service information receive from the nodeware cronjob.

container name

Host Name of the virtual machine encapsulated in the service, if any.

container type

Virtualisation driver to handle the encapsulated virtual machine, if any.

responsibles

List of administrators contact names for this service.

responsibles mail

List of administrators contact emails for this service.

primary node

Host name of the node where the service should be running in optimal situation.

nodes

All nodes where the service may be running in degraded situation (not in disaster recovery situation).

drp node

Host name of the node the service should be running in a disaster recovery situation.

drp nodes

All nodes where the service may be running in disaster recovery situation.

status

Display the per-node synthetic service status.

resources

Display the per-node per-resource detailled service status.

env

Display the raw service configuration file pushed by the nodeware.

topology

Sketches service-nodes-storage-sites relationships

Node tabs

In all views, clicking on a node name will open the node tabs to display the most comprehensive data set the collector knows about this node.

server, os, mem, cpu, location, power

Information displayed in these tabs are extracted from the asset table. Assets can be loaded from a tiers enterprise asset manager or filled-in manually.

stats

Generate and display graphs of performance metrics on the user-selected time range. The metrics are gathered daily from tools like glance and sar running on the nodes. The opensvc nodeware relays this information to database.

metrics

Metrics collection depend on the operating system. Some of them are:

• cpu: usr, sys, wait, irq, guest, steal, idle, nice, iowait, soft

• mem: commit, free, used by usr, buffer, cache, sys

• swap: free, used, used but cached

• load: avg1, avg5, avg15, runq size, proc list size

• all-disks: io/s, bandwidth

• per-disk: io/s, request size, wait time, service time, %util, bandwidth

screencast

Tool bar

The tool bar hosts generic tools and view-specific tools.

Tool	Scope	Description
	generic	This pager allows you to navigate the current filtered data cursor. It displays at most 10 pages of 25 rows. Clicking 'all' will display all rows in a single page (might be hard on the collector resources). Hovering the mouse over the 'all' marker will display the data cursor statistics. Power users can change the number of rows per page by setting the GET parameter 'perpage=10'.
	generic	Export table content as a CSV file.
	services	Generate a diagram of the current filtered data cursor mixing information from the service configurations, computer assets and storage assets. Example below.
	actions	Subscribe to RSS feed of changes on the current filtered data cursor.
	actions	Acknowledge failed actions whose checker box is marked as selected. Clicking on the ack icon will spawn an input box with active cursor so you can optionally enter an aknowledge comment. Comment, ack date and acker name will be displayed on hovering the mouse over the 'err' status markers.

dashboard

Usage

This page display all site's problems detected by the collector. The session filters applies, so administrators can only see the problems affecting the nodes and services they are responsible for.

This is the one-stop page for administrators morning checks, and for production team health check.

Layout:

• Summary : shows the number of problems detected per check type. Only the checks with problems are shown by default. Clicking on the table expands the table to show the checks with no detected problems too. Clicking on a table line reposition the page on the clicked check detailled table.

  

• Per-check details : a table per check type.

  

services view

Usage

This view presents services health.

Main usage:

• Quick insight of live service status.
  Example: primary node & responsible=me & status!=ok

• List services matching criteria.
  Example, 'How many non-production services do I manage for the ERP system':
  primary node & app=ERP & type=!PRD
  Example, 'I lost a power line, which services are impacted':
  nb_power_supply=1 & power_supply_1=POW_A

• Generate diagrams of clusters to better communicate on disaster recovery plans and allocated resources.

• Trigger actions on services or group of services

Screenshot

Fields

Responsible

The responsibles are persons receiving alerts for a service. Hovering the mouse over the icon spawns the name of the responsibles. No icon means no responsible, which is an anomaly, and as such, cause alerts to be emitted to the site's administrator/manager.

Status

Status gizmos present the overall service health on top, and individual resource type status below. Color codes are:

	green resource is up red resource is down orange resource has warnings grey resource type is not used by the service
	An all-greyed-out status marker indicates the presented data is too old to be trusted, though it displays the last overall known status. Hovering the mouse over the status markers will displayed the data refresh timestamp.

Tools

Actions

Dashboard notifications

actions view

Usage

This view displays the journal of all services events with their status.

Main usage:

• Quick insight of unacknowledged errors affecting a specific perimeter.
  Example: errors not acknowledged & responsible=me

• Retrospective problem analysis.
  Example: service=some_service & begin>2010-01-10% & end<2010-01-12%

• Problem patterns.
  Example: service=some_service & action=syncnodes & status!=ok

• Learn which system commands are run by the nodeware
  Example: log=%sg_persist%|%scu%

• Trace who was alerted of a problem, and when

Screenshot

Fields

Responsible

The responsibles are persons receiving alerts for a service. Hovering the mouse over the icon spawns the name of the responsibles. No icon means no responsible, which is an anomaly, and as such, cause alerts to be emitted to the site's administrator/manager.

Dashboard notifications

checks view

Usage

The OpenSVC nodeware gathers check test values from the nodes and pushes them to the collector. Current implemented checks are:

• fs_u : pushes each filesystems usage percent
• vg_u : pushes each volume group usage percent

More checks will be implemented when client need arise.

Each check is validated against high and low threshold values. If the current check value is outside boundaries, the check raises a warning in the dashboard and the value and crossed threshold are highlighted red in the checks view.

A check without value update for more than 24 hours will have its update date highlighted red. These checks instances should not be trusted.

Screenshot

Dashboard notifications

Setting thresholds

Check types have default high and low threshold values. These values can be customized for each node and instance of the check. Two customization methods are proposed:

• From the dashboard : clicking on a threshold value in the check warning line will redirect to an edit form. This method is preferred for small adjustements.

  

• From the checks view : select checks with the checkboxes and apply the same threshold value for the selected set using the toolbox buttons. This method is preferred for mass-setting.

  

Check default values can be adjusted by clicking on the check type in the dashboard warning line or in the checks view.

packages view

Usage

The OpenSVC nodeware gathers installed package list from the nodes and pushes it to the collector. Centralized package lists allow administrators to easily:

• Compare installed version of packages across a set of nodes
• Identify nodes vulnerable to bug or security problems known to certain package versions
• Identify nodes where a package is installed or not installed
• Plan upgrades

Packages database entries not refreshed for more than 24 hours will have its update date highlighted red. Information for such a package should not be trusted.

The toolbox of the node view proposes a tool to highlight package differences between selected nodes.

Screenshot

Dashboard notifications

Clicking on the number of difference pops up the details. For exemple, on the BSD cluster above, the details show the lsof and opensvc packages are installed only on the 'bsd1' node.

patches view

Usage

The OpenSVC nodeware gathers installed patches list from the nodes (Solaris) and pushes it to the collector. Centralized patches lists allow administrators to easily:

• Compare installed version of patches across a set of nodes
• Identify nodes vulnerable to bug or security problems known to certain patch levels
• Identify nodes where a patch is installed or not installed
• Plan patch compaigns

Patch database entries not refreshed for more than 24 hours will have its update date highlighted red. Information for such a patch should not be trusted.

Screenshot

Service availability

Usage

This view presents service availability data the collector collects. A chart helps assess the problems at a glance, and a table displays each unavailability time range and aggregate availability ratio on the selected period.

The begin and end points of the analyzed period can be adjusted.

Each unavailability time range can be either:

• unacknowledged : the duration of this unvailability lowers the avail%. The chart draws this state in red.
• acknowledged, accounted : the duration of this unvailability lowers the avail%. This is used to explain an undue unavailability. The chart draws this state in red.
• acknowledged, unaccounted : the duration of this unvailability does not lowers the avail%. This is used to explain an unavailability range that was either planned or due to a collection error. The chart draws this state in blue.

An acknowledged range will show the ack message, ack author and ack date when the cursor moves over the range dates.

Clicking on a date will display a state transition graph to help determine which ressource caused to outage.

Screenshot

operating system lifecycle

Usage

This view charts the operating system dispatch of all the nodes it collects. A first chart presents the dispatch per operating system family. Subsequent charts focus on each family. These charts allow administrators and managers to analyse:

• Operating system dependency
• Which operating system is gaining ground over the others
• Current status of an operating system upgrade compaign

Screenshot

Obsolescence management

The collector helps plan operating systems and hardware upgrades through constant notification of items beyond user-defined warning and alerting thresholds.

This view is accessed through the 'admin' tab.

Obsolescence setup

Operating systems and hardware are collected from the asset database. The refresh button parses the database in search of new items to add.

The count of nodes matching each different os and hardware type is displayed in the 'count' column. Clicking on the count number displays the list of nodes.

The 'Warn date' and 'Alert date' thresholds can be set by clicking on the table cell.

Obsolescence notifications

The notifications appear in the dashboard view.

Missing obsolescence information

This notification table is displayed only to users with the 'Manager' role.

It shows how many operating system and hardware type items have no warn and alert date set by the managers. Clicking on a number in the table will redirect to the obsolescence configuration view, with appropriate filters set.

Nodes above obsolescence thresholds

Alerts and warnings are in separate tables.

Operating systems and hardware are in separate tables.

Empty lists are not displayed.