OpenSVC


	Sources


	Downloads


	Live collector


	Documentation


	Contacts


	Welcome

General

Nodeware feature matrix

Node

Nodeware installation
Role of each item installed on the node

Services

Service creation
Service management
Service configuration reference

Collector

Usage
Tabulations
Persistent filter bar
Column filter bar
Tool bar
Service tabs
Node tabs
View: svcmon
View: actions
Obsolescence management
Collector guide (all in one page)

Storage

Netapp
Symmetrix clones
Linux LV delta replication
ZFS send/receive

Cookbook

Lxc clustered services on OVH
VM encapsulation in a service
HA setup

Tabulations

Views are accessed through the tabulations in the site's pages header.

Each view has a persistent filters bar (folded under the tabs), a column filters bar (top), and a tool bar (bottom).

Persistent filters bar

The orange thick line under the tabs can be unfolded by clicking on it or through the ctrl+: or ctrl+; shortcuts. The filters set there are stored in database and are applied on all collector views. They persist across navigation sessions. Filters can apply on data not presented in the current view or embed hardcoded complex queries, like 'nodes with services'.

Upon filter selection, a tag cloud appears to display all possible values of the filter. Clicking a tag will set the filter value box. The filter value follow the same syntax rules as the column filters.

A filter/value is added by clicking on the '+' blue icon and removed by clicking on the 'x' red icon.

Column filters bar

Each column of the table supports filtering. Filters are applied upon <enter> in any filter box. Filter box expressions are and'ed. Filter syntax supports :

%: Wildcard for SQL expressions. Can be used as a header, trailer or in the middle of a pattern (%gie%01%)

&: And multiple patterns (%gie%&%adomain.com)

|: Or multiple patterns (%gie%|%adomain.com)

!: Negate the pattern (!%gie%&%adomain.com)

empty: Select only empty cells (!empty)

Service tabs

In the 'svcmon' and 'actions' views, clicking on a service name will open the service tabs to display the most comprehensive data set the collector database contains about this service.

properties

opensvc version

Pushed to database daily by the opensvc nodeware cronjob.

unackowledged errors

A count of the unacknowledged errors on this service found in the actions table.

type

Service type can be PRD, DEV, ...

comment

Pushed to database daily by the opensvc nodeware cronjob. This information resides in the service env file on the nodes.

last update

Timestamp updated upon service information receive from the nodeware cronjob.

container name

Host Name of the virtual machine encapsulated in the service, if any.

container type

Virtualisation driver to handle the encapsulated virtual machine, if any.

responsibles

List of administrators contact names for this service.

responsibles mail

List of administrators contact emails for this service.

primary node

Host name of the node where the service should be running in optimal situation.

nodes

All nodes where the service may be running in degraded situation (not in disaster recovery situation).

drp node

Host name of the node the service should be running in a disaster recovery situation.

drp nodes

All nodes where the service may be running in disaster recovery situation.

status

Display the per-node synthetic service status.

resources

Display the per-node per-resource detailled service status.

env

Display the raw service configuration file pushed by the nodeware.

topology

Sketches service-nodes-storage-sites relationships

Node tabs

In the 'svcmon' and 'actions' views, clicking on a node name will open the node tabs to display the most comprehensive data set the collector database contains about this node.

server, os, mem, cpu, location, power

Information displayed in these tabs are extracted from the asset table. Assets can be loaded from a tiers enterprise asset manager or filled-in manually.

stats

Generate and display graphs of performance metrics on the user-selected time range. The metrics are gathered daily from tools like glance and sar running on the nodes. The opensvc nodeware relays this information to database.

metrics

Metrics collection depend on the operating system. Some of them are:

cpu: usr, sys, wait, irq, guest, steal, idle, nice, iowait, soft

mem: commit, free, used by usr, buffer, cache, sys

swap: free, used, used but cached

load: avg1, avg5, avg15, runq size, proc list size

all-disks: io/s, bandwidth

per-disk: io/s, request size, wait time, service time, %util, bandwidth

examples

Tool bar

The tool bar hosts generic tools and view-specific tools.

Tool	Scope	Description
	generic	This pager allows you to navigate the current filtered data cursor. It displays at most 10 pages of 25 rows. Clicking 'all' will display all rows in a single page (might be hard on the collector resources). Hovering the mouse over the 'all' marker will display the data cursor statistics. Power users can change the number of rows per page by setting the GET parameter 'perpage=10'.
	generic	Export table content as a CSV file.
	svcmon	Generate a diagram of the current filtered data cursor mixing information from the service configurations, computer assets and storage assets. Example below.
	actions	Subscribe to RSS feed of changes on the current filtered data cursor.
	actions	Acknowledge failed actions whose checker box is marked as selected. Clicking on the ack icon will spawn an input box with active cursor so you can optionally enter an aknowledge comment. Comment, ack date and acker name will be displayed on hovering the mouse over the 'err' status markers.

svcmon view

Usage

svcmon stands for service monitor. This is the view presenting the services health.

Main usage:

Quick insight of live service status.
Example: primary node & responsible=me & status!=ok

List services matching criteria.
Example, 'How many non-production services do I manage for the ERP system':
primary node & app=ERP & type=!PRD
Example, 'I lost a power line, which services are impacted':
nb_power_supply=1 & power_supply_1=POW_A

Generate diagrams of clusters to better communicate on disaster recovery plans and allocated resources.

Trigger actions on services or group of services

Screenshot

Fields

Container type

The container type reflect the kind of service integration selected for the service. This setting is set in the service configuration file on the nodes as the mode parameter. The possible values are, as of January 2010 :

hosted	The service is hosted on bare metal, non-virtualized resources : ip plumbed on a physical interface, applications and data in filesystems mounted on a real server.
zone	The service applications and data are deployed in a Solaris zone. The ip is still plumbed in the global zone. Filesystem resources are mounted in the global zone.
lxc	The service applications and data are deployed in a Linux Container (zone-like). The ip is plumbed on a virtual interface, with virtualized mac adress. Filesystem resources are mounted in the host system.
vm	The service applications and data are deployed in a Xen or KVM virtual machine. The ip is plumbed on a virtual interface, with virtualized mac adress. Filesystem resources are mounted in the virtual host.
hpvm	The service applications and data are deployed in a HP-UX native virtual machine. The ip is plumbed on a virtual interface, with virtualized mac adress. Filesystem resources are mounted in the virtual host.

Responsible

The responsibles are persons receiving alerts for a service. Hovering the mouse over the icon spawns the name of the responsibles. No icon means no responsible, which is an anomaly, and as such, cause alerts to be emitted to the site's administrator/manager.

Prefered node

Upon node reboot, only the services whose prefered node is set to this node are started. During those services start-up, the nodeware verifies the service is not running on another node and aborts the start-up if so.

Service type

This property is used by the nodeware to sanity check actions asked by the admin.

PRD	Production service. Services of this type can be started on any type of node.
DEV	Development service. Services of this type are not allowed to be started on production nodes.
TMP	Temporary service. Like DEV, but alerts won't be sent for these services.

App

The application code is a way to group services dedicated/paid by some corporate entity or project. You can setup any application code you want in your services configuration file, using the app parameter.

Status

Status gizmos present the overall service health on top, and individual resource type status below. Color codes are:

green	resource is up
red	resource is down
orange	resource has warnings
grey	resource type is not used by the service

An all-greyed-out status marker indicates the presented data is too old to be trusted, though it displays the last overall known status. Hovering the mouse over the status markers will displayed the data refresh timestamp.

actions view

Usage

This view displays the journal of all services events with their status.

Main usage:

Quick insight of unacknowledged errors affecting a specific perimeter.
Example: errors not acknowledged & responsible=me

Retrospective problem analysis.
Example: service=some_service & begin>2010-01-10% & end<2010-01-12%

Problem patterns.
Example: service=some_service & action=syncnodes & status!=ok

Learn which system commands are run by the nodeware
Example: log=%sg_persist%|%scu%

Trace who was alerted of a problem, and when

Screenshot

Fields

Service

Service name the action applied to. Private collectors usually report shortnames, shared collectors usually report service names with domain names.

App

Responsible

Node

Node where the action has be executed. The node name is a link to the asset view.

Action

The executed action name. An action usually aggregates a number of log lines, plus a line with no log message as a header. The default action view has an 'empty' log filter active so you are presented only actions without their logs. You can drill down a specific action by clicking on its pid.

Action	Description
start	start resources of type : ip, loop, disk group, zpool, fs, container, app
stop	stop resources of type : app, container, fs, zpool, disk group, loop, ip
diskstart	start resources of type : loop, disk group, zpool, fs
diskstop	stop resources of type : fs, zpool, disk group, loop
startip	start resources of type : ip
stopip	stop resources of type : ip
startloop	start resources of type : loop
stoploop	stop resources of type : loop
startvg	start resources of type : disk group
stopvg	stop resources of type : disk group
mount	start resources of type : fs
umount	stop resources of type : fs
prstart	acquire scsi persistent reservations on disks of the service (wrapped by startvg and diskstart)
prstop	release scsi persistent reservations on disks of the service (wrapped by stopvg and diskstop)
syncnodes	trigger hard-coded and user-defined file synchronization to secondary nodes. Optionally creates snapshots to send a coherent file set. No-op if run from a node not running the service.
syncdrp	trigger hard-coded and user-defined file synchronization to disaster recovery nodes. Optionally creates snapshots to send a coherent file set. No-op if run from a node not running the service.
print_status	print status of all service resources

Status

ok	The action completed succesfully.
warn	The action completed with some warnings. No acknowledgement needed.
err	The action completed with some error. Investigation and acknowledgement are needed.
err	The action completed with some error. Acknowledged by a user.
empty	The collector has been informed of the action begining but has no yet received ending logs.

Begin

Begin timestamp of the action.

End

End timestamp of the action.

Pid

Process identifier of the session handling the action on the node. Click to active a pid filter with this value.

Log

The action log as it is displayed on the node standart output.

Obsolescence management

The collector helps plan operating systems and hardware upgrades through constant notification of items beyond user-defined warning and alerting thresholds.

This view is accessed through the 'admin' tab.

Obsolescence setup

Operating systems and hardware are collected from the asset database. The refresh button parses the database in search of new items to add.

The count of nodes matching each different os and hardware type is displayed in the 'count' column. Clicking on the count number displays the list of nodes.

The 'Warn date' and 'Alert date' thresholds can be set by clicking on the table cell.

Obsolescence notifications

The notifications appear in the dashboard view.

Missing obsolescence information

This notification table is displayed only to users with the 'Manager' role.

It shows how many operating system and hardware type items have no warn and alert date set by the managers. Clicking on a number in the table will redirect to the obsolescence configuration view, with appropriate filters set.

Nodes above obsolescence thresholds

Alerts and warnings are in separate tables.

Operating systems and hardware are in separate tables.

Empty lists are not displayed.