Inkscope V1.3: migration from V1.2

edit: 18/01/2016 Inkscope 1.3.1 version  

One of the new features is the access control of Inkscope.

On the Inkscope server, this needs a new python module that you can install with:

    pip install flask-login

Then you can upgrade Inkscope packages with:

    apt-get update
    apt-get upgrade

You'll also have to upgrade all the servers where cephprobes and sysprobes are installed.

For the access control features, a new database (inkscope) is created in mongo at the first launch with two users:
- one with the admin role (User 'admin' with password 'admin')
- a second with supervizor role (User 'guest', no password)
Fill free to add your own admin user and delete the default one.

If your mongoDB is authenticated, you'll have to connect to mongoDB as admin and grant access for your mongo inkscope user to an 'inkscope' database.

For example:
    mongo -u <admin> -p <adminpwd> admin
    >use inkscope
    >db.system.users.insert(<Your_user_description>)
<Your_user_description> can be found with the following:
    >use <ceph fsid>
    >db.system.users.find()

Then restart probes and then apache

That's it !

Inkscope V1.3 installation on debian servers

edit: 18/01/2016 Inkscope 1.3.1 version 

 Typical installation of Inkscope

Inkscope and its mongoDB database should be installed on an independent server from ceph.
Inkscope provides probes to fetch metrics from the ceph cluster and its machines:

• Sysprobe is used to fetch system informations on each machine of the cluster.
• Cephprobe fetches information on the cluster itself. It is intended to be installed on a machine accessing the ceph-rest-api.

Before installing Inkscope, you'll need to install a mongoDB instance.
The initialisation of the Mongo databases will be done at first launch.

Add the Inkscope debian repository in your source list

Create /etc/apt/sources.list.d/inkscope.list:

echo "deb https://raw.githubusercontent.com/inkscope/inkscope-packaging/master/DEBS ./" | sudo tee /etc/apt/sources.list.d/inkscope.list

Then

sudo apt-get update 

Packages for Inkscope are:

• inkscope-common : install the configuration file for probes and web interface.
  inkscope-sysprobe : installs sysprobe
• inkscope-cephrestapi: installs all script to start a ceph-rest-api 
• inkscope-cephprobe: installs the cephprobe
• inkscope-admviz : installs inskcope web interface 

Install Inkscope server on one server :

sudo apt-get install inkscope-admviz inkscope-cephrestapi

Edit /opt/inkscope/etc/inkscope.conf  for mongoDB, ceph-rest-api and radosgw parameters

In this file, you can also specify a description for your platform in the 'platform' field. This string will be shown on every page of Inkscope in the header.

The configuration file is the same for every module and can be deployed on every machine.

If the Inkscope server is not a ceph node, you'll have to copy the ceph.conf file of the cluster.

Edit /etc/apache2/sites-available/inkScope.conf to fill parameters concerning ceph-rest-api

By default, conf files are provided with ceph-rest-api as a WSGI application
This make ceph-rest-api a part of the inkscope server by launching ceph-rest-api as an apache wsgi application. You can also use the ceph-rest-api as a standalone server. For more details : see  "Inkscope and ceph-rest-api"
In the following, we consider ceph-rest-api as a WSGI application.

You need to create a ceph user named client.restapi with caps : [mds] allow, [mon] allow * , [osd] allow *

sudo ceph auth get-or-create client.restapi mds 'allow' osd 'allow *' mon 'allow *' > /etc/ceph/ceph.client.restapi.keyring

Then you need to add the following section in your \<cluster\>.conf:

[client.restapi]
    log_file = /dev/null
    keyring = /etc/ceph/ceph.client.restapi.keyring

Edit /etc/apache2/ports.conf to add

Listen 8080 

Enable apache proxy or rewrite module according to your choice of access to ceph-rest-api (see "Inkscope and ceph-rest-api")

sudo a2enmod proxy_http  (for standalone server)

sudo a2enmod rewrite (for WSGI application)

Inkscope needs other python modules. To install them, use:

        pip install flask-login simple-json


Enable inkscope

sudo a2ensite inkScope 

Restart apache

sudo service apache2 restart 

 Now you can access inkscope in your browser

http://<inkscope-server>:8080/

On the first launch, two users are created :
- one with the admin role (User 'admin' with password 'admin')
- a second with supervizor role (User 'guest', no password)
Fill free to add your own admin user and delete the default one.

At this stage, Inkscope is not fully operationnal and needs the mongoDB database to be feed. For this, you'll have to install Inkscope probes.

Install sysprobe on each server:

sudo apt-get install inkscope-sysprobe 

Edit /opt/inkscope/etc/inkscope.conf  for mongoDB, ceph-rest-api and radosgw parameters or retrieve the one modified on the previous step.

You'll have to install lshw and sysstat

        apt-get install lshw sysstat

Also install psutil python module (don't use psutil 0.x or 3.x)

        pip install psutil==2.1.3

Start sysprobe:

/etc/init.d/sysprobe start

Install cephprobe on one server:

sudo apt-get install inkscope-cephprobe 

Edit /opt/inkscope/etc/inkscope.conf  for mongoDB, ceph-rest-api and radosgw parameters or retrieve the one modified on the previous step.

Install psutil python module (don't use psutil 0.x or 3.x)

        pip install psutil==2.1.3

Start cephprobe:

/etc/init.d/cephprobe start

If your cluster is healthy, you'll have a first page like this:

Et voila!

Inkscope V1.3 is available

A new release of Inkscope is available on Github. This release is tested on Firefly and Hammer.

Installation from scratch is described here.
Migration from V1.2 is described here. 


The main changes are the Access control to Inkscope with different roles

New management page for Inkscope users

and the management of Rados Block Devices. 

New management page for RBD

The complete list of changes is:

Enhancements:

• add Inkscope access control and roles
• add Inkscope user management page
• add Block Device Images management
• add about inkscope page
• enhance osd info and selection regarding relative used space
• enhance pool management errors
• add view inkscope config page
• add possibility to reweight osd in OSD status view
• add osd info in osd perf page
• enhance S3 bucket info
• enhance S3 user info
• add new pools graph in pool management
• add details on selected pool and relevant rule
• add pool distribution on osd map
• allow search in osd maps
• add the possibility to add a platform name on every screen just have to add "platform":"whatyouwant", in inkscope.conf
• add status history graph for OSD

Fixes:

• fix issue when deleting cache tiering concerned pools
• fix issue when deleting Swift key
• fix issue on pools list
• fix issues when inkscope is behind a firewall or proxy
• fix stat_cat_sum missing in 0.94 and later (#53)
• fix pb on logrotate
• fix sort and search on erasure code table (#51)
• fix osd map issues when some informations are missing

Inkscope and ceph-rest-api (edited)

Edited 2015/04/13: found a better way for apache configuration (proxypass vs rewrite)

Inkscope is essentially based on the use of the ceph-rest-api.

There are two ways to access the ceph-rest-api :

• as a standalone server.
• as an Apache WSGI application. 

ceph-rest-api standalone server

You can start a standalone instance form the command line:

nohup ceph-rest-api -i admin&

Inkscope V1.2 is available

Release 1.2 has been published and packaged. It comes with the following changes.
 
Enhancements:

• add OSD perf view
• add pg stuck list and pg menu
• add pool snapshot creation and deletion
• add MDS page
• enable more than one cephprobe
• add status to sysprobe and cephprobe
• add filters to Pool/PG/OSD display
• update js libraries
• allow use of https connexion to radosgw
• add the possibility to pause/resume the automatic refresh of pg states pie on dashboard

Fixes:

• fixes to probes (authentication with mongo, stat garbage time window)
• fixes to S3 object view issues ( #37 , #38 )
• fix dashboard if great number of PGs (#44)

Talking about Inkscope at Ceph day San Francisco 2015

Find a comparison of administration GUIs for ceph :

Ceph Day SF 2015 - SysAdmin's Toolbox: Tools for Running Ceph in Production from Ceph Community

Inkscope installation on debian servers

Typical installation of Inkscope

Inkscope  and its mongoDB database should be installed on an independent server from ceph.
Inkscope provides probes to fetch metrics from the ceph cluster and its machines:

• Sysprobe is used to fetch system informations on each machine of the cluster.
• Cephprobe fetches information on the cluster itself. It is intended to be installed on a machine accessing the ceph-rest-api.

Before installing Inkscope, you'll need to install a mongoDB database.

Add the Inkscope debian repository in your source list

Create /etc/apt/sources.list.d/inkscope.list and add line:

deb https://raw.githubusercontent.com/inkscope/inkscope-packaging/master/DEBS ./ 

Then

apt-get update 

Packages for Inkscope are:

• inkscope-common : install the configuration file for probes and web interface.
  inkscope-sysprobe : installs sysprobe
• inkscope-cephrestapi: installs all script to start a ceph-rest-api 
• inkscope-cephprobe: installs the cephprobe
• inkscope-admviz : installs inskcope web interface 

Install Inkscope server on one server :

sudo apt-get install inkscope-admviz 

Then you'll have to provide a running ceph-rest-api. "Inkscope and ceph-rest-api" describes how to provide access to the ceph-rest-api as an Apache WSGI application or a standalone server.

Edit /opt/inkscope/etc/inkscope.conf  for mongoDB, ceph-rest-api and radosgw parameters.

The configuration file is the same for every module and can be deployed on every machine.

Edit /etc/apache2/sites-available/inkScope.conf to fill parameters concerning ceph-rest-api
Edit /etc/apache2/ports.conf to add

Listen 8080 

Enable apache proxy or rewrite module according to your choice of access to ceph-rest-api (see "Inkscope and ceph-rest-api")

sudo a2enmod proxy_http  (for standalone server)

sudo a2enmod rewrite (for WSGI application)

Enable inkscope

sudo a2ensite inkscope 

Restart apache

sudo service apache2 restart 

 Now you can access inkscope in your browser

http://<inkscope-server>:8080/

At this stage, Inkscope is not fully operationnal and needs the mongoDB database to be feed. For this, you'll have to install Inkscope probes.

Install sysprobe on each server:

sudo apt-get install inkscope-sysprobe 

Edit /opt/inkscope/etc/inkscope.conf  for mongoDB, ceph-rest-api and radosgw parameters or retrieve the one modified on the previous step.

Start sysprobe:

/etc/init.d/sysprobe start

Install cephprobe on one server:

sudo apt-get install inkscope-cephprobe 

Edit /opt/inkscope/etc/inkscope.conf  for mongoDB, ceph-rest-api and radosgw parameters or retrieve the one modified on the previous step.

Start cephprobe:

/etc/init.d/cephprobe start

If your cluster is healthy, you'll have a first page like this:

Et voila!

Packages are now available for inkscope

inspired by Crystal Project Icons

We are proud to announce the availability of installation packages for Inkscope.

You'll find RPM or DEB files on the project repository on Github.

An article describes how to install Inkscope on Debian servers.

inkscope lite

Inkscope lite is a visualization only version of Inkscope.

All you need for it is Apache and a running ceph-rest-api. You don't need any database or probe.

The installation proccess is very simple:
1- download and unzip the inkscope-lite package (link)
2- start a ceph-rest-api
2- configure Apache
4- Enjoy!

For more detail on InkScope lite , go there .

Download inkscope-lite

Inkscope V1.1

A new release is available on github.

For packaging reason, we go directly from 0.4 to 1.1.
Here are the main changes in this release:

• unique config file for inkscopeCtrl and inkscope probes
• pool management enhancement (categorized parameters, erasure coded pool, cache tiering)
• erasure code profile management
• Rados Gateway object visualization
• fix compatibility issue with Giant
• show overall status on every pages (title color)
• allow to use the same mongo db for several ceph clusters
• access to health details with filter from the dashboard
• new colors for pg states
• allow to launch ceph-rest-api as an apache wsgi application

It has been tested on Firefly and Giant.

Manual installation is described here

Installation packages (rpm or deb) will be available soon.

Managing Crush map buckets with Ceph Rest API

Possibilities for managing buckets are:

• add-bucket
• link
• move
• rm or remove

The URL syntax is :

/api/v0.1/link?name=bucketname&args=buckettype=bucketname [args=buckettype=bucketname]

exemple:

R1- /api/v0.1/add-bucket?name=myhost&type=host

R2- /api/v0.1/link?name=myhost&args=datacenter=houston&args=root=default

R1 will create 'myhost' of type 'host' but with no link
R2 will create or establish a tree for 'myhost' in crushmap

root -> datacenter ->host
default -> Houston -> myhost