Sardana development guidelines

Overview

This document describes Sardana from the perspective of developers. Most importantly, it gives information for people who want to contribute code to the development of Sardana. So if you want to help out, read on!

How to contribute to Sardana

Sardana development is managed with the Sardana GitLab project.

Apart from directly contributing code, you can contribute to Sardana in many ways, such as reporting bugs or proposing new features. In all cases you will probably need a GitLab.com account and you are strongly encouraged to subscribe to the sardana-devel and sardana-users mailing lists.

We also invite you to join the regular project follow-up meetings. These are announced on the previously mentioned mailing list.

Detailed instructions on how to contribute to code or documentation can be found in CONTRIBUTING.md.

General design

In general, sardana was designed for using it with client-server architecture. Where all the business logic resides on the server side and client side is just a thin layer to enable control and monitoring.

One of the fundamental principle in sardana design is its clear separation between the core and the middleware. The sardana kernel is composed from objects of the core classes which are then exposed to the clients via server extensions.

Sardana kernel objects communicates using publisher-subscriber pattern in which messages are exchanged by means of synchronous invocation of callbacks. A class inheriting from EventGenerator acts as publisher and a class inheriting from EventReceiver as subscriber e.g. PoolMotor and PoolPseudoMotor respectively or Position and PoolMotor respectively. Moreover this mechanism is used for notifying server extension about sardana kernel events e.g. element’s state change.

Major core classes have state attribute which value is one of the State enum values.

Sardana uses threads for concurrency. More precise using Thread pool pattern implemented in ThreadPool and different instances of such thread pools exist within a kernel.

Important

Due to the use of Tango sardana ThreadPool instances must use special worker threads which execute jobs within EnsureOmniThread context manager.

Logging infrastructure is initialized on server startup using prepare_logging(). Most of the Sardana classes inherit from Logger and you can directly invoke their logging methods e.g. debug(), error(), etc.. For consistency we recommend to use Taurus logging utilities - taurus.core.util.log.

Software layers

The main software layers of sardana are:

../_images/software_layers.png

Main software layers of sardana on example of Device Pool (including controller and axis element)

In continuation we point you to more detailed information about each of them.

Client

In the wide range of built-in sardana clients we include:

All of them underneath use Taurus library and one of these model extensions:

Sardana background activities e.g. motion and acquisition actions or macro execution send Tango event notifications about the state changes. Clients then synchronize with these events using AttributeEventWait class.

Spock

Spock is basically an IPython extension.

It implements the load_ipython_extension() hook function where it exposes some variables and magic commands to the IPython namespace on startup.

There are two different groups of variables and magics: the built-in ones and the custom ones.

The built-in ones are exposed directly by the load_ipython_extension() function and are common to all Sardana systems and Spock sessions e.g. www and edmac magics or MACRO_SERVER and DOOR variables.

The custom ones are exposed by the MacroServer’s elements attribute listener as explained in the next paragraph.

Spock implements its own Taurus extensions for MacroServer and Door devices in SpockMacroServer and SpockDoor classes. This extenstion enhances the standard Taurus extensions for Sardana devices in the following way. In case of the MacroServer, when the SpockMacroServer receives the elements attribute event it exposes the current Device Pool elements as variables e.g. mot01, mntgrp01, and macros as magics e.g. sar_demo, mv, ct. On Spock startup, when the SpockMacroServer object is created, it will expose at once all variables corresponding to all elements proceding from all Device Pools the MacroServer is connected to and magics corresponding to all macros present in the MacroServer. In case of the Door, when the macro magics are executed the SpockDoor extensions executes them in synchronous way i.e. holds the Spock prompt until the macro has finished, and handles KeyboardInterrupt exceptions (Ctrl+C) to interrupt the macro execution.

Server extension

At the time of writing of this documentation, the only one production ready server extension is implemented using Tango, hence the remaining part of this guideline will refer to it. Nevertheless it was proofed that sardana could be used with a different middleware e.g. sardana-jupyter project shows how to run MacroServer within JupyterLab.

Server extension is a thin layer which principal responsibilities are:

  • instantiation of core classes based on the configuration and keeping their references

  • redirecting requests to kernel objects

  • propagating events from the kernel objects

Tango attributes/ command interface does not accept certain python types e.g. dict. For these cases we recommend using DevEncoded or DevString types (the later one when you need a memorized attribute). For coding/ decoding of data we recommend using Taurus codec utilities: CodecFactory.

Apart from using Tango as middleware, sardana uses some of Tango features:

  • serialization monitor

  • limits protection (partially, a part of it is re-implemented in the core)

  • attribute configuration

  • state machine (partially, a part of it is re-implemented in the core)

  • Tango DB:

    • for storing persistent configuration (but MacroServer environment)

    • naming service / broker

Note

SEP20 propose to use YAML configuration files for storing persistent configuration

Sardana kernel

Within sardana kernel resides objects of the sardana core classes. There are two principal objects:

  • MacroServer - macro execution engine

  • Device Pool - repository of hardware elements

Pool

Device Pool part of the kernel consist of one Pool object which acts as:

Main categories of objects managed by Device Pool:

classDiagram PoolObject <|-- PoolBaseElement PoolBaseElement <|-- PoolBaseController Controller <|-- MotorController MotorController <|-- MyCustomController PoolBaseController <|-- PoolController PoolBaseController "1" *-- Controller PoolController <|-- PoolPseudoMotorController PoolBaseController "0..*" *-- PoolElement PoolBaseElement <|-- PoolElement PoolElement <|-- PoolMotor PoolBaseGroup <|-- PoolPseudoMotor PoolBaseGroup "1..*" o-- PoolElement PoolElement <|-- PoolPseudoMotor PoolBaseGroup <|-- PoolGroupElement PoolBaseElement <|-- PoolGroupElement PoolGroupElement <|-- PoolMotorGroup cssClass "Controller,MotorController,MyCustomController" pluginNode cssClass "PoolObject,PoolBaseElement,PoolBaseController" coreNode cssClass "PoolController,PoolPseudoMotorController" coreNode cssClass "PoolBaseElement,PoolElement,PoolMotor" coreNode cssClass "PoolPseudoMotor,PoolBaseGroup,PoolGroupElement" coreNode cssClass "PoolMotorGroup" coreNode

Class diagram of motion related main classes

Note

The objects currently under Pool management are communicated to the clients with the Elements attribute.

state of pseudo axis elements or groups is composed from state’s of their associated elements e.g.

  • group state turns Moving when one of its associated elements reports Moving

  • group state turns On when all of its associated elements reports On

State changes are notified with the publisher-subscriber implementation.

Motion and acquisition are handled with PoolMotion and PoolAcquisition respectively (the latter one is composed from specific sub-actions). Each axis elements and pseudo axis elements aggregates one action instance for exclusive use during an independent motion or acquisition. Groups aggregate a different action instance for grouped motion or acquisition which involves all elements associated to the group.

MacroServer

MacroServer part of the kernel consist of one MacroServer object which acts as:

  • macro parameter types managed by TypeManager

  • macros managed by MacroManager

  • objects managed by Pool instances it is connected to (can be multiple of them)

The objects currently under MacroServer management are communicated to the clients with the Elements attribute.

Door is just a thin layer in macro execution process where the principal role plays MacroExecutor - one instance per Door.

Macros are executed asynchronously using threads by one ThreadPool with just one worker thread per MacroExecutor.

Note

sardana-jupyter executes macros synchronously.

Macro execution consists of:

  • user inputs macro parameters

  • composing of XML document with macro execution information e.g. parameters parsing in Spock

  • execution of RunMacro() command on Door Tango Device

  • parameters decoding and validation by MacroManager and ParamDecoder

  • creation of macro object from its meta equivalent MacroManager

  • macro execution using MacroExecutor

Macro execution can be stopped, aborted or released, and the following sketch demonstrate different activity flows depending on where the macro was interrupted:

../_images/macro_interruption_flow.jpeg

Activity diagram showing different execution flows of interrupting a macro

While Device Pool controllers are long-lived objects, recorders and macros are short-lived objects created on demand and destroyed when not needed anymore.

../_images/macros_recorders_software_layers.png

Main software layers of sardana on example of MacroServer macros and recorders

Plugins

In sardana we distinguish the following plugin types:

  • macros

  • recorders

  • controllers

Manager classes e.g. ControllerManager or MacroManager are mainly responsible for:

  • discovery and reloading

  • act as container of plugins

  • factory of plugin instances

Plugins are represented by meta classes e.g. ControllerClass or MacroClass.

Sardana plugins discovery is based on directory scanning and python modules introspection. The scanning process looks for classes inheriting from a certain base class e.g. MotorController or specially decorated functions e.g. macro and if found they are loaded into the kernel and available for instantiation. Plugins discovery takes place on the server startup and can be executed on user request at runtime.

Sardana comes with a catalogue of built-in plugins and allows for overriding of plugins based on the name. By means of configuration you define which directories and in which order (relevant for the overriding mechanism) will be scanned.

Note

SEP19 proposes to add a new way of registering plugins based on setuptools Entry Points.