Environment storage backends

The macro environment must survive Macro Server restarts: variables like the active measurement group or the scan file configuration are expected to still be there after a server upgrade or a crash. Where and how this persistence happens is a deployment decision, so since Sardana 3.7.0 the environment storage is designed as a set of interchangeable backends behind a single manager.

A modular manager

The EnvironmentManager implements all the environment logic that is independent of storage: parsing of values, the scoping rules (a lookup cascades from the door+macro scope, through the macro and door scopes, down to the global scope) and the notification of clients. The actual persistence is delegated to a backend object with a minimal get/set/unset/list interface. Three backends are provided:

ShelveEnvironmentBackend — the default, file based storage,
RedisEnvironmentBackend — storage in a Redis server,
TangoDBEnvironmentBackend — storage in the Tango database.

This refactoring kept the public API unchanged: macros and clients use the same environment commands regardless of the configured backend.

Backend selection

The backend is selected through the EnvironmentDb Tango device property of the MacroServer device, whose value is interpreted as a URI or path:

property — Tango database backend, storing the environment as properties of the MacroServer device itself; property:<name> stores them under an arbitrary object name instead,
redis://... or rediss://... — Redis backend, with keys prefixed by the MacroServer name,
anything else is treated as a filesystem path for the shelve backend (default: /tmp/tango-<user>/<ds_exec_name>/<ds_inst_name>/macroserver.properties).

If the configured backend cannot be initialized at server startup, the MacroServer falls back to a freshly created temporary shelve file (and rewrites its EnvironmentDb property accordingly) — except for the Redis backend, where a connection failure aborts the server startup instead of silently switching storage.

Trade-offs

Shelve (default)

The shelve backend uses the Python shelve module on top of a dbm database (GNU dbm when available, with a fallback to the pure Python dumb dbm; the choice can be forced with the MS_ENV_SHELVE_BACKEND sardana custom setting). It needs no external service, which makes it the zero-configuration default. The flip side is that the environment becomes a set of files on the host running the MacroServer: it is not shared, the exact on-disk format depends on which dbm implementation happened to be available, and the storage relies on file-system locking semantics, which makes hosting it on network file systems (e.g. NFS-mounted directories) fragile. Historically the dumb dbm fallback was also a source of corrupted/empty environment files on unclean shutdowns.

Redis

The Redis backend stores each variable as a separate key (prefixed with the MacroServer name) in an external Redis server. This decouples the environment from the MacroServer host: it can be inspected and backed up with standard Redis tooling and survives machine reinstallation. The cost is an external service to deploy and operate, an extra Python dependency (redis-py), and a hard runtime dependency: the connection is verified at startup and the MacroServer refuses to start if the Redis server is not reachable.

Tango database

The TangoDB backend stores the environment as free object properties in the Tango database, encoding the door/macro scope in the property name. Its appeal is that a Tango database is by definition already present in any Sardana installation, so, like shelve, it requires no additional service — while still being centralized and shared. On the other hand every environment access becomes a Tango database operation and the values must be encoded into database-friendly strings.