User Tools

Site Tools


robust_session_management

Robust Session Management

For live synth, we have to have the ability to change patches on the fly, in a very reliable and predictable fashion. There are a few projects out there which are working towards this now, especially the KXStudio and the Non toolsets, but this writer has not (at this writing, 2014-09-09) found any of them bulletproof enough for his very demanding live stage needs. If we’re live, we need to hit a button and know that it will do what we want, period, and the system has to be tolerant of as much dense, heavy, and fast data flow and changes as possible. I'll think that some day one or more GUI toolsets will reach this level, but until then, here is what I am using:

QJackCTL and Patchage for setup, diagnostics, and testing

QJackCTL is the venerable, generally default, and most complete single tool for setup, testing, and configuration of the Jack audio system and ALSA MIDI interprocess/hardware patching too. There is also Patchage, which is now extremely good for very complex connection situations, it draws and auto-arranges for you in nicely visual 2D. When building a new OS install, setting up for new hardware, and figuring out what I haven’t got set up right, I use these as primary tools, just as in the Primer. But like most audio tools for general-purpose computing platforms, these were not designed for the live stage, they were designed for the studio, so other tools need to be in place for the time to go out and play the music.

aj-snapshot for management of Jack and ALSA MIDI connections

aj-snapshot is a command-line and background-service tool which easily manipulates as many different Jack and ALSA MIDI connection sets as desired. It is not GUI at all, so often I create a connection set using QJackCTL or Patchage, and then I use aj-snapshot to write it to a file for future use in the patch I'm working on. It runs in the background at all times to maintain the current connection set.

The DBus version of Jack2

Jack is a background process which connects audio and MIDI applications and hardware. It used to be a simple user-level application, run as needed and then turned off when not. Jack2 is a version rebuilt with multicore CPUs in mind, and the DBus version is more recent yet. I have found the DBus version to be very cleanly and smoothly controllable, especially in the script/automation fashion I need for absolutely predictable startup and solid session management. Recent versions of QJackCTL works with this very well, one just has to make sure that the QJackCTL configuration matches script settings.

Overview of Operation

The idea here is, I want it to run well without keyboard, mouse or screen at all; and if I want to be able to switch patches, I want to just plug in a keyboard and use F-keys. So at powerup, it uses Linux built-in auto-login, starts X automatically, and then runs bash script START-INITIAL. START-INITIAL calls the default patch script, which is START-SRO. After START-SRO runs, it's ready to go, if MIDI and PA are connected it will sing!

The very beginning: START-INITIAL

When this machine boots, it does auto-logon, and runs the below. Items of note:

  • x11vnc gives me VNC control the machine, so I don't have to plug in a screen for GUI access of the live machine.
  • The logs can build up rather large; deletion is prudent.
  • Before starting Jack, prepare most of its parameters. You'll notice that I'm using a firewire interface now. In order to have MIDI devices visible in Jack, I load the ALSA MIDI device as a secondary, “slave”, driver.
  • 96 kHz audio. For precise and delicate highs, this helps. But it also requires more overall horsepower, and a tighter setup overall. Some otherwise excellent software tools right now don't do 96 kHz reliably, at least not under heavy load.
  • Not using HPET. For some reason HPET is just not available on a lot of otherwise excellent motherboards, and I happen to have one. The System setting is working very well for me, this is probably a matter of experimentation on any new hardware setup.
  • The last thing it does at boot, is initiate the change to patch SRO, using script INITPATCH.
#!/bin/bash
 
echo ''
echo 'Initiating remote control environment...'
echo ''
 
nohup /home/jeb/startx11vnc.sh &
 
echo ''
echo 'Cleaning up old logs...'
echo ''
 
rm ~/.log/jack/jackdbus.log
rm ~/.log/a2j/a2j.log
rm ~/.log/lash/lash.log
 
echo ''
echo 'Starting jackd via dBus and configuring...'
echo ''
 
jack_control ds firewire
jack_control dps rate 96000
jack_control dps period 64
 
# This is additional latency in frames
jack_control dps output-latency 1
jack_control asd alsarawmidi
 
jack_control start
 
jack_control eps realtime true
jack_control eps realtime-priority 90
jack_control eps clock-source 2		# NOT using HPET, using System
jack_control eps sync true
 
/home/jeb/INITPATCH SRO

The patch initialization script, INITPATCH

Every time a patch is started, it is called by INITPATCH. Through quite a large number of hours of experimentation I have found it best to use this hand-off method. First all audio- and MIDI-related processes are terminated; then Jackd itself, by two different methods; then a clean jackd start; and then the new patch itself is started up. Every other way I have tried, runs into problems with processes from previous patches being left over into new ones, as well as jackd data being left over or not being able to initiate cleanly; and I think also, zombie processes lying around and causing problems.

#!/bin/bash
 
echo "Stop any running audio elements..."
 
/home/jeb/KILLPROCESSES
 
echo "Retarting Jackd..."
 
/home/jeb/KILLJACKDBUS
jack_control exit
jack_control start
 
/home/jeb/PATCH-$1

Terminating the Processes

Below are KILLPROCESSES and KILLJACKDBUS. Previously I have used simpler command-lines (killall and pkill) and commonly recommended bash programming methods with them, but I have found the following to work much better, more predictably, and faster. You'll notice how KILLPROCESSES has a list of processes programmed in: as my patches change, so do I need to add and subtract from the list. You'll also notice how the list ends with 'PATCH-', this is to catch any patch script runs which may have stalled somehow. I'm not certain we need that last item at this point, but I have seen enough strange things in heated moments to want it there :-)

#!/usr/bin/env python
 
#################
# KILLPROCESSES #
#############################################################
# Part of the Ponderworthy Robust Session Management system #
#############################################################
# Kills all signal generators, filters, and MIDI tools # 
########################################################
 
 
# Requires Python library 'psutil', available by name in Arch community repo
 
import psutil
 
PROCESSES = [
	'yoshimi', 
	'fluidsynth',
	'aj-snapshot',
	'calfjackhost',
	'/home/jeb/Combine',
	'/home/jeb/OctaveDown',
	'/home/jeb/Chan2Port',
	'/home/jeb/FilterNotes',
	'/home/jeb/MIDImonitor',
	'/usr/share/carla/carla',
	'PATCH-'
	]
 
while True:
	# At every loop iterate, set alldone to True
	alldone = True
 
	for proc in psutil.process_iter():
		for procname in PROCESSES:
			if procname in proc.cmdline():
				# If found an appropriate process to kill,
				# set alldone to False, will recheck at least once.
				alldone = False
				proc.kill()
	if alldone == True:
		break
#!/usr/bin/env python
 
################
# KILLJACKDBUS #
#############################################################
# Part of the Ponderworthy Robust Session Management system #
#############################################################
# Kills the jackdbus process # 
##############################
 
 
# Requires Python library 'psutil', available by name in Arch community repo
 
import psutil
 
while True:
	# At every loop iterate, set alldone to True
	alldone = True
 
	for proc in psutil.process_iter():
		if '/usr/bin/jackdbus' in proc.cmdline():
			# If found an appropriate process to kill,
			# set alldone to False, will recheck at least once.
			alldone = False
			proc.kill()
	if alldone == True:
		break

A patch script, PATCH-SRO

There are as many of these as there patches in the current rig. They all work like this:

  1. Start all relevant audio generation and filter processes needed for this patch. 'schedtool' runs them at high priority. There is much documentation suggesting that this is not necessary and possibly disindicated, but I have many hours of testing to prove otherwise to my satisfaction :-) I believe that this is because, although this is not necessary for Jackd to be excellent, it is necessary for the DSP (digital signal processing, i.e., synthesis and filtration) code involved to override background tasks standard to any operationg system.
  2. Mididings is a programming tool by which one can make a Python script to combine and multiply MIDI streams reliably (it can do a whole lot more than that as well!). This patch needs it badly, because we have one MIDI source going to three processes; without this, lost keydowns, keyups, pedaldowns, and pedalups have been observed. Mine is called “Combine”, visible further down.
  3. Copy the correct connection set file (AJSRO.xml) for this patch, onto the current connection set file which aj-snapshot uses at all times (AJRunning.xml).
  4. Start aj-snapshot, have it read its current connection set file (AJRunning.xml) and make all connections it finds.

This particular patch, SRO (if you're wondering, it stands for Supermega Rumblic Organ), combines three simultaneous Yoshimi processes, running different configurations stored in the Yoshimi storage area (/home/username/YOSHIMI), running three separate Calf filters (highly recommendable!): the twelve-band EQ, the reverb, and the multiband compressor. The multiband compressor is new to my setup, but it has proven itself very valuable since I tend to drive heavy at both low and high tonal ranges at the same time: if I use the more common single-band compression, a dense group of high notes can cause unwanted changes in the bass output, and vice versa.

Here is PATCH-SRO:

#!/bin/bash
 
echo "Start all relevant audio elements..."
 
nohup schedtool -R -p 45 -e /home/jeb/Combine 	\
	> /home/jeb/LOGS/Combine.log &
nohup schedtool -R -p 45 -e calfjackhost --client CalfSRO 	\tonal range
	eq12:SRO ! reverb:SRO ! multibandcompressor:SRO 			\
	> /home/jeb/LOGS/calfjackhost-SRO.log &
nohup schedtool -R -p 45 -e yoshimi -N YoshSRO1 -j -l /home/jeb/YOSHIMI/SROpart1.xmz  \
	> /home/jeb/LOGS/Yoshimi-SRO1.log &
nohup schedtool -R -p 45 -e yoshimi -N YoshSRO2 -j -l /home/jeb/YOSHIMI/SROpart2.xmz  \
	> /home/jeb/LOGS/Yoshimi-SRO2.log &
nohup schedtool -R -p 45 -e yoshimi -N YoshSRO3 -j -l /home/jeb/YOSHIMI/SROpart3.xmz  \
	> /home/jeb/LOGS/Yoshimi-SRO3.log &
 
echo "And lastly, create jackd connections using aj-snapshot..."
 
cp /home/jeb/AJSRO.xml /home/jeb/AJRunning.xml
nohup schedtool -R -p 90 -e aj-snapshot -d AJRunning.xml &

And here is Combine. Mididings does have to be installed, it is available in many distros.

#!/usr/bin/python
 
from mididings import *
 
config(
    backend='jack',
    client_name='Combine',
    in_ports=1,
    out_ports=1,
)
 
run(Port(1))

Control in LXDE

Solid controllability is always a dire need on the live stage. One simple method is to be to assign function keys to run patch scripts using window manager keymappings. I often use LXDE as my window manager for synth because it places very little burden on the system, and yet gives me all the GUI I want to get things done; and happily, LXDE uses a very straightforward XML file for keyboard configuration among other things.

To set a new keystroke to run START-SRO, I do as follows:

  1. Edit the appropriate file:
    /home/username/.config/openbox/lxde-rc.xml
  2. Search for the end of the keyboard setup area, which is marked by:
    </keyboard>
  3. Add a keystroke item just before the end of the keyboard area, a la the below, which assigns Ctrl-F12 to run the patch file START-SRO.
        <keybind key="C-F12">
          <action name="Execute">
              <command>/home/username/START-SRO</command>
          </action>
        </keybind>
  4. Log out and back in, or reboot.

Postscript

GUI is great – the Calf filters have truly marvelous GUIs which help users understand what their settings are doing to the signals! – but reliability and durability is paramount, when one has a few seconds at most to change from one very complex patch to another, over and over again. Some have suggested that I use a completely GUI-less setup in production…but if I did this it would be far harder for me to change my patches and invent new ones. So this hybrid approach will do very nicely, unless and until some really amazing development in the GUI world is done. I wish to express my most profuse gratitude and thanks, to the many many highly gifted folks who have made all of this possible.

robust_session_management.txt · Last modified: 2014/09/15 21:50 by jeb