API

The top level pandablocks module contains a number of packages that can be used from code:

• pandablocks.commands: The control commands that can be sent to a PandA

• pandablocks.responses: The control and data responses that will be received

• pandablocks.connections: Control and data connections that implements the parsing logic

• pandablocks.asyncio: An asyncio client that uses the control and data connections

• pandablocks.blocking: A blocking client that uses the control and data connections

• pandablocks.hdf: Some helpers for efficiently writing data responses to disk

Commands

There is a Command subclass for every sort of command that can be sent to the ControlConnection of a PandA. Many common actions can be accomplished with a simple Get or Put, but some convenience commands like GetBlockNumbers, GetFields, etc. are provided that parse output into specific classes.

class pandablocks.commands.Command

Abstract baseclass for all ControlConnection commands to be inherited from

exception pandablocks.commands.CommandException

Raised if a Command receives a mal-formed response

class pandablocks.commands.Raw(inp: List[str])

Send a raw command

Parameters

inp – The input lines to send

For example:

Raw(["PCAP.ACTIVE?"]) -> ["OK =1"]
Raw(["SEQ1.TABLE>", "1", "1", "0", "0", ""]) -> ["OK"]
Raw(["SEQ1.TABLE?"]) -> ["!1", "!1", "!0", "!0", "."])

class pandablocks.commands.Get(field: str)

Get the value of a field or star command.

Parameters

field – The field, attribute, or star command to get

For example:

Get("PCAP.ACTIVE") -> "1"
Get("SEQ1.TABLE") -> ["1048576", "0", "1000", "1000"]
Get("*IDN") -> "PandA 1.1..."

class pandablocks.commands.Put(field: str, value: Union[str, List[str]] = '')

Put the value of a field.

Parameters

• field – The field, attribute, or star command to put

• value – The value, possibly multiline, to put

For example:

Put("PCAP.TRIG", "PULSE1.OUT")
Put("SEQ1.TABLE", ["1048576", "0", "1000", "1000"])

class pandablocks.commands.Arm

Arm PCAP for an acquisition by sending *PCAP.ARM=

class pandablocks.commands.Disarm

Disarm PCAP, stopping acquisition by sending *PCAP.DISARM=

class pandablocks.commands.GetBlockNumbers

Get the name and number of each block type in a dictionary, alphabetically ordered

For example:

GetBlockNumbers() -> {"LUT": 8, "PCAP": 1, ...}

class pandablocks.commands.GetFields(block: str)

Get the fields of a block, returning a FieldType for each one, ordered to match the definition order in the PandA

Parameters

block – The name of the block type

For example:

GetFields("LUT") -> {"INPA": FieldType("bit_mux"), ...}

class pandablocks.commands.GetPcapBitsLabels

Get the labels for the bit fields in PCAP.

For example:

GetPcapBitsLabels() -> PcapBitsLabels()

class pandablocks.commands.ChangeGroup(value)

Which group of values to ask for *CHANGES on: https://pandablocks-server.readthedocs.io/en/latest/commands.html#system-commands

ALL = ''

All the groups below

CONFIG = '.CONFIG'

Configuration settings

BITS = '.BITS'

Bits on the system bus

POSN = '.POSN'

Positions

READ = '.READ'

Polled read values

ATTR = '.ATTR'

Attributes (included capture enable flags)

TABLE = '.TABLE'

Table changes

METADATA = '.METADATA'

Table changes

class pandablocks.commands.GetChanges(group: pandablocks.commands.ChangeGroup = <ChangeGroup.ALL: ''>)

Get a Changes object showing which fields have changed since the last time this was called

Parameters

group – Restrict to a particular ChangeGroup

For example:

GetChanges() -> Changes(
    value={"PCAP.TRIG": "PULSE1.OUT"},
    no_value=["SEQ1.TABLE"],
    in_error=["BAD.ENUM"],
)

class pandablocks.commands.GetState

Get the state of all the fields in a PandA that should be saved as a list of raw lines that could be sent with SetState.

For example:

GetState() -> [
    "SEQ1.TABLE<B"
    "234fds0SDklnmnr"
    ""
    "PCAP.TRIG=PULSE1.OUT",
]

class pandablocks.commands.SetState(state: List[str])

Set the state of all the fields in a PandA

Parameters

state – A list of raw lines as produced by GetState

For example:

SetState([
    "SEQ1.TABLE<B"
    "234fds0SDklnmnr"
    ""
    "PCAP.TRIG=PULSE1.OUT",
])

Responses

Classes used in responses from both the ControlConnection and DataConnection of a PandA live in this package.

class pandablocks.responses.FieldType(type: str, subtype: Optional[str] = None)

Field type and subtype as exposed by TCP server: https://pandablocks-server.readthedocs.io/en/latest/fields.html#field-types

type

Field type, like “param”, “bit_out”, “pos_mux”, etc.

Type

str

subtype

Some types have subtype, like “uint”, “scalar”, “lut”, etc.

Type

Optional[str]

class pandablocks.responses.Changes(values: Dict[str, str], no_value: List[str], in_error: List[str])

The changes returned from a *CHANGES command

values: Dict[str, str]

Map field -> value for single-line values that were returned

no_value: List[str]

The fields that were present but without value

in_error: List[str]

The fields that were in error

class pandablocks.responses.EndReason(value)

The reason that a PCAP acquisition completed

OK = 'Ok'

Experiment completed by falling edge of PCAP.ENABLE`

DISARMED = 'Disarmed'

Experiment manually completed by *PCAP.DISARM= command

EARLY_DISCONNECT = 'Early disconnect'

Client disconnect detected

DATA_OVERRUN = 'Data overrun'

Client not taking data quickly or network congestion, internal buffer overflow

FRAMING_ERROR = 'Framing error'

Triggers too fast for configured data capture

DRIVER_DATA_OVERRUN = 'Driver data overrun'

Probable CPU overload on PandA, should not occur

DMA_DATA_ERROR = 'DMA data error'

Data capture too fast for memory bandwidth

class pandablocks.responses.FieldCapture(name: str, type: numpy.dtype, capture: str, scale: float = 1.0, offset: float = 0.0, units: str = '')

Information about a field that is being captured

name

Name of captured field

Type

str

type

Numpy data type of the field as transmitted

Type

numpy.dtype

capture

Value of CAPTURE field used to enable this field

Type

str

scale

Scaling factor, default 1.0

Type

float

offset

Offset, default 0.0

Type

float

units

Units string, default “”

Type

str

class pandablocks.responses.Data

Baseclass for all responses yielded by a DataConnection

class pandablocks.responses.ReadyData

Yielded once when the connection is established and ready to take data

class pandablocks.responses.StartData(fields: List[pandablocks.responses.FieldCapture], missed: int, process: str, format: str, sample_bytes: int)

Yielded when a new PCAP acquisition starts.

fields

Information about each captured field as a FieldCapture object

Type

List[pandablocks.responses.FieldCapture]

missed

Number of samples missed by late data port connection

Type

int

process

Data processing option, only “Scaled” or “Raw” are requested

Type

str

format

Data delivery formatting, only “Framed” is requested

Type

str

sample_bytes

Number of bytes in one sample

Type

int

class pandablocks.responses.FrameData(data: numpy.ndarray)

Yielded when a new data frame is flushed.

data

A numpy Structured Array

Type

numpy.ndarray

Data is structured into complete columns. Each column name is <name>.<capture> from the corresponding FieldType. Data can be accessed with these column names. For example:

# Table view with 2 captured fields
>>> fdata.data
array([(0, 10),
       (1, 11),
       (2, 12)],
      dtype=[('COUNTER1.OUT.Value', '<f8'), ('COUNTER2.OUT.Value', '<f8')])
# Row view
>>> fdata.data[0]
(0, 10)
# Column names
>>> fdata.column_names
('COUNTER1.OUT.Value', 'COUNTER2.OUT.Value')
# Column view
>>> fdata.data['COUNTER1.OUT.Value']
(0, 1, 2)

property column_names

Return all the column names

class pandablocks.responses.EndData(samples: int, reason: pandablocks.responses.EndReason)

Yielded when a PCAP acquisition ends.

samples

The total number of samples (rows) that were yielded

Type

int

reason

The EndReason for the end of acquisition

Type

pandablocks.responses.EndReason

Connections

Sans-IO connections for both the Control and Data ports of PandA TCP server.

exception pandablocks.connections.NeedMoreData

Raised if the Buffer isn’t full enough to return the requested bytes

class pandablocks.connections.Buffer

Byte storage that provides line reader and bytes reader interfaces. For example:

buf = Buffer()
buf += bytes_from_server
line = buf.read_line()  # raises NeedMoreData if no line
for line in buf:
    pass
bytes = buf.read_bytes(50)  # raises NeedMoreData if not enough bytes

read_bytes(num: int) → bytearray

Read and pop num bytes from the beginning of the buffer, raising NeedMoreData if the buffer isn’t full enough to do so

peek_bytes(num: int) → bytearray

Read but do not pop num bytes from the beginning of the buffer, raising NeedMoreData if the buffer isn’t full enough to do so

read_line()

Read and pop a newline terminated line (without terminator) from the beginning of the buffer, raising NeedMoreData if the buffer isn’t full enough to do so

class pandablocks.connections.ControlConnection

Sans-IO connection to control port of PandA TCP server, supporting a Command based interface. For example:

cc = ControlConnection()
# Connection says what bytes should be sent to execute command
to_send = cc.send(command)
socket.sendall(to_send)
while True:
    # Repeatedly process bytes from the PandA
    received = socket.recv()
    # Sending any subsequent bytes to be sent back to the PandA
    to_send = cc.receive_bytes(received)
    socket.sendall(to_send)
    # And processing the produced responses
    for command, response in cc.responses()
        do_something_with(response)

receive_bytes(received: bytes) → bytes

Tell the connection that you have received some bytes off the network. Parse these into high level responses which are yielded back by responses. Return any bytes to send back

responses() → Iterator[Tuple[pandablocks.commands.Command, Any]]

Get the (command, response) tuples generated as part of the last receive_bytes

send(command: pandablocks.commands.Command) → bytes

Tell the connection you want to send an event, and it will return some bytes to send down the network

class pandablocks.connections.DataConnection

Sans-IO connection to data port of PandA TCP server, supporting an flushable iterator interface. For example:

dc = DataConnection()
# Single connection string to send
to_send = dc.connect()
socket.sendall(to_send)
while True:
    # Repeatedly process bytes from the PandA looking for data
    received = socket.recv()
    for data in dc.receive_bytes(received):
        do_something_with(data)

receive_bytes(received: bytes, flush_every_frame=True) → Iterator[pandablocks.responses.Data]

Tell the connection that you have received some bytes off the network. Parse these into Data structures and yield them back.

Parameters

• received – the bytes you received from the socket

• flush_every_frame – Whether to flush FrameData as soon as received. If False then they will only be sent if flush is called or end of acquisition reached

flush() → Iterator[pandablocks.responses.FrameData]

If there is a partial data frame, pop and yield it

connect(scaled: bool) → bytes

Return the bytes that need to be sent on connection

Asyncio Client

This is an asyncio wrapper to the ControlConnection and DataConnection objects, allowing async calls to send(command) and iterate over data().

class pandablocks.asyncio.AsyncioClient(host: str)

Asyncio implementation of a PandABlocks client. For example:

async with AsyncioClient("hostname-or-ip") as client:
    # Control and data ports are now connected
    resp1, resp2 = await asyncio.gather(client.send(cmd1), client.send(cmd2))
    resp3 = await client.send(cmd3)
    async for data in client.data():
        handle(data)
# Control and data ports are now disconnected

async connect()

Connect to the control and data ports, and be ready to handle commands

async close()

Close the control and data connections, and wait for completion

async send(command: pandablocks.commands.Command[T]) → T

Send a command to control port of the PandA, returning its response.

Parameters

command – The Command to send

data(scaled: bool = True, flush_period: Optional[float] = None, frame_timeout: Optional[float] = None) → AsyncGenerator[pandablocks.responses.Data, None]

Connect to data port and yield data frames

Parameters

• scaled – Whether to scale and average data frames, reduces throughput

• flush_period – How often to flush partial data frames, None is on every chunk of data from the server

• frame_timeout – If no data is received for this amount of time, raise asyncio.TimeoutError

Blocking Client

This is a blocking wrapper to the ControlConnection and DataConnection objects, allowing blocking calls to send(commands) and iterate over data().

class pandablocks.blocking.BlockingClient(host: str)

Blocking implementation of a PandABlocks client. For example:

with BlockingClient("hostname-or-ip") as client:
    # Control and data ports are now connected
    resp1, resp2 = client.send([cmd1, cmd2])
    resp3 = client.send(cmd3)
    for data in client.data():
        handle(data)
# Control and data ports are now disconnected

connect()

Connect to the control and data ports, and be ready to handle commands

close()

Close the control and data connections, and wait for completion

send(commands: pandablocks.commands.Command[T], timeout: int = 'None') → T

send(commands: Iterable[pandablocks.commands.Command], timeout: int = 'None') → List

Send a command to control port of the PandA, returning its response.

Parameters

• commands – If single Command, return its response. If a list of commands return a list of reponses

• timeout – If no reponse in this time, raise socket.timeout

data(scaled: bool = True, frame_timeout: Optional[int] = None) → Iterator[pandablocks.responses.Data]

Connect to data port and yield data frames

Parameters

• scaled – Whether to scale and average data frames, reduces throughput

• frame_timeout – If no data is received for this amount of time, raise socket.timeout

HDF Writing

This package contains components needed to write PCAP data to and HDF file in the most efficient way. The oneshot write_hdf_files is exposed in the commandline interface. It assembles a short Pipeline of:

AsyncioClient -> FrameProcessor -> HDFWriter

The FrameProcessor and HDFWriter run in their own threads as most of the heavy lifting is done by numpy and h5py, so running in their own threads gives multi-CPU benefits without hitting the limit of the GIL.

See also

How to use the library to capture HDF files, How fast can we write HDF files?

class pandablocks.hdf.Pipeline

Helper class that runs a pipeline consumer process in its own thread

what_to_do

Subclasses should create this dictionary with handlers for each data type, returning transformed data that should be passed downstream

stop()

Stop the processing after the current queue has been emptied

class pandablocks.hdf.HDFWriter(scheme: str)

Write an HDF file per data collection. Each field will be written in a 1D dataset /<field.name>.<field.capture>.

Parameters

scheme – Filepath scheme, where %d will be replaced with the acquisition number, starting with 1

class pandablocks.hdf.FrameProcessor

Scale field data according to the information in the StartData

pandablocks.hdf.create_pipeline(*elements: pandablocks.hdf.Pipeline) → List[pandablocks.hdf.Pipeline]

Create a pipeline of elements, wiring them and starting them before returning them

pandablocks.hdf.stop_pipeline(pipeline: List[pandablocks.hdf.Pipeline])

Stop and join each element of the pipeline

async pandablocks.hdf.write_hdf_files(client: pandablocks.asyncio.AsyncioClient, scheme: str, num: int = 1, arm: bool = False)

Connect to host PandA data port, and write num acquisitions to HDF file according to scheme

Parameters

• client – The AsyncioClient to use for communications

• scheme – Filenaming scheme for HDF files, with %d for scan number starting at 1

• num – The number of acquisitions to store in separate files

• arm – Whether to arm PCAP at the start, and after each successful acquisition