API

The top level pandablocks module contains a number of packages that can be used from code:

• pandablocks.commands: The control commands that can be sent to a PandA

• pandablocks.responses: The control and data responses that will be received

• pandablocks.connections: Control and data connections that implements the parsing logic

• pandablocks.asyncio: An asyncio client that uses the control and data connections

• pandablocks.blocking: A blocking client that uses the control and data connections

• pandablocks.hdf: Some helpers for efficiently writing data responses to disk

Commands

There is a Command subclass for every sort of command that can be sent to the ControlConnection of a PandA. Many common actions can be accomplished with a simple Get or Put, but some convenience commands like GetBlockNumbers, GetFields, etc. are provided that parse output into specific classes.

exception pandablocks.commands.CommandException

Raised if a Command receives a mal-formed response

class pandablocks.commands.Command

Abstract baseclass for all ControlConnection commands to be inherited from

lines() → Union[bytes, List[bytes]]

Return lines that should be sent to the PandA, with no newlines in them

response(lines: Union[bytes, List[bytes]]) → T

Create a response from the lines received from the PandA

ok_if(ok, lines: Union[bytes, List[bytes]])

If not ok then raise a suitable CommandException

class pandablocks.commands.Get(field: str)

Get the value of a field or star command.

Parameters

field – The field, attribute, or star command to get

For example:

Get("PCAP.ACTIVE") -> b"1"
Get("SEQ1.TABLE") -> [b"1048576", b"0", b"1000", b"1000"]
Get("*IDN") -> b"PandA 1.1..."

response(lines: Union[bytes, List[bytes]]) → Union[bytes, List[bytes]]

The value that was requested as a byte string. If it is multiline then it will be a list of byte strings

class pandablocks.commands.Put(field: str, value: Union[str, List[str]] = '')

Put the value of a field.

Parameters

• field – The field, attribute, or star command to put

• value – The value, possibly multiline, to put

For example:

Put("PCAP.TRIG", "PULSE1.OUT")
Put("SEQ1.TABLE", ["1048576", "0", "1000", "1000"])

class pandablocks.commands.Arm

Arm PCAP for an acquisition by sending *PCAP.ARM=

class pandablocks.commands.GetBlockNumbers

Get the descriptions and field lists of the requested Blocks.

For example:

GetBlockNumbers() -> {"LUT": 8, "PCAP": 1, ...}

response(lines: Union[bytes, List[bytes]]) → Dict[str, int]

The name and number of each block type in a dictionary, alphabetically ordered

class pandablocks.commands.GetFields(block: str)

Get the fields of a block, returning a FieldType for each one.

Parameters

block – The name of the block type

For example:

GetFields("LUT") -> {"INPA": FieldType("bit_mux"), ...}

response(lines: Union[bytes, List[bytes]]) → Dict[str, pandablocks.responses.FieldType]

The name and FieldType of each field in a dictionary, ordered to match the definition order in the PandA

class pandablocks.commands.GetPcapBitsLabels

Get the labels for the bit fields in PCAP.

For example:

GetPcapBitsLabels() -> PcapBitsLabels()

class pandablocks.commands.GetChanges

Get the changes since the last time this was called.

For example:

GetChanges() -> Changes()

class pandablocks.commands.Raw(inp: List[str])

Send a raw command

Parameters

inp – The input lines to send

For example:

Raw(["PCAP.ACTIVE?"]) -> ["OK =1"]
Raw(["SEQ1.TABLE>", "1", "1", "0", "0"]) -> ["OK"]
Raw(["SEQ1.TABLE?"]) -> ["!1", "!1", "!0", "!0", "."])

response(lines: Union[bytes, List[bytes]]) → List[str]

The lines that PandA responded, including the multiline markup

Responses

Classes used in responses from both the ControlConnection and DataConnection of a PandA live in this package.

class pandablocks.responses.FieldType(type: str, subtype: Optional[str] = None)

Field type and subtype as exposed by TCP server: https://pandablocks-server.readthedocs.io/en/latest/fields.html#field-types

type

Field type, like “param”, “bit_out”, “pos_mux”, etc.

subtype

Some types have subtype, like “uint”, “scalar”, “lut”, etc.

class pandablocks.responses.EndReason(value)

The reason that a PCAP acquisition completed

OK = 'Ok'

Experiment completed by falling edge of PCAP.ENABLE`

DISARMED = 'Disarmed'

Experiment manually completed by *PCAP.DISARM= command

EARLY_DISCONNECT = 'Early disconnect'

Client disconnect detected

DATA_OVERRUN = 'Data overrun'

Client not taking data quickly or network congestion, internal buffer overflow

FRAMING_ERROR = 'Framing error'

Triggers too fast for configured data capture

DRIVER_DATA_OVERRUN = 'Driver data overrun'

Probable CPU overload on PandA, should not occur

DMA_DATA_ERROR = 'DMA data error'

Data capture too fast for memory bandwidth

class pandablocks.responses.FieldCapture(name: str, type: numpy.dtype, capture: str, scale: float = 1.0, offset: float = 0.0, units: str = '')

Information about a field that is being captured

name

Name of captured field

type

Numpy data type of the field as transmitted

capture

Value of CAPTURE field used to enable this field

scale

Scaling factor, default 1.0

offset

Offset, default 0.0

units

Units string, default “”

class pandablocks.responses.Data

Baseclass for all responses yielded by a DataConnection

class pandablocks.responses.ReadyData

Yielded once when the connection is established and ready to take data

class pandablocks.responses.StartData(fields: List[pandablocks.responses.FieldCapture], missed: int, process: str, format: str, sample_bytes: int)

Yielded when a new PCAP acquisition starts.

fields

Information about each captured field as a FieldCapture object

missed

Number of samples missed by late data port connection

process

Data processing option, only “Scaled” or “Raw” are requested

format

Data delivery formatting, only “Framed” is requested

sample_bytes

Number of bytes in one sample

class pandablocks.responses.FrameData(data: numpy.ndarray)

Yielded when a new data frame is flushed.

data

A numpy Structured Array

Data is structured into complete columns. Each column name is <name>.<capture> from the corresponding FieldType. Data can be accessed with these column names. For example:

# Table view with 2 captured fields
>>> fdata.data
array([(0, 10),
       (1, 11),
       (2, 12)],
      dtype=[('COUNTER1.OUT.Value', '<f8'), ('COUNTER2.OUT.Value', '<f8')])
# Row view
>>> fdata.data[0]
(0, 10)
# Column names
>>> fdata.column_names
('COUNTER1.OUT.Value', 'COUNTER2.OUT.Value')
# Column view
>>> fdata.data['COUNTER1.OUT.Value']
(0, 1, 2)

property column_names

Return all the column names

class pandablocks.responses.EndData(samples: int, reason: pandablocks.responses.EndReason)

Yielded when a PCAP acquisition ends.

samples

The total number of samples (rows) that were yielded

reason

The EndReason for the end of acquisition

Connections

Sans-IO connections for both the Control and Data ports of PandA TCP server.

exception pandablocks.connections.NeedMoreData

Raised if the Buffer isn’t full enough to return the requested bytes

class pandablocks.connections.Buffer

Byte storage that provides line reader and bytes reader interfaces. For example:

buf = Buffer()
buf += bytes_from_server
line = buf.read_line()  # raises NeedMoreData if no line
for line in buf:
    pass
bytes = buf.read_bytes(50)  # raises NeedMoreData if not enough bytes

read_bytes(num: int) → bytearray

Read and pop num bytes from the beginning of the buffer, raising NeedMoreData if the buffer isn’t full enough to do so

peek_bytes(num: int) → bytearray

Read but do not pop num bytes from the beginning of the buffer, raising NeedMoreData if the buffer isn’t full enough to do so

read_line()

Read and pop a newline terminated line (without terminator) from the beginning of the buffer, raising NeedMoreData if the buffer isn’t full enough to do so

class pandablocks.connections.ControlConnection

Sans-IO connection to control port of PandA TCP server, supporting a Command based interface. For example:

cc = ControlConnection()
# Connection says what bytes should be sent to execute command
to_send = cc.send(command)
socket.sendall(to_send)
while True:
    # Repeatedly process bytes from the server looking for responses
    from_server = socket.recv()
    for command, response in cc.receive_bytes(from_server):
        do_something_with(response)

receive_bytes(received: bytes) → Iterator[Tuple[pandablocks.commands.Command, Any]]

Tell the connection that you have received some bytes off the network. Parse these into high level responses which are yielded back with the command that triggered this response

send(command: pandablocks.commands.Command) → bytes

Tell the connection you want to send an event, and it will return some bytes to send down the network

class pandablocks.connections.DataConnection

Sans-IO connection to data port of PandA TCP server, supporting an flushable iterator interface. For example:

dc = ControlConnection()
# Single connection string to send
to_send = dc.connect()
socket.sendall(to_send)
while True:
    # Repeatedly process bytes from the server looking for data
    from_server = socket.recv()
    for data in dc.receive_bytes(from_server):
        do_something_with(data)

receive_bytes(received: bytes, flush_every_frame=True) → Iterator[pandablocks.responses.Data]

Tell the connection that you have received some bytes off the network. Parse these into Data structures and yield them back.

Parameters

• received – the bytes you received from the socket

• flush_every_frame – Whether to flush FrameData as soon as received. If False then they will only be sent if flush is called or end of acquisition reached

flush() → Iterator[pandablocks.responses.FrameData]

If there is a partial data frame, pop and yield it

connect(scaled: bool) → bytes

Return the bytes that need to be sent on connection

Asyncio Client

This is an asyncio wrapper to the ControlConnection and DataConnection objects, allowing async calls to send(command) and iterate over data().

class pandablocks.asyncio.AsyncioClient(host: str)

Asyncio implementation of a PandABlocks client. For example:

async with AsyncioClient("hostname-or-ip") as client:
    # Control and data ports are now connected
    resp1, resp2 = await asyncio.gather(client.send(cmd1), client.send(cmd2))
    resp3 = await client.send(cmd3)
    async for data in client.data():
        handle(data)
# Control and data ports are now disconnected

async connect()

Connect to the control and data ports, and be ready to handle commands

async close()

Close the control and data connections, and wait for completion

async send(command: pandablocks.commands.Command[T]) → T

Send a command to control port of the PandA, returning its response.

Parameters

command – The Command to send

data(scaled: bool = True, flush_period: float = None, frame_timeout: float = None) → AsyncGenerator[pandablocks.responses.Data, None]

Connect to data port and yield data frames

Parameters

• scaled – Whether to scale and average data frames, reduces throughput

• flush_period – How often to flush partial data frames, None is on every chunk of data from the server

• frame_timeout – If no data is received for this amount of time, raise asyncio.TimeoutError

Blocking Client

This is a blocking wrapper to the ControlConnection and DataConnection objects, allowing blocking calls to send(commands) and iterate over data().

class pandablocks.blocking.BlockingClient(host: str)

Blocking implementation of a PandABlocks client. For example:

with BlockingClient("hostname-or-ip") as client:
    # Control and data ports are now connected
    resp1, resp2 = client.send([cmd1, cmd2])
    resp3 = client.send(cmd3)
    for data in client.data():
        handle(data)
# Control and data ports are now disconnected

connect()

Connect to the control and data ports, and be ready to handle commands

close()

Close the control and data connections, and wait for completion

send(commands: pandablocks.commands.Command[T], timeout: int = 'None') → T

send(commands: Iterable[pandablocks.commands.Command], timeout: int = 'None') → List

Send a command to control port of the PandA, returning its response.

Parameters

• commands – If single Command, return its response. If a list of commands return a list of reponses

• timeout – If no reponse in this time, raise socket.timeout

data(scaled: bool = True, frame_timeout: int = None) → Iterator[pandablocks.responses.Data]

Connect to data port and yield data frames

Parameters

• scaled – Whether to scale and average data frames, reduces throughput

• frame_timeout – If no data is received for this amount of time, raise socket.timeout

HDF Writing

This package contains components needed to write PCAP data to and HDF file in the most efficient way. The oneshot write_hdf_files is exposed in the commandline interface. It assembles a short Pipeline of:

AsyncioClient -> FrameProcessor -> HDFWriter

The FrameProcessor and HDFWriter run in their own threads as most of the heavy lifting is done by numpy and h5py, so running in their own threads gives multi-CPU benefits without hitting the limit of the GIL.

See also

How to use the library to capture HDF files, How fast can we write HDF files?

class pandablocks.hdf.Pipeline

Helper class that runs a pipeline consumer process in its own thread

what_to_do: Dict[Type, Callable]

Subclasses should create this dictionary with handlers for each data type, returning transformed data that should be passed downstream

stop()

Stop the processing after the current queue has been emptied

class pandablocks.hdf.HDFWriter(scheme: str)

Write an HDF file per data collection. Each field will be written in a 1D dataset /<field.name>.<field.capture>.

Parameters

scheme – Filepath scheme, where %d will be replaced with the acquisition number, starting with 1

class pandablocks.hdf.FrameProcessor

Scale field data according to the information in the StartData

pandablocks.hdf.create_pipeline(*elements: pandablocks.hdf.Pipeline) → List[pandablocks.hdf.Pipeline]

Create a pipeline of elements, wiring them and starting them before returning them

pandablocks.hdf.stop_pipeline(pipeline: List[pandablocks.hdf.Pipeline])

Stop and join each element of the pipeline

async pandablocks.hdf.write_hdf_files(client: pandablocks.asyncio.AsyncioClient, scheme: str, num: int = 1, arm: bool = False)

Connect to host PandA data port, and write num acquisitions to HDF file according to scheme

Parameters

• client – The AsyncioClient to use for communications

• scheme – Filenaming scheme for HDF files, with %d for scan number starting at 1

• num – The number of acquisitions to store in separate files

• arm – Whether to arm PCAP at the start, and after each successful acquisition