Python code generator

The Python code generator creates an accessor class for reading, writing and printing register and field values. The class will have a named method for each of these operations on each register and field. The methods use native Python types (e.g. int, float, Enum) to represent values that are read or written. This means that the user never has to do any bit slicing or casting.

• PythonPickleGenerator saves a Python pickle file containing the RegisterList, along with a small wrapper file to recreate the object.

• PythonAccessorGenerator creates a Python class with methods to read/write/print each register and field.

These can be used in a Python-based system test environment to conveniently perform register read/writes on the target device.

The artifacts are generated like this:

Python code that parses the example TOML file and generates Python register artifacts.

# Standard libraries
import sys
from pathlib import Path

# First party libraries
from hdl_registers.generator.python.accessor import PythonAccessorGenerator
from hdl_registers.generator.python.pickle import PythonPickleGenerator
from hdl_registers.parser.toml import from_toml

THIS_DIR = Path(__file__).parent


def main(output_folder: Path):
    """
    Create register Python artifacts from the TOML example file.
    """
    register_list = from_toml(
        name="example",
        toml_file=THIS_DIR.parent.parent / "user_guide" / "toml" / "toml_format.toml",
    )

    PythonPickleGenerator(register_list=register_list, output_folder=output_folder).create()

    PythonAccessorGenerator(register_list=register_list, output_folder=output_folder).create()


if __name__ == "__main__":
    main(output_folder=Path(sys.argv[1]))

Pickle code

The script below is generated by the PythonPickleGenerator and can be used to re-create the RegisterList object from the Python pickle file.

Click to expand/collapse code.

Example Python pickle re-creation script

# This file is automatically generated by hdl-registers version 5.2.1-dev.
# Code generator PythonPickleGenerator version 1.0.0.
# Generated 2024-07-26 20:51 from file toml_format.toml at commit fde82ed5a49c96e3.
# Register hash 4df9765ebb584803b583628671e4659579eb85f4.

# Standard libraries
import pickle
from pathlib import Path
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    # Third party libraries
    from hdl_registers.register_list import RegisterList

THIS_DIR = Path(__file__).parent
PICKLE_FILE = THIS_DIR / "example.pickle"


class Example:
    """
    Instantiate this class to get the ``RegisterList`` object for the
    ``example`` register list.
    """

    def __new__(cls):
        """
        Recreate the ``RegisterList`` object from binary pickle.
        """
        with PICKLE_FILE.open("rb") as file_handle:
            return pickle.load(file_handle)


def get_register_list() -> "RegisterList":
    """
    Return a ``RegisterList`` object with the registers/constants from the
    ``example`` register list.
    Recreated from a Python pickle file.
    """
    with PICKLE_FILE.open("rb") as file_handle:
        return pickle.load(file_handle)

Accessor code

The class below is generated by the PythonAccessorGenerator for performing register and field operations on a target device. Interesting things to notice:

1. Register read/write operations use a generated Python dataclass to represent a register with field value members. See e.g. read_config and write_config methods, and the ExampleConfigValue type in the generated code.

2. Field values are represented using native Python types.

1. A bit field is represented as a Python int.

2. A bit vector field WITHOUT fractional bits, whether signed or unsigned, is represented as a Python int.

3. A bit vector field WITH fractional bits, whether signed or unsigned, is represented as a Python float.

4. An enumeration field is represented using a custom Python Enum type. See e.g. ExampleConfigValue.Direction.

5. An integer field, whether signed or unsigned, is represented as a Python int.

3. In cases where the register has no fields, the read and write use a plain integer instead. See e.g. read_status.

4. Each writeable register has methods to write

   1. The entire register value (e.g. write_config), which takes a dataclass value as argument.

   2. Individual field values (e.g. write_config_direction), which takes the native Python field representation as argument.

   Field write methods will either read-modify-write or plain write the register, depending on what the register mode allows.

Printing values

Each register value type has a to_string method to convert the field values to a human-readable string. The accessor class also implements the print_registers method, which will print all register and field values:

Example register printout with many fields of different types.

Note how only readable registers are printed. Field values are present firstly as their native type, but also as their decimal, hexadecimal and binary encodings.

Color

By default, color output is enabled, since it improves readability when a lot of information is presented. It can be turned of by setting the no_color argument to print_registers, or by setting the environment variable NO_COLOR to any value.

Register accessor interface

The code generated by PythonAccessorGenerator performs the address calculation, mode logic, bit slicing, and type casting for you. It can not, however, implement the actual reading and writing of register values on your target device. This must be implemented by the user, given the methods of access that are available in your environment (SSH, telnet, UART, etc). Hdl-registers can not implement this in a generic way that is useful in all environments, since device setup is so different in different projects.

The class constructor takes a register_accessor argument of type PythonRegisterAccessorInterface. This object must be constructed by the user, must inherit the PythonRegisterAccessorInterface, and must implement the read_register and write_register methods. See the API documentation for more details.

Code

Example code generated by PythonAccessorGenerator. Using the generated code is done by

1. Generating artifacts as shown here.

2. Adding the generated Python module to your Python PATH.

3. Instantiating the object, by either

1. Class instantiation, e.g. accessor = ExampleAccessor(register_accessor=register_accessor), where “Example” in the class name is the name of the register list.

2. Running the convenience function accessor = get_accessor(register_accessor=register_accessor) where the register list name is not needed anywhere.

Example Python accessor class

# This file is automatically generated by hdl-registers version 5.2.1-dev.
# Code generator PythonAccessorGenerator version 0.1.0.
# Generated 2024-07-26 20:51 from file toml_format.toml at commit fde82ed5a49c96e3.
# Register hash 4df9765ebb584803b583628671e4659579eb85f4.

# Standard libraries
import pickle
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import TYPE_CHECKING
from termcolor import colored

# Third party libraries
from hdl_registers.field.numerical_interpretation import to_unsigned_binary
from tsfpga.math_utils import to_binary_nibble_string, to_hex_byte_string

if TYPE_CHECKING:
    # Third party libraries
    from hdl_registers.generator.python.register_accessor_interface import (
        PythonRegisterAccessorInterface,
    )
    from hdl_registers.register_list import RegisterList


THIS_DIR = Path(__file__).parent


class ExampleAccessor:
    """
    Class to read/write registers and fields of the ``example`` register list.
    """

    def __init__(self, register_accessor: "PythonRegisterAccessorInterface"):
        """
        Arguments:
            register_accessor: Object that implements register access methods in your system.
                Must have the methods ``read_register`` and ``write_register``
                with the same interface and behavior as the interface class
                :class:`.PythonRegisterAccessorInterface`.
                It is highly recommended that your accessor class inherits from that class.
        """
        self._register_accessor = register_accessor

        pickle_path = THIS_DIR / "example.pickle"
        if not pickle_path.exists():
            raise FileNotFoundError(
                f"Could not find the pickle file {pickle_path}, "
                "make sure this artifact is generated."
            )

        with pickle_path.open("rb") as file_handle:
            self._register_list: "RegisterList" = pickle.load(file_handle)

    def _read_register(self, register_index: int) -> int:
        """
        Read a register value via the register accessor provided by the user.
        We perform sanity checks in both directions here, since this is the glue logic between
        two interfaces.
        """
        if not 0 <= register_index <= 9:
            raise ValueError(
                f"Register index {register_index} out of range. This is likely an internal error."
            )

        register_value = self._register_accessor.read_register(
            register_list_name="example", register_address=4 * register_index
        )
        if not 0 <= register_value < 2**32:
            raise ValueError(
                f'Register read value "{register_value}" from accessor is out of range.'
            )

        return register_value

    def _write_register(self, register_index: int, register_value: int) -> None:
        """
        Write a register value via the register accessor provided by the user.
        We perform sanity checks in both directions here, since this is the glue logic between
        two interfaces.
        """
        if not 0 <= register_index <= 9:
            raise ValueError(
                f"Register index {register_index} out of range. This is likely an internal error."
            )

        if not 0 <= register_value < 2**32:
            raise ValueError(f'Register write value "{register_value}" is out of range.')

        self._register_accessor.write_register(
            register_list_name="example",
            register_address=4 * register_index,
            register_value=register_value,
        )

    def read_config(self) -> "ExampleConfigValue":
        """
        Read the 'config' register.
        """
        index = self._register_list.get_register_index(
            register_name="config",
            register_array_name=None,
            register_array_index=None,
        )
        value = self._read_register(register_index=index)
        register = self._register_list.get_register(
            register_name="config", register_array_name=None
        )
        return ExampleConfigValue(
            enable=register.fields[0].get_value(register_value=value),
            direction=ExampleConfigValue.Direction(
                register.fields[1].get_value(register_value=value).name
            ),
        )

    def write_config(self, register_value: "ExampleConfigValue") -> None:
        """
        Write the whole 'config' register.

        Arguments:
            register_value: Object with the field values that shall be written.
        """
        index = self._register_list.get_register_index(
            register_name="config",
            register_array_name=None,
            register_array_index=None,
        )
        register = self._register_list.get_register(
            register_name="config", register_array_name=None
        )
        integer_value = 0
        integer_value += register.fields[0].set_value(field_value=register_value.enable)
        integer_value += register.fields[1].set_value(
            field_value=register.fields[1].get_element_by_name(register_value.direction.value)
        )
        self._write_register(register_index=index, register_value=integer_value)

    def write_config_enable(self, field_value: "int") -> None:
        """
        Write the 'enable' field in the 'config' register.
        Will read-modify-write the ``config`` register, setting the ``enable`` field
        to the provided value, while leaving the other fields at their previous values.

        Arguments:
            field_value: The field value to be written.
                Single bit. Range: 0-1.
        """
        register_value = self.read_config()
        register_value.enable = field_value
        self.write_config(register_value=register_value)

    def write_config_direction(self, field_value: "ExampleConfigValue.Direction") -> None:
        """
        Write the 'direction' field in the 'config' register.
        Will read-modify-write the ``config`` register, setting the ``direction`` field
        to the provided value, while leaving the other fields at their previous values.

        Arguments:
            field_value: The field value to be written.
                Enumeration. Possible values: DATA_IN, HIGH_Z, DATA_OUT.
        """
        register_value = self.read_config()
        register_value.direction = field_value
        self.write_config(register_value=register_value)

    def read_status(self) -> "int":
        """
        Read the 'status' register.
        Return type is a plain unsigned integer since the register has no fields.
        """
        index = self._register_list.get_register_index(
            register_name="status",
            register_array_name=None,
            register_array_index=None,
        )
        value = self._read_register(register_index=index)
        return value

    def read_channels_read_address(self, array_index: int) -> "int":
        """
        Read the 'read_address' register within the 'channels' register array.
        Return type is a plain unsigned integer since the register has no fields.

        Arguments:
            array_index: Register array iteration index (range 0-3).
        """
        index = self._register_list.get_register_index(
            register_name="read_address",
            register_array_name="channels",
            register_array_index=array_index,
        )
        value = self._read_register(register_index=index)
        return value

    def write_channels_read_address(self, register_value: "int", array_index: int) -> None:
        """
        Write the whole 'read_address' register within the 'channels' register array.

        Arguments:
            register_value: Value to be written.
                Is a plain unsigned integer since the register has no fields.
            array_index: Register array iteration index (range 0-3).
        """
        index = self._register_list.get_register_index(
            register_name="read_address",
            register_array_name="channels",
            register_array_index=array_index,
        )
        self._write_register(register_index=index, register_value=register_value)

    def write_channels_config(
        self, register_value: "ExampleChannelsConfigValue", array_index: int
    ) -> None:
        """
        Write the whole 'config' register within the 'channels' register array.

        Arguments:
            register_value: Object with the field values that shall be written.
            array_index: Register array iteration index (range 0-3).
        """
        index = self._register_list.get_register_index(
            register_name="config",
            register_array_name="channels",
            register_array_index=array_index,
        )
        register = self._register_list.get_register(
            register_name="config", register_array_name="channels"
        )
        integer_value = 0
        integer_value += register.fields[0].set_value(field_value=register_value.enable)
        integer_value += register.fields[1].set_value(field_value=register_value.tuser)
        self._write_register(register_index=index, register_value=integer_value)

    def write_channels_config_enable(self, field_value: "int", array_index: int) -> None:
        """
        Write the 'enable' field in the 'config' register within the 'channels' register array.
        Will write the ``config`` register, setting the ``enable`` field to the
        provided value, and all other fields to their default values.

        Arguments:
            field_value: The field value to be written.
                Single bit. Range: 0-1.
            array_index: Register array iteration index (range 0-3).
        """
        index = self._register_list.get_register_index(
            register_name="config",
            register_array_name="channels",
            register_array_index=array_index,
        )

        register = self._register_list.get_register(
            register_name="config", register_array_name="channels"
        )
        register_value = 0
        register_value += register.fields[0].set_value(field_value)
        register_value += register.fields[1].default_value_uint << register.fields[1].base_index

        self._write_register(register_index=index, register_value=register_value)

    def write_channels_config_tuser(self, field_value: "int", array_index: int) -> None:
        """
        Write the 'tuser' field in the 'config' register within the 'channels' register array.
        Will write the ``config`` register, setting the ``tuser`` field to the
        provided value, and all other fields to their default values.

        Arguments:
            field_value: The field value to be written.
                Unsigned bit vector. Width: 8. Range: 0 - 255.
            array_index: Register array iteration index (range 0-3).
        """
        index = self._register_list.get_register_index(
            register_name="config",
            register_array_name="channels",
            register_array_index=array_index,
        )

        register = self._register_list.get_register(
            register_name="config", register_array_name="channels"
        )
        register_value = 0
        register_value += register.fields[0].default_value_uint << register.fields[0].base_index
        register_value += register.fields[1].set_value(field_value)

        self._write_register(register_index=index, register_value=register_value)

    def print_registers(self, no_color: bool = False) -> None:
        """
        Print all registers and their values to STDOUT.

        Arguments:
            no_color: Disable color output.
                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
        """
        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'config' "
            + colored("." * 61, color="dark_grey", no_color=no_color)
            + " (index 0, address 0):"
        )
        register_value = self.read_config()
        print(register_value.to_string(indentation="  ", no_color=no_color))
        print()

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'status' "
            + colored("." * 61, color="dark_grey", no_color=no_color)
            + " (index 1, address 4):"
        )
        register_value = self.read_status()
        formatted_value = _format_unsigned_number(value=register_value, width=32)
        print(f"  {formatted_value}\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[0].read_address' "
            + colored("." * 43, color="dark_grey", no_color=no_color)
            + " (index 2, address 8):"
        )
        register_value = self.read_channels_read_address(array_index=0)
        formatted_value = _format_unsigned_number(value=register_value, width=32)
        print(f"  {formatted_value}\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[0].config' "
            + colored("." * 49, color="dark_grey", no_color=no_color)
            + " (index 3, address 12):"
        )
        print("  Not readable.\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[1].read_address' "
            + colored("." * 43, color="dark_grey", no_color=no_color)
            + " (index 4, address 16):"
        )
        register_value = self.read_channels_read_address(array_index=1)
        formatted_value = _format_unsigned_number(value=register_value, width=32)
        print(f"  {formatted_value}\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[1].config' "
            + colored("." * 49, color="dark_grey", no_color=no_color)
            + " (index 5, address 20):"
        )
        print("  Not readable.\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[2].read_address' "
            + colored("." * 43, color="dark_grey", no_color=no_color)
            + " (index 6, address 24):"
        )
        register_value = self.read_channels_read_address(array_index=2)
        formatted_value = _format_unsigned_number(value=register_value, width=32)
        print(f"  {formatted_value}\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[2].config' "
            + colored("." * 49, color="dark_grey", no_color=no_color)
            + " (index 7, address 28):"
        )
        print("  Not readable.\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[3].read_address' "
            + colored("." * 43, color="dark_grey", no_color=no_color)
            + " (index 8, address 32):"
        )
        register_value = self.read_channels_read_address(array_index=3)
        formatted_value = _format_unsigned_number(value=register_value, width=32)
        print(f"  {formatted_value}\n")

        print(
            colored("Register", color="light_yellow", no_color=no_color)
            + " 'channels[3].config' "
            + colored("." * 49, color="dark_grey", no_color=no_color)
            + " (index 9, address 36):"
        )
        print("  Not readable.\n")


def get_accessor(register_accessor: "PythonRegisterAccessorInterface") -> ExampleAccessor:
    """
    Factory function to create an accessor object.
    """
    return ExampleAccessor(register_accessor=register_accessor)


def _format_unsigned_number(value: int, width: int, include_decimal: bool = True) -> str:
    """
    Format a string representation of the provided ``value``.
    Suitable for printing field and register values.
    """
    result = ""
    if include_decimal:
        result += f"unsigned decimal {value}, "

    hex_string = to_hex_byte_string(value=value, result_width_bits=width)
    result += f"hexadecimal {hex_string}, "

    binary_string = to_binary_nibble_string(value=value, result_width_bits=width)
    result += f"binary {binary_string}"

    return result


@dataclass
class ExampleConfigValue:
    """
    Represents the field values of the 'config' register.
    Uses native Python types for easy handling.
    """

    class Direction(Enum):
        """
        Enumeration elements for the ``direction`` field.
        """

        DATA_IN = "data_in"
        HIGH_Z = "high_z"
        DATA_OUT = "data_out"

    # Single bit. Range: 0-1.
    enable: int
    # Enumeration. Possible values: DATA_IN, HIGH_Z, DATA_OUT.
    direction: Direction

    def to_string(self, indentation: str = "", no_color: bool = False) -> str:
        """
        Get a string of the field values, suitable for printing.

        Arguments:
            indentation: Optionally indent each field value line.
            no_color: Disable color output.
                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
        """
        values = []

        value = str(self.enable)
        field_name = colored("enable", color="light_cyan", no_color=no_color)
        values.append(f"{indentation}{field_name}: {value}")

        enum_index = list(self.Direction).index(self.direction)
        value = f"{self.direction.name} ({enum_index})"
        field_name = colored("direction", color="light_cyan", no_color=no_color)
        values.append(f"{indentation}{field_name}: {value}")

        return "\n".join(values)


@dataclass
class ExampleChannelsConfigValue:
    """
    Represents the field values of the 'config' register within the 'channels' register array.
    Uses native Python types for easy handling.
    """

    # Single bit. Range: 0-1.
    enable: int
    # Unsigned bit vector. Width: 8. Range: 0 - 255.
    tuser: int

    def to_string(self, indentation: str = "", no_color: bool = False) -> str:
        """
        Get a string of the field values, suitable for printing.

        Arguments:
            indentation: Optionally indent each field value line.
            no_color: Disable color output.
                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
        """
        values = []

        value = str(self.enable)
        field_name = colored("enable", color="light_cyan", no_color=no_color)
        values.append(f"{indentation}{field_name}: {value}")

        value_comment = _format_unsigned_number(value=self.tuser, width=8, include_decimal=False)
        value = f"{self.tuser} ({value_comment})"
        field_name = colored("tuser", color="light_cyan", no_color=no_color)
        values.append(f"{indentation}{field_name}: {value}")

        return "\n".join(values)