Python code generator

The Python code generator creates an accessor class for reading, writing and printing register and field values. The class will have a named method for each of these operations on each register and field. The methods use native Python types (e.g. int, float, Enum) to represent values that are read or written. This means that the user never has to do any bit slicing or casting.

These can be used in a Python-based system test environment to conveniently perform register read/writes on the target device.

The artifacts are generated like this:

Python code that parses the example TOML file and generates Python register artifacts.
 1# Standard libraries
 2import sys
 3from pathlib import Path
 4
 5# First party libraries
 6from hdl_registers.generator.python.accessor import PythonAccessorGenerator
 7from hdl_registers.generator.python.pickle import PythonPickleGenerator
 8from hdl_registers.parser.toml import from_toml
 9
10THIS_DIR = Path(__file__).parent
11
12
13def main(output_folder: Path):
14    """
15    Create register Python artifacts from the TOML example file.
16    """
17    register_list = from_toml(
18        name="example",
19        toml_file=THIS_DIR.parent.parent / "user_guide" / "toml" / "toml_format.toml",
20    )
21
22    PythonPickleGenerator(register_list=register_list, output_folder=output_folder).create()
23
24    PythonAccessorGenerator(register_list=register_list, output_folder=output_folder).create()
25
26
27if __name__ == "__main__":
28    main(output_folder=Path(sys.argv[1]))

Pickle code

The script below is generated by the PythonPickleGenerator and can be used to re-create the RegisterList object from the Python pickle file.

Click to expand/collapse code.
Example Python pickle re-creation script
 1# This file is automatically generated by hdl-registers version 5.2.1-dev.
 2# Code generator PythonPickleGenerator version 1.0.0.
 3# Generated 2024-05-21 20:51 from file toml_format.toml at commit 2f4170d05b13abac.
 4# Register hash 0d636d16ec9bdadff4d5e144aaacd9416830c91d.
 5
 6# Standard libraries
 7import pickle
 8from pathlib import Path
 9from typing import TYPE_CHECKING
10
11if TYPE_CHECKING:
12    # Third party libraries
13    from hdl_registers.register_list import RegisterList
14
15THIS_DIR = Path(__file__).parent
16PICKLE_FILE = THIS_DIR / "example.pickle"
17
18
19class Example:
20    """
21    Instantiate this class to get the ``RegisterList`` object for the
22    ``example`` register list.
23    """
24
25    def __new__(cls):
26        """
27        Recreate the ``RegisterList`` object from binary pickle.
28        """
29        with PICKLE_FILE.open("rb") as file_handle:
30            return pickle.load(file_handle)
31
32
33def get_register_list() -> "RegisterList":
34    """
35    Return a ``RegisterList`` object with the registers/constants from the
36    ``example`` register list.
37    Recreated from a Python pickle file.
38    """
39    with PICKLE_FILE.open("rb") as file_handle:
40        return pickle.load(file_handle)

Accessor code

The class below is generated by the PythonAccessorGenerator for performing register and field operations on a target device. Interesting things to notice:

  1. Register read/write operations use a generated Python dataclass to represent a register with field value members. See e.g. read_config and write_config methods, and the ExampleConfigValue type in the generated code.

  2. Field values are represented using native Python types.

  1. A bit field is represented as a Python int.

  2. A bit vector field WITHOUT fractional bits, whether signed or unsigned, is represented as a Python int.

  3. A bit vector field WITH fractional bits, whether signed or unsigned, is represented as a Python float.

  4. An enumeration field is represented using a custom Python Enum type. See e.g. ExampleConfigValue.Direction.

  5. An integer field, whether signed or unsigned, is represented as a Python int.

  1. In cases where the register has no fields, the read and write use a plain integer instead. See e.g. read_status.

  2. Each writeable register has methods to write

    1. The entire register value (e.g. write_config), which takes a dataclass value as argument.

    2. Individual field values (e.g. write_config_direction), which takes the native Python field representation as argument.

    Field write methods will either read-modify-write or plain write the register, depending on what the register mode allows.

Printing values

Each register value type has a to_string method to convert the field values to a human-readable string. The accessor class also implements the print_registers method, which will print all register and field values:

../../_images/print_registers.png

Example register printout with many fields of different types.

Note how only readable registers are printed. Field values are present firstly as their native type, but also as their decimal, hexadecimal and binary encodings.

Color

By default, color output is enabled, since it improves readability when a lot of information is presented. It can be turned of by setting the no_color argument to print_registers, or by setting the environment variable NO_COLOR to any value.

Register accessor interface

The code generated by PythonAccessorGenerator performs the address calculation, mode logic, bit slicing, and type casting for you. It can not, however, implement the actual reading and writing of register values on your target device. This must be implemented by the user, given the methods of access that are available in your environment (SSH, telnet, UART, etc). Hdl-registers can not implement this in a generic way that is useful in all environments, since device setup is so different in different projects.

The class constructor takes a register_accessor argument of type PythonRegisterAccessorInterface. This object must be constructed by the user, must inherit the PythonRegisterAccessorInterface, and must implement the read_register and write_register methods. See the API documentation for more details.

Code

Example code generated by PythonAccessorGenerator. Using the generated code is done by

  1. Generating artifacts as shown here.

  2. Adding the generated Python module to your Python PATH.

  3. Instantiating the object, by either

  1. Class instantiation, e.g. accessor = ExampleAccessor(register_accessor=register_accessor), where “Example” in the class name is the name of the register list.

  2. Running the convenience function accessor = get_accessor(register_accessor=register_accessor) where the register list name is not needed anywhere.

Example Python accessor class
  1# This file is automatically generated by hdl-registers version 5.2.1-dev.
  2# Code generator PythonAccessorGenerator version 0.1.0.
  3# Generated 2024-05-21 20:51 from file toml_format.toml at commit 2f4170d05b13abac.
  4# Register hash 0d636d16ec9bdadff4d5e144aaacd9416830c91d.
  5
  6# Standard libraries
  7import pickle
  8from dataclasses import dataclass
  9from enum import Enum
 10from pathlib import Path
 11from typing import TYPE_CHECKING
 12from termcolor import colored
 13
 14# Third party libraries
 15from hdl_registers.field.register_field_type import to_unsigned_binary
 16from tsfpga.math_utils import to_binary_nibble_string, to_hex_byte_string
 17
 18if TYPE_CHECKING:
 19    # Third party libraries
 20    from hdl_registers.generator.python.register_accessor_interface import (
 21        PythonRegisterAccessorInterface,
 22    )
 23    from hdl_registers.register_list import RegisterList
 24
 25
 26THIS_DIR = Path(__file__).parent
 27
 28
 29class ExampleAccessor:
 30    """
 31    Class to read/write registers and fields of the ``example`` register list.
 32    """
 33
 34    def __init__(self, register_accessor: "PythonRegisterAccessorInterface"):
 35        """
 36        Arguments:
 37            register_accessor: Object that implements register access methods in your system.
 38                Must have the methods ``read_register`` and ``write_register``
 39                with the same interface and behavior as the interface class
 40                :class:`.PythonRegisterAccessorInterface`.
 41                It is highly recommended that your accessor class inherits from that class.
 42        """
 43        self._register_accessor = register_accessor
 44
 45        pickle_path = THIS_DIR / "example.pickle"
 46        if not pickle_path.exists():
 47            raise FileNotFoundError(
 48                f"Could not find the pickle file {pickle_path}, "
 49                "make sure this artifact is generated."
 50            )
 51
 52        with pickle_path.open("rb") as file_handle:
 53            self._register_list: "RegisterList" = pickle.load(file_handle)
 54
 55    def _read_register(self, register_index: int) -> int:
 56        """
 57        Read a register value via the register accessor provided by the user.
 58        We perform sanity checks in both directions here, since this is the glue logic between
 59        two interfaces.
 60        """
 61        if not 0 <= register_index <= 9:
 62            raise ValueError(
 63                f"Register index {register_index} out of range. This is likely an internal error."
 64            )
 65
 66        register_value = self._register_accessor.read_register(
 67            register_list_name="example", register_address=4 * register_index
 68        )
 69        if not 0 <= register_value < 2**32:
 70            raise ValueError(
 71                f'Register read value "{register_value}" from accessor is out of range.'
 72            )
 73
 74        return register_value
 75
 76    def _write_register(self, register_index: int, register_value: int) -> None:
 77        """
 78        Write a register value via the register accessor provided by the user.
 79        We perform sanity checks in both directions here, since this is the glue logic between
 80        two interfaces.
 81        """
 82        if not 0 <= register_index <= 9:
 83            raise ValueError(
 84                f"Register index {register_index} out of range. This is likely an internal error."
 85            )
 86
 87        if not 0 <= register_value < 2**32:
 88            raise ValueError(f'Register write value "{register_value}" is out of range.')
 89
 90        self._register_accessor.write_register(
 91            register_list_name="example",
 92            register_address=4 * register_index,
 93            register_value=register_value,
 94        )
 95
 96    def read_config(self) -> "ExampleConfigValue":
 97        """
 98        Read the 'config' register.
 99        """
100        index = self._register_list.get_register_index(
101            register_name="config",
102            register_array_name=None,
103            register_array_index=None,
104        )
105        value = self._read_register(register_index=index)
106        register = self._register_list.get_register(name="config", register_array_name=None)
107        return ExampleConfigValue(
108            enable=register.fields[0].get_value(register_value=value),
109            direction=ExampleConfigValue.Direction(
110                register.fields[1].get_value(register_value=value).name
111            ),
112        )
113
114    def write_config(self, register_value: "ExampleConfigValue") -> None:
115        """
116        Write the whole 'config' register.
117
118        Arguments:
119            register_value: Object with the field values that shall be written.
120        """
121        index = self._register_list.get_register_index(
122            register_name="config",
123            register_array_name=None,
124            register_array_index=None,
125        )
126        register = self._register_list.get_register(name="config", register_array_name=None)
127        integer_value = 0
128        integer_value += register.fields[0].set_value(field_value=register_value.enable)
129        integer_value += register.fields[1].set_value(
130            field_value=register.fields[1].get_element_by_name(register_value.direction.value)
131        )
132        self._write_register(register_index=index, register_value=integer_value)
133
134    def write_config_enable(self, field_value: "int") -> None:
135        """
136        Write the 'enable' field in the 'config' register.
137        Will read-modify-write the ``config`` register, setting the ``enable`` field
138        to the provided value, while leaving the other fields at their previous values.
139
140        Arguments:
141            field_value: The field value to be written.
142                Single bit. Range: 0-1.
143        """
144        register_value = self.read_config()
145        register_value.enable = field_value
146        self.write_config(register_value=register_value)
147
148    def write_config_direction(self, field_value: "ExampleConfigValue.Direction") -> None:
149        """
150        Write the 'direction' field in the 'config' register.
151        Will read-modify-write the ``config`` register, setting the ``direction`` field
152        to the provided value, while leaving the other fields at their previous values.
153
154        Arguments:
155            field_value: The field value to be written.
156                Enumeration. Possible values: DATA_IN, HIGH_Z, DATA_OUT.
157        """
158        register_value = self.read_config()
159        register_value.direction = field_value
160        self.write_config(register_value=register_value)
161
162    def read_status(self) -> "int":
163        """
164        Read the 'status' register.
165        Return type is a plain unsigned integer since the register has no fields.
166        """
167        index = self._register_list.get_register_index(
168            register_name="status",
169            register_array_name=None,
170            register_array_index=None,
171        )
172        value = self._read_register(register_index=index)
173        return value
174
175    def read_channels_read_address(self, array_index: int) -> "int":
176        """
177        Read the 'read_address' register within the 'channels' register array.
178        Return type is a plain unsigned integer since the register has no fields.
179
180        Arguments:
181            array_index: Register array iteration index (range 0-3).
182        """
183        index = self._register_list.get_register_index(
184            register_name="read_address",
185            register_array_name="channels",
186            register_array_index=array_index,
187        )
188        value = self._read_register(register_index=index)
189        return value
190
191    def write_channels_read_address(self, register_value: "int", array_index: int) -> None:
192        """
193        Write the whole 'read_address' register within the 'channels' register array.
194
195        Arguments:
196            register_value: Value to be written.
197                Is a plain unsigned integer since the register has no fields.
198            array_index: Register array iteration index (range 0-3).
199        """
200        index = self._register_list.get_register_index(
201            register_name="read_address",
202            register_array_name="channels",
203            register_array_index=array_index,
204        )
205        self._write_register(register_index=index, register_value=register_value)
206
207    def write_channels_config(
208        self, register_value: "ExampleChannelsConfigValue", array_index: int
209    ) -> None:
210        """
211        Write the whole 'config' register within the 'channels' register array.
212
213        Arguments:
214            register_value: Object with the field values that shall be written.
215            array_index: Register array iteration index (range 0-3).
216        """
217        index = self._register_list.get_register_index(
218            register_name="config",
219            register_array_name="channels",
220            register_array_index=array_index,
221        )
222        register = self._register_list.get_register(name="config", register_array_name="channels")
223        integer_value = 0
224        integer_value += register.fields[0].set_value(field_value=register_value.enable)
225        integer_value += register.fields[1].set_value(field_value=register_value.tuser)
226        self._write_register(register_index=index, register_value=integer_value)
227
228    def write_channels_config_enable(self, field_value: "int", array_index: int) -> None:
229        """
230        Write the 'enable' field in the 'config' register within the 'channels' register array.
231        Will write the ``config`` register, setting the ``enable`` field to the
232        provided value, and all other fields to their default values.
233
234        Arguments:
235            field_value: The field value to be written.
236                Single bit. Range: 0-1.
237            array_index: Register array iteration index (range 0-3).
238        """
239        index = self._register_list.get_register_index(
240            register_name="config",
241            register_array_name="channels",
242            register_array_index=array_index,
243        )
244
245        register = self._register_list.get_register(name="config", register_array_name="channels")
246        register_value = 0
247        register_value += register.fields[0].set_value(field_value)
248        register_value += register.fields[1].default_value_uint << register.fields[1].base_index
249
250        self._write_register(register_index=index, register_value=register_value)
251
252    def write_channels_config_tuser(self, field_value: "int", array_index: int) -> None:
253        """
254        Write the 'tuser' field in the 'config' register within the 'channels' register array.
255        Will write the ``config`` register, setting the ``tuser`` field to the
256        provided value, and all other fields to their default values.
257
258        Arguments:
259            field_value: The field value to be written.
260                Unsigned bit vector. Width: 8. Range: 0 - 255.
261            array_index: Register array iteration index (range 0-3).
262        """
263        index = self._register_list.get_register_index(
264            register_name="config",
265            register_array_name="channels",
266            register_array_index=array_index,
267        )
268
269        register = self._register_list.get_register(name="config", register_array_name="channels")
270        register_value = 0
271        register_value += register.fields[0].default_value_uint << register.fields[0].base_index
272        register_value += register.fields[1].set_value(field_value)
273
274        self._write_register(register_index=index, register_value=register_value)
275
276    def print_registers(self, no_color: bool = False) -> None:
277        """
278        Print all registers and their values to STDOUT.
279
280        Arguments:
281            no_color: Disable color output.
282                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
283        """
284        print(
285            colored("Register", color="light_yellow", no_color=no_color)
286            + " 'config' "
287            + colored("." * 61, color="dark_grey", no_color=no_color)
288            + " (index 0, address 0):"
289        )
290        register_value = self.read_config()
291        print(register_value.to_string(indentation="  ", no_color=no_color))
292        print()
293
294        print(
295            colored("Register", color="light_yellow", no_color=no_color)
296            + " 'status' "
297            + colored("." * 61, color="dark_grey", no_color=no_color)
298            + " (index 1, address 4):"
299        )
300        register_value = self.read_status()
301        formatted_value = _format_unsigned_number(value=register_value, width=32)
302        print(f"  {formatted_value}\n")
303
304        print(
305            colored("Register", color="light_yellow", no_color=no_color)
306            + " 'channels[0].read_address' "
307            + colored("." * 43, color="dark_grey", no_color=no_color)
308            + " (index 2, address 8):"
309        )
310        register_value = self.read_channels_read_address(array_index=0)
311        formatted_value = _format_unsigned_number(value=register_value, width=32)
312        print(f"  {formatted_value}\n")
313
314        print(
315            colored("Register", color="light_yellow", no_color=no_color)
316            + " 'channels[0].config' "
317            + colored("." * 49, color="dark_grey", no_color=no_color)
318            + " (index 3, address 12):"
319        )
320        print("  Not readable.\n")
321
322        print(
323            colored("Register", color="light_yellow", no_color=no_color)
324            + " 'channels[1].read_address' "
325            + colored("." * 43, color="dark_grey", no_color=no_color)
326            + " (index 4, address 16):"
327        )
328        register_value = self.read_channels_read_address(array_index=1)
329        formatted_value = _format_unsigned_number(value=register_value, width=32)
330        print(f"  {formatted_value}\n")
331
332        print(
333            colored("Register", color="light_yellow", no_color=no_color)
334            + " 'channels[1].config' "
335            + colored("." * 49, color="dark_grey", no_color=no_color)
336            + " (index 5, address 20):"
337        )
338        print("  Not readable.\n")
339
340        print(
341            colored("Register", color="light_yellow", no_color=no_color)
342            + " 'channels[2].read_address' "
343            + colored("." * 43, color="dark_grey", no_color=no_color)
344            + " (index 6, address 24):"
345        )
346        register_value = self.read_channels_read_address(array_index=2)
347        formatted_value = _format_unsigned_number(value=register_value, width=32)
348        print(f"  {formatted_value}\n")
349
350        print(
351            colored("Register", color="light_yellow", no_color=no_color)
352            + " 'channels[2].config' "
353            + colored("." * 49, color="dark_grey", no_color=no_color)
354            + " (index 7, address 28):"
355        )
356        print("  Not readable.\n")
357
358        print(
359            colored("Register", color="light_yellow", no_color=no_color)
360            + " 'channels[3].read_address' "
361            + colored("." * 43, color="dark_grey", no_color=no_color)
362            + " (index 8, address 32):"
363        )
364        register_value = self.read_channels_read_address(array_index=3)
365        formatted_value = _format_unsigned_number(value=register_value, width=32)
366        print(f"  {formatted_value}\n")
367
368        print(
369            colored("Register", color="light_yellow", no_color=no_color)
370            + " 'channels[3].config' "
371            + colored("." * 49, color="dark_grey", no_color=no_color)
372            + " (index 9, address 36):"
373        )
374        print("  Not readable.\n")
375
376
377def get_accessor(register_accessor: "PythonRegisterAccessorInterface") -> ExampleAccessor:
378    """
379    Factory function to create an accessor object.
380    """
381    return ExampleAccessor(register_accessor=register_accessor)
382
383
384def _format_unsigned_number(value: int, width: int, include_decimal: bool = True) -> str:
385    """
386    Format a string representation of the provided ``value``.
387    Suitable for printing field and register values.
388    """
389    result = ""
390    if include_decimal:
391        result += f"unsigned decimal {value}, "
392
393    hex_string = to_hex_byte_string(value=value, result_width_bits=width)
394    result += f"hexadecimal {hex_string}, "
395
396    binary_string = to_binary_nibble_string(value=value, result_width_bits=width)
397    result += f"binary {binary_string}"
398
399    return result
400
401
402@dataclass
403class ExampleConfigValue:
404    """
405    Represents the field values of the 'config' register.
406    Uses native Python types for easy handling.
407    """
408
409    class Direction(Enum):
410        """
411        Enumeration elements for the ``direction`` field.
412        """
413
414        DATA_IN = "data_in"
415        HIGH_Z = "high_z"
416        DATA_OUT = "data_out"
417
418    # Single bit. Range: 0-1.
419    enable: int
420    # Enumeration. Possible values: DATA_IN, HIGH_Z, DATA_OUT.
421    direction: Direction
422
423    def to_string(self, indentation: str = "", no_color: bool = False) -> str:
424        """
425        Get a string of the field values, suitable for printing.
426
427        Arguments:
428            indentation: Optionally indent each field value line.
429            no_color: Disable color output.
430                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
431        """
432        values = []
433
434        value = str(self.enable)
435        field_name = colored("enable", color="light_cyan", no_color=no_color)
436        values.append(f"{indentation}{field_name}: {value}")
437
438        enum_index = list(self.Direction).index(self.direction)
439        value = f"{self.direction.name} ({enum_index})"
440        field_name = colored("direction", color="light_cyan", no_color=no_color)
441        values.append(f"{indentation}{field_name}: {value}")
442
443        return "\n".join(values)
444
445
446@dataclass
447class ExampleChannelsConfigValue:
448    """
449    Represents the field values of the 'config' register within the 'channels' register array.
450    Uses native Python types for easy handling.
451    """
452
453    # Single bit. Range: 0-1.
454    enable: int
455    # Unsigned bit vector. Width: 8. Range: 0 - 255.
456    tuser: int
457
458    def to_string(self, indentation: str = "", no_color: bool = False) -> str:
459        """
460        Get a string of the field values, suitable for printing.
461
462        Arguments:
463            indentation: Optionally indent each field value line.
464            no_color: Disable color output.
465                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
466        """
467        values = []
468
469        value = str(self.enable)
470        field_name = colored("enable", color="light_cyan", no_color=no_color)
471        values.append(f"{indentation}{field_name}: {value}")
472
473        value_comment = _format_unsigned_number(value=self.tuser, width=8, include_decimal=False)
474        value = f"{self.tuser} ({value_comment})"
475        field_name = colored("tuser", color="light_cyan", no_color=no_color)
476        values.append(f"{indentation}{field_name}: {value}")
477
478        return "\n".join(values)