Python code generator

The Python code generator creates an accessor class for reading, writing and printing register and field values. The class will have a named method for each of these operations on each register and field. The methods use native Python types (e.g. int, float, Enum) to represent values that are read or written. This means that the user never has to do any bit slicing or casting.

These can be used in a Python-based system test environment to conveniently perform register read/writes on the target device.

The artifacts are generated like this:

Python code that parses the example TOML file and generates Python register artifacts.
 1# Standard libraries
 2import sys
 3from pathlib import Path
 4
 5# First party libraries
 6from hdl_registers.generator.python.accessor import PythonAccessorGenerator
 7from hdl_registers.generator.python.pickle import PythonPickleGenerator
 8from hdl_registers.parser.toml import from_toml
 9
10THIS_DIR = Path(__file__).parent
11
12
13def main(output_folder: Path):
14    """
15    Create register Python artifacts from the TOML example file.
16    """
17    register_list = from_toml(
18        name="example",
19        toml_file=THIS_DIR.parent.parent / "user_guide" / "toml" / "toml_format.toml",
20    )
21
22    PythonPickleGenerator(register_list=register_list, output_folder=output_folder).create()
23
24    PythonAccessorGenerator(register_list=register_list, output_folder=output_folder).create()
25
26
27if __name__ == "__main__":
28    main(output_folder=Path(sys.argv[1]))

Pickle code

The script below is generated by the PythonPickleGenerator and can be used to re-create the RegisterList object from the Python pickle file.

Click to expand/collapse code.
Example Python pickle re-creation script
 1# This file is automatically generated by hdl-registers version 5.2.1-dev.
 2# Code generator PythonPickleGenerator version 1.0.0.
 3# Generated 2024-07-23 20:52 from file toml_format.toml at commit fde82ed5a49c96e3.
 4# Register hash 4df9765ebb584803b583628671e4659579eb85f4.
 5
 6# Standard libraries
 7import pickle
 8from pathlib import Path
 9from typing import TYPE_CHECKING
10
11if TYPE_CHECKING:
12    # Third party libraries
13    from hdl_registers.register_list import RegisterList
14
15THIS_DIR = Path(__file__).parent
16PICKLE_FILE = THIS_DIR / "example.pickle"
17
18
19class Example:
20    """
21    Instantiate this class to get the ``RegisterList`` object for the
22    ``example`` register list.
23    """
24
25    def __new__(cls):
26        """
27        Recreate the ``RegisterList`` object from binary pickle.
28        """
29        with PICKLE_FILE.open("rb") as file_handle:
30            return pickle.load(file_handle)
31
32
33def get_register_list() -> "RegisterList":
34    """
35    Return a ``RegisterList`` object with the registers/constants from the
36    ``example`` register list.
37    Recreated from a Python pickle file.
38    """
39    with PICKLE_FILE.open("rb") as file_handle:
40        return pickle.load(file_handle)

Accessor code

The class below is generated by the PythonAccessorGenerator for performing register and field operations on a target device. Interesting things to notice:

  1. Register read/write operations use a generated Python dataclass to represent a register with field value members. See e.g. read_config and write_config methods, and the ExampleConfigValue type in the generated code.

  2. Field values are represented using native Python types.

  1. A bit field is represented as a Python int.

  2. A bit vector field WITHOUT fractional bits, whether signed or unsigned, is represented as a Python int.

  3. A bit vector field WITH fractional bits, whether signed or unsigned, is represented as a Python float.

  4. An enumeration field is represented using a custom Python Enum type. See e.g. ExampleConfigValue.Direction.

  5. An integer field, whether signed or unsigned, is represented as a Python int.

  1. In cases where the register has no fields, the read and write use a plain integer instead. See e.g. read_status.

  2. Each writeable register has methods to write

    1. The entire register value (e.g. write_config), which takes a dataclass value as argument.

    2. Individual field values (e.g. write_config_direction), which takes the native Python field representation as argument.

    Field write methods will either read-modify-write or plain write the register, depending on what the register mode allows.

Printing values

Each register value type has a to_string method to convert the field values to a human-readable string. The accessor class also implements the print_registers method, which will print all register and field values:

../../_images/print_registers.png

Example register printout with many fields of different types.

Note how only readable registers are printed. Field values are present firstly as their native type, but also as their decimal, hexadecimal and binary encodings.

Color

By default, color output is enabled, since it improves readability when a lot of information is presented. It can be turned of by setting the no_color argument to print_registers, or by setting the environment variable NO_COLOR to any value.

Register accessor interface

The code generated by PythonAccessorGenerator performs the address calculation, mode logic, bit slicing, and type casting for you. It can not, however, implement the actual reading and writing of register values on your target device. This must be implemented by the user, given the methods of access that are available in your environment (SSH, telnet, UART, etc). Hdl-registers can not implement this in a generic way that is useful in all environments, since device setup is so different in different projects.

The class constructor takes a register_accessor argument of type PythonRegisterAccessorInterface. This object must be constructed by the user, must inherit the PythonRegisterAccessorInterface, and must implement the read_register and write_register methods. See the API documentation for more details.

Code

Example code generated by PythonAccessorGenerator. Using the generated code is done by

  1. Generating artifacts as shown here.

  2. Adding the generated Python module to your Python PATH.

  3. Instantiating the object, by either

  1. Class instantiation, e.g. accessor = ExampleAccessor(register_accessor=register_accessor), where “Example” in the class name is the name of the register list.

  2. Running the convenience function accessor = get_accessor(register_accessor=register_accessor) where the register list name is not needed anywhere.

Example Python accessor class
  1# This file is automatically generated by hdl-registers version 5.2.1-dev.
  2# Code generator PythonAccessorGenerator version 0.1.0.
  3# Generated 2024-07-23 20:52 from file toml_format.toml at commit fde82ed5a49c96e3.
  4# Register hash 4df9765ebb584803b583628671e4659579eb85f4.
  5
  6# Standard libraries
  7import pickle
  8from dataclasses import dataclass
  9from enum import Enum
 10from pathlib import Path
 11from typing import TYPE_CHECKING
 12from termcolor import colored
 13
 14# Third party libraries
 15from hdl_registers.field.numerical_interpretation import to_unsigned_binary
 16from tsfpga.math_utils import to_binary_nibble_string, to_hex_byte_string
 17
 18if TYPE_CHECKING:
 19    # Third party libraries
 20    from hdl_registers.generator.python.register_accessor_interface import (
 21        PythonRegisterAccessorInterface,
 22    )
 23    from hdl_registers.register_list import RegisterList
 24
 25
 26THIS_DIR = Path(__file__).parent
 27
 28
 29class ExampleAccessor:
 30    """
 31    Class to read/write registers and fields of the ``example`` register list.
 32    """
 33
 34    def __init__(self, register_accessor: "PythonRegisterAccessorInterface"):
 35        """
 36        Arguments:
 37            register_accessor: Object that implements register access methods in your system.
 38                Must have the methods ``read_register`` and ``write_register``
 39                with the same interface and behavior as the interface class
 40                :class:`.PythonRegisterAccessorInterface`.
 41                It is highly recommended that your accessor class inherits from that class.
 42        """
 43        self._register_accessor = register_accessor
 44
 45        pickle_path = THIS_DIR / "example.pickle"
 46        if not pickle_path.exists():
 47            raise FileNotFoundError(
 48                f"Could not find the pickle file {pickle_path}, "
 49                "make sure this artifact is generated."
 50            )
 51
 52        with pickle_path.open("rb") as file_handle:
 53            self._register_list: "RegisterList" = pickle.load(file_handle)
 54
 55    def _read_register(self, register_index: int) -> int:
 56        """
 57        Read a register value via the register accessor provided by the user.
 58        We perform sanity checks in both directions here, since this is the glue logic between
 59        two interfaces.
 60        """
 61        if not 0 <= register_index <= 9:
 62            raise ValueError(
 63                f"Register index {register_index} out of range. This is likely an internal error."
 64            )
 65
 66        register_value = self._register_accessor.read_register(
 67            register_list_name="example", register_address=4 * register_index
 68        )
 69        if not 0 <= register_value < 2**32:
 70            raise ValueError(
 71                f'Register read value "{register_value}" from accessor is out of range.'
 72            )
 73
 74        return register_value
 75
 76    def _write_register(self, register_index: int, register_value: int) -> None:
 77        """
 78        Write a register value via the register accessor provided by the user.
 79        We perform sanity checks in both directions here, since this is the glue logic between
 80        two interfaces.
 81        """
 82        if not 0 <= register_index <= 9:
 83            raise ValueError(
 84                f"Register index {register_index} out of range. This is likely an internal error."
 85            )
 86
 87        if not 0 <= register_value < 2**32:
 88            raise ValueError(f'Register write value "{register_value}" is out of range.')
 89
 90        self._register_accessor.write_register(
 91            register_list_name="example",
 92            register_address=4 * register_index,
 93            register_value=register_value,
 94        )
 95
 96    def read_config(self) -> "ExampleConfigValue":
 97        """
 98        Read the 'config' register.
 99        """
100        index = self._register_list.get_register_index(
101            register_name="config",
102            register_array_name=None,
103            register_array_index=None,
104        )
105        value = self._read_register(register_index=index)
106        register = self._register_list.get_register(
107            register_name="config", register_array_name=None
108        )
109        return ExampleConfigValue(
110            enable=register.fields[0].get_value(register_value=value),
111            direction=ExampleConfigValue.Direction(
112                register.fields[1].get_value(register_value=value).name
113            ),
114        )
115
116    def write_config(self, register_value: "ExampleConfigValue") -> None:
117        """
118        Write the whole 'config' register.
119
120        Arguments:
121            register_value: Object with the field values that shall be written.
122        """
123        index = self._register_list.get_register_index(
124            register_name="config",
125            register_array_name=None,
126            register_array_index=None,
127        )
128        register = self._register_list.get_register(
129            register_name="config", register_array_name=None
130        )
131        integer_value = 0
132        integer_value += register.fields[0].set_value(field_value=register_value.enable)
133        integer_value += register.fields[1].set_value(
134            field_value=register.fields[1].get_element_by_name(register_value.direction.value)
135        )
136        self._write_register(register_index=index, register_value=integer_value)
137
138    def write_config_enable(self, field_value: "int") -> None:
139        """
140        Write the 'enable' field in the 'config' register.
141        Will read-modify-write the ``config`` register, setting the ``enable`` field
142        to the provided value, while leaving the other fields at their previous values.
143
144        Arguments:
145            field_value: The field value to be written.
146                Single bit. Range: 0-1.
147        """
148        register_value = self.read_config()
149        register_value.enable = field_value
150        self.write_config(register_value=register_value)
151
152    def write_config_direction(self, field_value: "ExampleConfigValue.Direction") -> None:
153        """
154        Write the 'direction' field in the 'config' register.
155        Will read-modify-write the ``config`` register, setting the ``direction`` field
156        to the provided value, while leaving the other fields at their previous values.
157
158        Arguments:
159            field_value: The field value to be written.
160                Enumeration. Possible values: DATA_IN, HIGH_Z, DATA_OUT.
161        """
162        register_value = self.read_config()
163        register_value.direction = field_value
164        self.write_config(register_value=register_value)
165
166    def read_status(self) -> "int":
167        """
168        Read the 'status' register.
169        Return type is a plain unsigned integer since the register has no fields.
170        """
171        index = self._register_list.get_register_index(
172            register_name="status",
173            register_array_name=None,
174            register_array_index=None,
175        )
176        value = self._read_register(register_index=index)
177        return value
178
179    def read_channels_read_address(self, array_index: int) -> "int":
180        """
181        Read the 'read_address' register within the 'channels' register array.
182        Return type is a plain unsigned integer since the register has no fields.
183
184        Arguments:
185            array_index: Register array iteration index (range 0-3).
186        """
187        index = self._register_list.get_register_index(
188            register_name="read_address",
189            register_array_name="channels",
190            register_array_index=array_index,
191        )
192        value = self._read_register(register_index=index)
193        return value
194
195    def write_channels_read_address(self, register_value: "int", array_index: int) -> None:
196        """
197        Write the whole 'read_address' register within the 'channels' register array.
198
199        Arguments:
200            register_value: Value to be written.
201                Is a plain unsigned integer since the register has no fields.
202            array_index: Register array iteration index (range 0-3).
203        """
204        index = self._register_list.get_register_index(
205            register_name="read_address",
206            register_array_name="channels",
207            register_array_index=array_index,
208        )
209        self._write_register(register_index=index, register_value=register_value)
210
211    def write_channels_config(
212        self, register_value: "ExampleChannelsConfigValue", array_index: int
213    ) -> None:
214        """
215        Write the whole 'config' register within the 'channels' register array.
216
217        Arguments:
218            register_value: Object with the field values that shall be written.
219            array_index: Register array iteration index (range 0-3).
220        """
221        index = self._register_list.get_register_index(
222            register_name="config",
223            register_array_name="channels",
224            register_array_index=array_index,
225        )
226        register = self._register_list.get_register(
227            register_name="config", register_array_name="channels"
228        )
229        integer_value = 0
230        integer_value += register.fields[0].set_value(field_value=register_value.enable)
231        integer_value += register.fields[1].set_value(field_value=register_value.tuser)
232        self._write_register(register_index=index, register_value=integer_value)
233
234    def write_channels_config_enable(self, field_value: "int", array_index: int) -> None:
235        """
236        Write the 'enable' field in the 'config' register within the 'channels' register array.
237        Will write the ``config`` register, setting the ``enable`` field to the
238        provided value, and all other fields to their default values.
239
240        Arguments:
241            field_value: The field value to be written.
242                Single bit. Range: 0-1.
243            array_index: Register array iteration index (range 0-3).
244        """
245        index = self._register_list.get_register_index(
246            register_name="config",
247            register_array_name="channels",
248            register_array_index=array_index,
249        )
250
251        register = self._register_list.get_register(
252            register_name="config", register_array_name="channels"
253        )
254        register_value = 0
255        register_value += register.fields[0].set_value(field_value)
256        register_value += register.fields[1].default_value_uint << register.fields[1].base_index
257
258        self._write_register(register_index=index, register_value=register_value)
259
260    def write_channels_config_tuser(self, field_value: "int", array_index: int) -> None:
261        """
262        Write the 'tuser' field in the 'config' register within the 'channels' register array.
263        Will write the ``config`` register, setting the ``tuser`` field to the
264        provided value, and all other fields to their default values.
265
266        Arguments:
267            field_value: The field value to be written.
268                Unsigned bit vector. Width: 8. Range: 0 - 255.
269            array_index: Register array iteration index (range 0-3).
270        """
271        index = self._register_list.get_register_index(
272            register_name="config",
273            register_array_name="channels",
274            register_array_index=array_index,
275        )
276
277        register = self._register_list.get_register(
278            register_name="config", register_array_name="channels"
279        )
280        register_value = 0
281        register_value += register.fields[0].default_value_uint << register.fields[0].base_index
282        register_value += register.fields[1].set_value(field_value)
283
284        self._write_register(register_index=index, register_value=register_value)
285
286    def print_registers(self, no_color: bool = False) -> None:
287        """
288        Print all registers and their values to STDOUT.
289
290        Arguments:
291            no_color: Disable color output.
292                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
293        """
294        print(
295            colored("Register", color="light_yellow", no_color=no_color)
296            + " 'config' "
297            + colored("." * 61, color="dark_grey", no_color=no_color)
298            + " (index 0, address 0):"
299        )
300        register_value = self.read_config()
301        print(register_value.to_string(indentation="  ", no_color=no_color))
302        print()
303
304        print(
305            colored("Register", color="light_yellow", no_color=no_color)
306            + " 'status' "
307            + colored("." * 61, color="dark_grey", no_color=no_color)
308            + " (index 1, address 4):"
309        )
310        register_value = self.read_status()
311        formatted_value = _format_unsigned_number(value=register_value, width=32)
312        print(f"  {formatted_value}\n")
313
314        print(
315            colored("Register", color="light_yellow", no_color=no_color)
316            + " 'channels[0].read_address' "
317            + colored("." * 43, color="dark_grey", no_color=no_color)
318            + " (index 2, address 8):"
319        )
320        register_value = self.read_channels_read_address(array_index=0)
321        formatted_value = _format_unsigned_number(value=register_value, width=32)
322        print(f"  {formatted_value}\n")
323
324        print(
325            colored("Register", color="light_yellow", no_color=no_color)
326            + " 'channels[0].config' "
327            + colored("." * 49, color="dark_grey", no_color=no_color)
328            + " (index 3, address 12):"
329        )
330        print("  Not readable.\n")
331
332        print(
333            colored("Register", color="light_yellow", no_color=no_color)
334            + " 'channels[1].read_address' "
335            + colored("." * 43, color="dark_grey", no_color=no_color)
336            + " (index 4, address 16):"
337        )
338        register_value = self.read_channels_read_address(array_index=1)
339        formatted_value = _format_unsigned_number(value=register_value, width=32)
340        print(f"  {formatted_value}\n")
341
342        print(
343            colored("Register", color="light_yellow", no_color=no_color)
344            + " 'channels[1].config' "
345            + colored("." * 49, color="dark_grey", no_color=no_color)
346            + " (index 5, address 20):"
347        )
348        print("  Not readable.\n")
349
350        print(
351            colored("Register", color="light_yellow", no_color=no_color)
352            + " 'channels[2].read_address' "
353            + colored("." * 43, color="dark_grey", no_color=no_color)
354            + " (index 6, address 24):"
355        )
356        register_value = self.read_channels_read_address(array_index=2)
357        formatted_value = _format_unsigned_number(value=register_value, width=32)
358        print(f"  {formatted_value}\n")
359
360        print(
361            colored("Register", color="light_yellow", no_color=no_color)
362            + " 'channels[2].config' "
363            + colored("." * 49, color="dark_grey", no_color=no_color)
364            + " (index 7, address 28):"
365        )
366        print("  Not readable.\n")
367
368        print(
369            colored("Register", color="light_yellow", no_color=no_color)
370            + " 'channels[3].read_address' "
371            + colored("." * 43, color="dark_grey", no_color=no_color)
372            + " (index 8, address 32):"
373        )
374        register_value = self.read_channels_read_address(array_index=3)
375        formatted_value = _format_unsigned_number(value=register_value, width=32)
376        print(f"  {formatted_value}\n")
377
378        print(
379            colored("Register", color="light_yellow", no_color=no_color)
380            + " 'channels[3].config' "
381            + colored("." * 49, color="dark_grey", no_color=no_color)
382            + " (index 9, address 36):"
383        )
384        print("  Not readable.\n")
385
386
387def get_accessor(register_accessor: "PythonRegisterAccessorInterface") -> ExampleAccessor:
388    """
389    Factory function to create an accessor object.
390    """
391    return ExampleAccessor(register_accessor=register_accessor)
392
393
394def _format_unsigned_number(value: int, width: int, include_decimal: bool = True) -> str:
395    """
396    Format a string representation of the provided ``value``.
397    Suitable for printing field and register values.
398    """
399    result = ""
400    if include_decimal:
401        result += f"unsigned decimal {value}, "
402
403    hex_string = to_hex_byte_string(value=value, result_width_bits=width)
404    result += f"hexadecimal {hex_string}, "
405
406    binary_string = to_binary_nibble_string(value=value, result_width_bits=width)
407    result += f"binary {binary_string}"
408
409    return result
410
411
412@dataclass
413class ExampleConfigValue:
414    """
415    Represents the field values of the 'config' register.
416    Uses native Python types for easy handling.
417    """
418
419    class Direction(Enum):
420        """
421        Enumeration elements for the ``direction`` field.
422        """
423
424        DATA_IN = "data_in"
425        HIGH_Z = "high_z"
426        DATA_OUT = "data_out"
427
428    # Single bit. Range: 0-1.
429    enable: int
430    # Enumeration. Possible values: DATA_IN, HIGH_Z, DATA_OUT.
431    direction: Direction
432
433    def to_string(self, indentation: str = "", no_color: bool = False) -> str:
434        """
435        Get a string of the field values, suitable for printing.
436
437        Arguments:
438            indentation: Optionally indent each field value line.
439            no_color: Disable color output.
440                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
441        """
442        values = []
443
444        value = str(self.enable)
445        field_name = colored("enable", color="light_cyan", no_color=no_color)
446        values.append(f"{indentation}{field_name}: {value}")
447
448        enum_index = list(self.Direction).index(self.direction)
449        value = f"{self.direction.name} ({enum_index})"
450        field_name = colored("direction", color="light_cyan", no_color=no_color)
451        values.append(f"{indentation}{field_name}: {value}")
452
453        return "\n".join(values)
454
455
456@dataclass
457class ExampleChannelsConfigValue:
458    """
459    Represents the field values of the 'config' register within the 'channels' register array.
460    Uses native Python types for easy handling.
461    """
462
463    # Single bit. Range: 0-1.
464    enable: int
465    # Unsigned bit vector. Width: 8. Range: 0 - 255.
466    tuser: int
467
468    def to_string(self, indentation: str = "", no_color: bool = False) -> str:
469        """
470        Get a string of the field values, suitable for printing.
471
472        Arguments:
473            indentation: Optionally indent each field value line.
474            no_color: Disable color output.
475                Can also be achieved by setting the environment variable ``NO_COLOR`` to any value.
476        """
477        values = []
478
479        value = str(self.enable)
480        field_name = colored("enable", color="light_cyan", no_color=no_color)
481        values.append(f"{indentation}{field_name}: {value}")
482
483        value_comment = _format_unsigned_number(value=self.tuser, width=8, include_decimal=False)
484        value = f"{self.tuser} ({value_comment})"
485        field_name = colored("tuser", color="light_cyan", no_color=no_color)
486        values.append(f"{indentation}{field_name}: {value}")
487
488        return "\n".join(values)