Skip to content

Find the Tcl/Tk of the Python installation in every simulator - #4

Closed
ru551n wants to merge 1 commit into
mainfrom
tcl-library-paths
Closed

ru551n wants to merge 1 commit into
mainfrom
tcl-library-paths

Conversation

@ru551n

@ru551n ru551n commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

With a standalone Python build (python-build-standalone) on Linux, every test using tkinter (the PySimpleGUI dialogs, Matplotlib windows) fails inside the simulator, in two ways:

ImportError: libtcl9tk9.0.so: cannot open shared object file: No such file or directory
_tkinter.TclError: Cannot find a usable init.tcl in the following directories: ...
  1. The Tcl/Tk shared libraries are not found. These builds keep them next to libpython and find them only through the RPATH of the python executable ($ORIGIN/../lib). In a simulator the executable is nvc, ghdl or vsim, so _tkinter cannot load them. They have no SONAME, so preloading them from Python with ctypes does not help either (tried).
  2. The example points Tcl at a directory that does not exist. set_tcl_installation in tb_example.vhd always set TCL_LIBRARY/TK_LIBRARY to tcl8.6/tk8.6 under sys.prefix. In a virtual environment sys.prefix is the environment rather than the Python installation, and a Python with Tcl 9 has no tcl8.6. Without the override Tcl finds its libraries itself. unset_tcl_installation (old_environ = environ) kept a reference to the same object, so nothing was restored.

Changes

  • native_library.add_python_libraries_to_library_path(), called from the package setup() for every simulator: on Linux, when sys.base_prefix/lib has libtcl*.so, that directory is appended once to LD_LIBRARY_PATH of the VUnit process, which every simulator process inherits. The same approach as add_python_dll_to_path() for PATH on Windows. Distribution Pythons, macOS and Windows are left alone.
  • examples/embedded_python/tcl_installation.py: the Tcl/Tk setup of the example as a module (see the comment below for why), imported by set_tcl_installation.
  • A unit test of the new function.

Tested

Linux, NVC 1.23-devel and GHDL 7.0.0-dev (mcode), a python-build-standalone CPython 3.14, no LD_LIBRARY_PATH or other workaround:

NVC GHDL
PySimpleGUI test without a display gets past Tcl, fails on "no display" (before: the Tcl errors above) same
Example, --without-attributes .expected_failure --without-attributes .optional_deps 17/17 17/17
tests/run.py 106/106 106/106
pytest tests/test_python_bridge.py 86 passed

Tk with a display found its scripts (Tk 9.0.4) with the same lookup code. Not tested: Questa/ModelSim, Riviera-PRO, Active-HDL, macOS and Windows.

🤖 Generated with Claude Code

Standalone Python builds (python-build-standalone) keep Tcl/Tk next to
libpython on Linux. Only the python executable finds them, through its
RPATH, so tkinter failed to load in any simulator embedding the
interpreter. The libraries have no SONAME, so loading them from Python
first does not help. The package setup now adds that directory to
LD_LIBRARY_PATH of the VUnit process, like add_python_dll_to_path does
for PATH on Windows, and every simulator process inherits it.

The embedded_python example pointed TCL_LIBRARY and TK_LIBRARY at a
tcl8.6/tk8.6 directory under sys.prefix whether it existed or not. In a
virtual environment sys.prefix is not the Python installation, and a
Python with Tcl 9 has no tcl8.6 directory, so Tcl stopped finding its
own libraries. unset_tcl_installation did not restore anything. The
logic moves to tcl_installation.py: it uses sys.base_prefix, accepts
any Tcl version, only sets a variable to a directory that has the Tcl
or Tk scripts, and restores the previous values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ru551n

ru551n commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Why tcl_installation.py

Why the example still sets TCL_LIBRARY/TK_LIBRARY at all. The comment above the call says it was added to avoid a mixup with the Tcl installation of Riviera-PRO. Riviera-PRO, Active-HDL and Questa run a Tcl 8.6 of their own in the simulator process. A Python whose tkinter also uses Tcl 8.6 (python.org on Windows, older builds) asks for a library of the same name, can be given the simulator's copy, and that Tcl then looks for its scripts in the simulator's installation. Pointing the variables at the scripts of the Python installation avoids that. I could not test those simulators, so this PR keeps the behaviour for them instead of removing it.

What changed in the behaviour. The old code always set the variables, to tcl8.6/tk8.6 under sys.prefix, whether the directories existed or not. The new code:

  • looks under sys.base_prefix, the Python installation, since sys.prefix is the virtual environment;
  • accepts any Tcl version (tcl[0-9]*, tk[0-9]*) in lib (Linux, macOS) or tcl (Windows);
  • only sets a variable to a directory that has the Tcl (init.tcl) or Tk (tk.tcl) scripts, and otherwise leaves Tcl to find its libraries itself;
  • restores the previous values in unset_tcl_installation (the old old_environ = environ kept a reference to the same object).

Why a module rather than exec strings in the testbench. With the loops and conditions above, the Python no longer fits the one-line exec style of the rest of the testbench. As VHDL string concatenation with indentation written by hand it cannot be linted, formatted or read easily, and a mistake only shows up as a traceback inside the simulator. This is the same reason device_model.py gives for keeping a model in a module.

Why it works on every simulator. set_tcl_installation runs at the start of every test, so it must work with the VHPI package of Riviera-PRO/Active-HDL too, which has only part of the API. It uses import_module_from_file, which is built on exec, and exec for the calls. call is not used because the VHPI package does not have it.

It stays in the example because it only concerns tkinter, which the bridge itself does not use. If users' own tkinter code needs the same, it could move into the runtime of the bridge.

@ru551n

ru551n commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #3, which fixes the example's Tcl/Tk setup with tcl_library.py. The bridge part of this PR, adding the Python installation's lib directory to LD_LIBRARY_PATH, moves there.

@ru551n ru551n closed this Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant