Skip to content

Fix the Tcl/Tk setup of the example's plot tests - #3

Open
ru551n wants to merge 6 commits into
VUnit:mainfrom
ru551n:fix-plot-examples
Open

ru551n wants to merge 6 commits into
VUnit:mainfrom
ru551n:fix-plot-examples

Conversation

@ru551n

@ru551n ru551n commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Lars reported that the matplotlib tests of the example fail with some simulators. A temporary CI job running them on Windows (Python 3.10 and 3.14, NVC and GHDL, prebuilt DLLs) reproduced it with Python 3.10:

_tkinter.TclError: Can't find a usable init.tcl in the following directories:
    {D:\a\_temp\venv\tcl\tcl8.6} ...

Tcl looks for its scripts next to the executable, which inside a simulation is the simulator. The example points TCL_LIBRARY/TK_LIBRARY to the Python installation, but used sys.prefix, which in a virtual environment is the environment and has no Tcl. The new tcl_library.py does the same with sys.base_prefix, and finds the scripts for any Tcl version and layout (tcl/ on Windows, lib/ elsewhere) instead of assuming 8.6. It sets nothing for a Tcl 9 that carries its scripts in its library. The testbench loads it with import_module_from_file, which works on every simulator (exec_file is not available on Riviera-PRO/Active-HDL). unset_tcl_installation is removed: environ = old_environ only rebinds a name, so it never restored anything.

A comment above the plot tests notes a second, separate problem on Windows: a simulator with a Tcl of its own (NVC, Questa) can crash when matplotlib opens a Tk window and Python has another Tcl version, because Pillow and matplotlib use the first Tcl they find in the process. The comment gives the workarounds: a Python with the simulator's Tcl version, or another backend such as MPLBACKEND=QtAgg. The proper fix belongs in Pillow and matplotlib.

The tests behave as before otherwise; Test simple plot still waits for its window to be closed. On Linux the tests passed before when the Python had no Tk, because matplotlib then falls back to the non-interactive Agg backend.

Tested:

  • Windows, the temporary CI job: GHDL on 3.10 and 3.14 opens the plot windows without TclError.
  • Linux with a Tk-enabled Python (TkAgg): both plot tests on NVC, GHDL and Questa; the rest of the example on NVC and GHDL (17/17).

Refs #5: fixes its Linux part. The Windows Tcl version clash of the plot tests stays open there.

🤖 Generated with Claude Code

ru551n and others added 2 commits September 27, 2026 10:32
On Windows in a virtual environment, Tk found no Tcl scripts: the example
pointed TCL_LIBRARY to sys.prefix, the virtual environment, instead of the
Python installation, and Tcl looks next to the executable, the simulator.
tcl_library.py now finds the scripts under sys.base_prefix, for any Tcl
version and layout, and sets nothing for a Tcl 9 that carries them in its
library. The revert of the variables was a no-op and is gone.

Test simple plot waited for its window to be closed, which hangs whenever
Tk works; it now closes the window after 5 s like Test advanced plot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It is how the test shows the plot; the timer changed that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ru551n ru551n changed the title Fix the plot tests of the example with a Tk window Fix the Tcl/Tk setup of the example's plot tests in a virtual environment Sep 27, 2026
@LarsAsplund

Copy link
Copy Markdown
Contributor

There is still a problem on my side:

# Attempting stack trace sig 11
# Signal caught: signo [11]
# vsim_stacktrace.vstf written
# Current time Sun Sep 27 10:58:34 2026
# Program = vsim
# Id = "2025.2"
# Version = "2025.05"
# Date = "May 31 2025"
# Platform = win64
# Signature = f904e1ac6172cb7e246298baaaab7d69
# End of Stack Trace


# ** Fatal: (SIGSEGV) Bad handle or reference.
#    Time: 0 ps  Iteration: 10  Process: /tb_example/test_runner File: C:/Users/larsa/AppData/Local/Python/pythoncore-3.14-64/Lib/site-packages/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd
# Fatal error in Subprogram p_exec at C:/Users/larsa/AppData/Local/Python/pythoncore-3.14-64/Lib/site-packages/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd line 306
#
# HDL call sequence:
# Stopped at C:/Users/larsa/AppData/Local/Python/pythoncore-3.14-64/Lib/site-packages/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd 306 Subprogram p_exec
# called from  C:/Users/larsa/AppData/Local/Python/pythoncore-3.14-64/Lib/site-packages/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd 459 Subprogram exec
#
#
# Test Run Failed!
#
# Stack trace result from 'tb' command
#  C:/Users/larsa/AppData/Local/Python/pythoncore-3.14-64/Lib/site-packages/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd 306 return [address 0x7ff4f6bb08d6] Subprogram p_exec
# called from  C:/Users/larsa/AppData/Local/Python/pythoncore-3.14-64/Lib/site-packages/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd 459 return [address 0x7ff4f6bf4789] Subprogram exec
#
#
# Surrounding code from 'see' command
#   301 :   ) return boolean is
#   302 :     constant logger : logger_t := get_logger(get_id(session));
#   303 :   begin
#   304 :     return p_begin(session, operation)
#   305 :       and p_succeeded(p_send(text), operation, logger)
# ->306 :       and p_succeeded(vpy_execute(is_file), operation, logger);
#   307 :   end;
#   308 :
#   309 :   impure function p_exec_file(
#   310 :     file_name : string; session : python_session_t := default_session
#
fail (P=0 S=0 F=1 T=1) lib.tb_example.Test simple plot (5.7 s)

==== Summary ===========================================
fail lib.tb_example.Test simple plot (5.7 s)
========================================================
pass 0 of 1
fail 1 of 1
========================================================
Total time was 5.7 s
Elapsed time was 5.7 s
========================================================
Some failed!

@ru551n

ru551n commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks! I think this is a different problem from the one this PR fixes, and not in the bridge: a clash between two Tcl versions in the same process, which Pillow (used by matplotlib's Tk backend) doesn't handle.

On Windows, Pillow finds the Tcl/Tk functions by walking all DLLs loaded in the process and taking the first one that exports Tcl_CreateCommand (load_tkinter_funcs in src/Tk/tkImaging.c). Inside a simulator that has its own Tcl, that is the simulator's Tcl, loaded when it started, not the one Python's tkinter uses. If the versions differ, Pillow calls the wrong Tcl with Python's interpreter and the process crashes. Questa then reports it inside p_exec, since that's the VHDL call that was running.

I reproduced it on Windows CI with NVC 1.22.1, which ships Tcl 9, and Python 3.10 with Tcl 8.6: an access violation, with the Python stack ending in PIL\ImageTk.py, _pyimagingtkcall. NVC with Python 3.14 and GHDL (no Tcl) with both Pythons pass. Your case looks like the same thing the other way around: Questa's own Tcl (8.6, I believe) against Python 3.14's Tcl 9.

Could you try these?

  1. Confirm it's Pillow. The Python stack at the crash should end in PIL\ImageTk.py:
    set PYTHONFAULTHANDLER=1
    python examples\embedded_python\run.py "*simple plot*"
    
  2. A matplotlib backend without Tk. If this works, the Tcl clash is the cause:
    pip install PySide6
    set MPLBACKEND=QtAgg
    python examples\embedded_python\run.py "*plot*"
    
  3. A Python whose Tcl version matches Questa's. Python 3.12 or 3.13 from python.org ships Tcl/Tk 8.6, so if Questa's Tcl is 8.6 both plot tests should pass with Tk there. You can check Questa's version with puts [info patchlevel] in the vsim console.

If this is confirmed, the proper fix belongs in Pillow: on Windows it should use the Tcl/Tk DLLs that _tkinter imports instead of the first match in the process. I can open an issue there with the reproduction.

ru551n and others added 2 commits September 27, 2026 11:26
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ru551n ru551n changed the title Fix the Tcl/Tk setup of the example's plot tests in a virtual environment Fix the Tcl/Tk setup of the example's plot tests Sep 27, 2026
Standalone Python builds (python-build-standalone) keep Tcl/Tk next to
libpython on Linux. Only the python executable finds them, through its
RPATH, so tkinter failed to load in any simulator embedding the
interpreter:

    ImportError: libtcl9tk9.0.so: cannot open shared object file

The libraries have no SONAME, so loading them from Python first does
not help. The package setup now adds that directory to LD_LIBRARY_PATH
of the VUnit process, like add_python_dll_to_path does for PATH on
Windows, and every simulator process inherits it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ru551n

ru551n commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Added: the Tcl/Tk libraries of the Python installation in every simulator (e343720)

tcl_library.py fixes where Tcl looks for its scripts. With a standalone Python build (python-build-standalone) on Linux, tkinter fails one step earlier, before any script is looked for:

ImportError: libtcl9tk9.0.so: cannot open shared object file: No such file or directory

Why. These builds keep the Tcl/Tk shared libraries next to libpython, and only the python executable finds them, through its RPATH ($ORIGIN/../lib). _tkinter has no search path of its own. In a simulator the executable is nvc, ghdl or vsim, so the libraries are not found, whatever TCL_LIBRARY says.

Why in the bridge, and why LD_LIBRARY_PATH. The libraries have no SONAME, so loading them from Python first with ctypes does not satisfy _tkinter (tried). They must be on the search path when the simulator process starts, which only the process starting it can arrange. setup() of the package now calls add_python_libraries_to_library_path(), which appends sys.base_prefix/lib to LD_LIBRARY_PATH of the VUnit process, once, on Linux, and only when that directory has libtcl*.so. Every simulator process inherits it, so no simulator specific hook is needed. This is the same approach as add_python_dll_to_path() for PATH on Windows. Distribution Pythons, macOS and Windows are left alone.

Why both changes are needed. With only the library path, the old set_tcl_installation still pointed TCL_LIBRARY at a tcl8.6 directory that a Tcl 9 Python does not have, and Tcl failed with Cannot find a usable init.tcl. With only tcl_library.py, the libraries do not load at all.

Tested on Linux with NVC 1.23-devel and GHDL 7.0.0-dev (mcode) and a python-build-standalone CPython 3.14, with no LD_LIBRARY_PATH or other workaround:

NVC GHDL
Test querying for randomization seed, without a display gets past Tcl, fails on "no display" (before: the error above) same
Example, --without-attributes .expected_failure --without-attributes .optional_deps 17/17 17/17
tests/run.py 106/106 106/106
pytest tests/test_python_bridge.py 86 passed

Not tested: Questa/ModelSim, Riviera-PRO, Active-HDL, macOS and Windows.

How to run the tests that need no extra Python packages and no user
input, and that the PySimpleGUI tests wait for answers to their
dialogs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ru551n added a commit that referenced this pull request Sep 27, 2026
The GitHub release of a version now says what release_notes/<version>.md
says when there is one, before the tag message and the generated notes.
The notes of 0.1.0 list the known Tcl/Tk issues: standalone Python builds
on Linux, fixed by #3, and the Tcl version clash on Windows, #5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants