Skip to content

Support PostgreSQL; add automated tests and CI - #227

Merged
codeling merged 4 commits into
mainfrom
claude/busy-knuth-m2ipx0
Sep 30, 2026
Merged

codeling merged 4 commits into
mainfrom
claude/busy-knuth-m2ipx0

Conversation

@codeling

@codeling codeling commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Fixes #206. Companion PR: codeling/com_bfstop#13 (same branch name).

Problem

On Joomla with PostgreSQL, the plugin installed "successfully" but created no tables, because only MySQL install SQL existed. Even with tables in place, most queries used MySQL-only SQL (DATE_SUB/DATE_ADD … INTERVAL, LOCATE, REGEXP, INET_ATON, INET6_ATON, CONV, CAST … AS SIGNED, a table alias in UPDATE … SET). DatabaseHelper catches every database exception and returns 0/false/[], so the plugin failed open: it never blocked anything.

Changes: PostgreSQL support

  • SQL files: sql/install.postgresql.utf8.sql, sql/uninstall.postgresql.utf8.sql and a PostgreSQL schema path sql/updates/postgresql/ (baseline 2.0.0.sql), all wired into bfstop.xml. They include the username statistics table from Username statistics  #136 and the failedlogin (username, logtime) index. Differences from the MySQL schema:
    • handled is smallint, because the code compares it with 0/1, which PostgreSQL doesn't allow for boolean.
    • No unsigned integer types.
    • ipaddress is varchar(49) in bannedip/allowlist, as on MySQL installs upgraded through 1.2.0.sql.
  • IP subnet matching now runs in PHP (IpHelper::isInSubnet) instead of through MySQL-only SQL:
  • Time windows: the fixed ones are computed in PHP. The two column-based "crdate + duration minutes" expressions get a PostgreSQL variant via addMinutesSql(). Joomla's DatabaseQuery::dateAdd() can't be used for these, because on PostgreSQL it doesn't accept a column as the duration.
  • Username statistics (Username statistics  #136, merged from main): the upsert uses INSERT … ON CONFLICT (username) DO UPDATE on PostgreSQL, since ON DUPLICATE KEY UPDATE is MySQL-only.
  • Smaller fixes:
    • saveParams(): removed the table alias in UPDATE … SET, which PostgreSQL rejects.
    • updatescript.php: the old whitelist → allowlist rename now only runs on MySQL.

Changes: automated tests and CI

There was no CI before. tests/README.md explains how to run everything locally.

  • tests/Unit: subnet matching (moved from unittests/).
  • tests/Integration: runs against a real Joomla site with plugin and component installed through Joomla's own extension installer:
    • InstallTest: the installer created all tables, enabled the plugin and recorded the schema version (Joomla reports success even when it created nothing).
    • DatabaseHelperTest: every DatabaseHelper query, checked for correct results: counting, exact/expired/unlimited/unblocked blocks, IPv4/IPv6 subnets, corrupted entries, allow list, tokens, known IPs, DNS cache, username statistics, purging, saveParams.
    • ComponentTest: the component's database access, including saving new blocks and allow-list entries through the admin models, email-token unblocking, the username statistics view and the "admin" user warning.
    • PluginEventsTest: the plugin as Joomla runs it. Failed-login events lead to a block, and a request from the blocked IP or subnet is rejected (run as a separate PHP process, since the plugin exits). The allow list takes precedence, and a successful login records a known IP.
    • Every integration test fails if the code under test logged an error, because the plugin turns database exceptions into log entries, which would otherwise look like "no matching rows".
  • tests/ci/: start-database.sh starts a database container. install-joomla.sh downloads Joomla, installs it on that database and installs plugin and component from the checkouts.
  • .github/workflows/ci.yml:
    • Syntax check on PHP 8.1 and 8.4, plus the unit tests.
    • Integration tests on Joomla 5.4.8 (MySQL 8.0, MariaDB 10.11, PostgreSQL 12) and 6.1.3 (MySQL 8.4, MariaDB 11.4, PostgreSQL 17).
    • The component is checked out from its branch of the same name if one exists, otherwise main. The Joomla versions are pinned in the matrix.
    • Uses actions/checkout@v7, which runs on Node 24.
  • Removed unittests/cryptotest.php: it referred to classes and files that no longer exist. TokenHelperTest replaces it.

Testing

  • Locally, Joomla 5.4.8 and 6.1.3 (release packages), each on PostgreSQL 16 and MariaDB 10.11, after merging main: all 25 tests pass.
  • postgres:12 in Docker: ran both repos' workflow integration jobs locally by extracting their run: steps from ci.yml; all pass.
  • Negative checks:
    • On a PostgreSQL site set up with the code from before these PRs, every test fails, including InstallTest ("array contains jos_bfstop_failedlogin").
    • With ON DUPLICATE KEY UPDATE also used on PostgreSQL, the username statistics tests fail.
  • Component bug found by the new tests: ComponentTest showed that saving a new block or allow-list entry from the backend failed on PostgreSQL. The fix is in Support PostgreSQL; add CI com_bfstop#13.
  • Static checks: actionlint and shellcheck are clean.
  • Not run locally: MySQL 8.x (only MariaDB was available here); CI covers it.

Notes / not changed

  • The per-username failed-login counter, the known-IP lookup and the username statistics compare usernames with the database's own rules: case-sensitive on PostgreSQL, case-insensitive on MySQL. That matches Joomla's own login on each database.
  • Existing, not addressed here: the MySQL fresh-install schema uses varchar(45) for bannedip/allowlist ipaddress, while upgraded MySQL installs have varchar(49).

🤖 Generated with Claude Code

https://claude.ai/code/session_01LsFEnqepXD2eL6yLVZmSB9

On a Joomla site running on PostgreSQL, the plugin installed "successfully"
without creating any tables (only MySQL install SQL existed), and even with
the tables in place most queries used MySQL-only SQL. All database errors
are caught and turned into 0/false/empty results, so the plugin silently
never blocked anything.

- Add PostgreSQL install/uninstall SQL and a PostgreSQL update schema path
  (starting at 2.0.0)
- Match IP subnets (CIDR) of block and allow list entries in PHP
  (IpHelper::isInSubnet) instead of with INET_ATON/INET6_ATON/LOCATE/REGEXP;
  only the literal address match remains in SQL, compared case-insensitively
  as MySQL's collations did
- Compute fixed time windows (DATE_SUB/DATE_ADD with constant intervals) in
  PHP; the two column-based "crdate + duration minutes" expressions get a
  PostgreSQL variant
- saveParams: no table alias in UPDATE ... SET (not allowed in PostgreSQL)
- updatescript: only try the old whitelist rename on MySQL
- insertFailedLogin: catch database exceptions like the other methods
- Add unit test for the subnet matching

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsFEnqepXD2eL6yLVZmSB9
- tests/Unit: IpHelper subnet matching (moved from unittests/)
- tests/Integration: run against a real Joomla site with bfstop installed:
  installer result (tables, schema version), every DatabaseHelper query,
  the component's database access (if COM_BFSTOP_ROOT is set), and the
  plugin reacting to Joomla events (failed logins leading to a block,
  requests from blocked IPs/subnets being rejected, allow list). Tests fail
  if the code under test logged an error, since the plugin turns database
  exceptions into log entries.
- tests/ci: scripts to start a database container and to set up a Joomla
  site with plugin and component installed from the checkouts
- CI workflow: syntax check on PHP 8.1 and 8.4, unit tests, and the
  integration tests on Joomla 5.4 and 6.1 with MySQL 8.0/8.4,
  MariaDB 10.11/11.4 and PostgreSQL 12/17
- Remove unittests/cryptotest.php, which referred to no longer existing
  classes and files; replaced by tests/Integration/TokenHelperTest.php

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsFEnqepXD2eL6yLVZmSB9
@codeling codeling changed the title Support PostgreSQL Support PostgreSQL; add automated tests and CI Sep 27, 2026
v4 runs on the deprecated Node.js 20 runtime, which GitHub warned about;
v7 runs on Node.js 24.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsFEnqepXD2eL6yLVZmSB9
Merge the per-username failed login statistics (#136) and the utf8mb4
change from main, and make the statistics work on PostgreSQL too:

- install/uninstall.postgresql.utf8.sql: add the #__bfstop_username_stats
  table and the #__bfstop_failedlogin (username, logtime) index
- recordUsernameAttempt: use INSERT ... ON CONFLICT DO UPDATE on PostgreSQL,
  as ON DUPLICATE KEY UPDATE is MySQL-only
- tests: cover the statistics in the plugin (counting, not purged) and the
  component's UsernamestatsModel

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsFEnqepXD2eL6yLVZmSB9
@codeling
codeling merged commit af65298 into main Sep 30, 2026
8 checks passed
@codeling
codeling deleted the claude/busy-knuth-m2ipx0 branch October 1, 2026 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unknown support regarding postgresql

2 participants