[tripleaceme]

Summary

The Wisdom converter kept relationship types that Ossie's direction can't carry as text in the relationship's ai_context, and read them back to restore relationshipType. As apache/ossie-converters#29 describes, that made the note text a wire format, misread or ignored hand-written notes, and put a one-to-many relationship note on a relationship whose from is the many side.

This moves the type into a WISDOM entry in custom_extensions, following the plan agreed on the issue.

Wisdom → Ossie

• Every relationship that isn't MANY_TO_ONE gets a WISDOM entry with its original relationshipType and the from/to datasets it was written with:

  custom_extensions:
    - vendor_name: WISDOM
      data: '{"_v": 1, "normalizedEndpoints": ["orders", "customers"], "relationshipType": "ONE_TO_MANY"}'

  The key names follow the Microsoft converter (normalizedEndpoints, _v) and Wisdom's own relationshipType.
• The one-to-many relationship note is no longer written. The one-to-one and many-to-many notes stay as plain ai_context for readers, but the export no longer depends on them.

Ossie → Wisdom

• The type, and the left/right and column swap for ONE_TO_MANY, are restored from the WISDOM entry.
• If the entry is malformed, has an unknown _v, or one of its recorded endpoints is no longer the relationship's from or to (it was moved to another dataset after import), the edge is exported as MANY_TO_ONE with a new RELATIONSHIP_TYPE_UNRESOLVED warning. None of these cases raise. Reversing from/to is not stale: ONE_TO_ONE and MANY_TO_MANY are symmetric, and a reversed ONE_TO_MANY still describes the relationship once its sides are swapped back.
• Entries the converter reads are no longer reported as CUSTOM_EXTENSION_DROPPED. Other vendors' entries still are.
• Without a WISDOM entry, the exact (whitespace-stripped) notes from RELATIONSHIP_TYPE_NOTES are still read, including the old one-to-many note, so documents generated before this change still round-trip. A note that states the exported type is no longer reported as AI_CONTEXT_DROPPED; any other note still is.

The payload is encoded and decoded in one pair of helpers (encode_extension_data / decode_extension_data), so if #522 changes custom_extensions[].data to an object, only those two functions change.

The README's cardinality table and export section are updated.

For reviewers to decide

1. A stale or malformed entry is reported once, as RELATIONSHIP_TYPE_UNRESOLVED, not also as CUSTOM_EXTENSION_DROPPED.
2. When an entry and a conflicting note are both present, the entry wins and the note is reported as AI_CONTEXT_DROPPED.
3. normalizedEndpoints stores dataset names only. An edit that changes only the join columns isn't treated as stale; it doesn't affect which side is swapped.

If apache/ossie-converters#1 lands before this, I'll re-open it there.

Related Issues

Closes apache/ossie-converters#29

Tests

Wisdom → Ossie:

• ONE_TO_MANY, ONE_TO_ONE and MANY_TO_MANY get the entry with the exact stored payload; MANY_TO_ONE gets none
• ONE_TO_MANY has no note; ONE_TO_ONE and MANY_TO_MANY keep theirs
• a non-string relationshipType still converts as before

Ossie → Wisdom:

• the entry restores each type, and the ONE_TO_MANY swap with distinct column names on each side

• reversed ONE_TO_ONE, MANY_TO_MANY and ONE_TO_MANY keep their type with no warning

• an endpoint moved to another dataset, and malformed entries (bad JSON, deeply nested JSON, non-object, unknown type, bad endpoints, missing endpoints, _v of 2 or "1") export MANY_TO_ONE with the warning

• a read entry isn't reported as dropped, while an unrelated vendor's entry still is

• legacy notes with no entry restore all three types

• the entry wins over a conflicting note, and a stale entry doesn't fall back to the note

• the existing full round trip on sample_export.json passes unchanged

• cd converters/wisdom && uv run pytest: 63 passed (37 on main)

[jbonofre]

Because this compares the endpoints in order, a reversed relationships count as stale. For ONE_TO_ONE and MANY_TO_MANY that's a regression. The README says ONE_TO_ONE direction is arbitrary, but it you swap from/to on an imported one, it now exports as MANY_TO_ONE with RELATIONSHIP_TYPE_UNRESOLVED, plus AI_CONTEXT_DROPPED for the note. On main, the note alone kept ONE_TO_ONE after the same edit.

A reversed ONE_TO_MANY isn't stale either. Applying it and swapping the sides back means the same as MANY_TO_ONE in the new direction, so the warning reports a loss that didn't happen.

Accepting both orders fixes all three cases, and an endpoint moved to a different dataset is still caught:

Suggested change

	# Edited endpoints may no longer match the stored type (e.g. a ONE_TO_MANY swap), so it is not applied.
	if data.get("normalizedEndpoints") != [relationship.from_dataset, relationship.to]:
	return None
	# Reversing from/to keeps the meaning: ONE_TO_ONE and MANY_TO_MANY are symmetric and
	# ONE_TO_MANY is swapped back below. Only endpoints moved to other datasets are stale.
	endpoints = [relationship.from_dataset, relationship.to]
	if data.get("normalizedEndpoints") not in (endpoints, endpoints[::-1]):
	return None

test_wisdom_extension_with_stale_endpoints_exports_many_to_one then needs endpoints that point to a different dataset (e.g. ["orders", "users"], as in test_note_is_not_read_when_wisdom_extension_is_stale).

Could we also add a test for a reversed ONE_TO_ONE and MANY_TO_MANY?

[tripleaceme]

You're right, and it was a regression against main for the symmetric types. Applied your suggestion in 1c9bb5c, so the recorded endpoints are accepted in either order. The stale test now uses ["orders", "users"]. I added tests for a reversed ONE_TO_ONE and MANY_TO_MANY, each with its note, that keep the type with no issues, and one for a reversed ONE_TO_MANY that exports customers ONE_TO_MANY orders with no warning. I changed "swapped back below" in the comment to "swapped back on export", since the swap happens in _convert_relationships, not in this helper. The README and PR description are updated to match.

[jbonofre]

json.loads raises RecursionError on deeply nested input, and that isn't a ValueError. So data = "[" * 200_000 crashes the export with a traceback, instead of giving the RELATIONSHIP_TYPE_UNRESOLVED warning that other malformed input gets:

Suggested change

	except (TypeError, ValueError):
	except (TypeError, ValueError, RecursionError):

Could we add "[" * 200_000 to the cases in test_malformed_wisdom_extension_exports_many_to_one?

[tripleaceme]

Applied in 1c9bb5c, and "[" * 200_000 is now one of the malformed cases. I gave it the id deeply-nested so pytest doesn't print the 200,000-character string as the test name.

[jbonofre]

@tripleaceme as you probably know (and it has been shared on the dev mailing list), the converters have moved to a dedicated repository: https://github.com/apache/ossie-converters

I propose you move this PR (create) on the new repo and we will resume the review together there.

Thanks!

[tripleaceme]

Thanks @jbonofre. Moved to apache/ossie-converters#9. It has the same two commits with the converters/ prefix dropped, and the wisdom tests pass there (63) against the pinned ossie submodule.