Problems with "reconcile" design, remove restrictions around enums and keys, add dedicated filter

Feature Ideas
1

Reposting from Better reconciliation primitive, upsert isn't enough · Issue #18 · stalwartlabs/cli · GitHub

A match key is required. reconcile refuses the by-value fallback that upsert allows: an operation with no matchOn on a type that also has no label property is rejected at plan time. Value matching treats any drifted field as a non-match, which under reconcile would delete the existing object and recreate it under a new id; requiring an explicit matchOn (or a label property) makes that impossible to trigger by accident.

Deletion is scoped per variant. For multi-variant types, only the @type variants that actually appear in the plan value are eligible for deletion. A reconcile of MtaRoute that lists only Local routes never touches Mx routes, and an empty-value reconcile on a multi-variant type deletes nothing (no variant is named).

What is the reasoning for that?
I think those restrictions should be removed.

First one unnecessarily removes the possibility of by-value match, just makes it harder to use for objects that have no clear labels. Recreate then delete is not a problem, there is no “by accident”, how would you run a plan “by accident”… Although maybe could be replaced with an explicit by value match, but then idk how that should look.
Anyways should be no different from upsert, upsert will too recreate objects when they drift, leaking objects isn’t “better”, if anything, it is worse, and accidental delete of live objects is prohibited anyways.
Though personally I don’t care about that much.

And second one defeats the point and makes it useless for objects that contain multi variant types…

› echo '{ "@type": "reconcile", "object": "Tracer", "matchOn": [ "@type" ], "value": { "tracer-a": { "@type": "Stdout", "level": "info" } } }' | stalwart-cli apply --stdin
Plan: 0 destroy, 0 update, 0 create, 0 upsert, 1 reconcile (1 objects)
✓ reconciled Tracer (1)
Done: 0 destroyed, 0 updated, 1 created (0 failed)

› stalwart-cli query Tracer                                                                                   
Id            Method  Logging level  Enable this tracer
i9xzygqybyqa  Stdout  Info           Yes               

› echo '{ "@type": "reconcile", "object": "Tracer", "matchOn": [ "@type" ], "value": { } }' | stalwart-cli apply --stdin 
Plan: 0 destroy, 0 update, 0 create, 0 upsert, 1 reconcile (0 objects)
✓ reconciled Tracer (0)
Done: 0 destroyed, 0 updated, 0 created (0 failed)

› stalwart-cli query Tracer                                                                                             
Id            Method  Logging level  Enable this tracer
i9xzygqybyqa  Stdout  Info           Yes               

› echo '{ "@type": "reconcile", "object": "Tracer", "matchOn": [ "@type" ], "value": { "tracer-c": { "@type": "Journal", "level": "info" } } }' | stalwart-cli apply --stdin
Plan: 0 destroy, 0 update, 0 create, 0 upsert, 1 reconcile (1 objects)
✓ reconciled Tracer (1)
Done: 0 destroyed, 0 updated, 1 created (0 failed)

› stalwart-cli query Tracer                                                                                             
Id            Method   Logging level  Enable this tracer
i9x93c7qbzaa  Journal  Info           Yes               
i9xzygqybyqa  Stdout   Info           Yes

Like how would you delete a value using reconcile here? That was the point so it would be deleted, otherwise reconcile isn’t useful.

If you want to add scoping I suggest adding an explicit filter, to filter based on type, destroy op has JMAP filters for delete after all.
OpenTofu/Terraform also have somehow similar feature that allows to ignore drifted property for update:
lifecycle { ignore_changes = [some_property] } lifecycle meta-argument reference | Terraform | HashiCorp Developer
I don’t have any concrete suggestion right now for the API shape though.

Probably just having the same filter as the one on the destroy op is enough:
Eg, can introduce “destroyFilter” field on “reconcile” op

{
  "@type": "reconcile",
  "matchOn": [
    "namespace",
    "key"
  ],
  "destroyFilter": {
    "namespace": "spam-traps"
  },
  "object": "MemoryLookupKey",
  "value": {
    "lk-a": {
      "isGlobPattern": true,
      "key": "foo@*",
      "namespace": "spam-traps"
    },
    "lk-b": {
      "isGlobPattern": true,
      "key": "bar@*",
      "namespace": "spam-traps"
    }
  }
}

As a replacement for:

{
  "@type": "destroy",
  "object": "MemoryLookupKey",
  "value": {
    "namespace": "spam-traps"
  }
}
{
  "@type": "upsert",
  "matchOn": [
    "namespace",
    "key"
  ],
  "object": "MemoryLookupKey",
  "value": {
    "lk-a": {
      "isGlobPattern": true,
      "key": "foo@*",
      "namespace": "spam-traps"
    },
    "lk-b": {
      "isGlobPattern": true,
      "key": "bar@*",
      "namespace": "spam-traps"
    }
  }
}

In particular for MemoryLookupKey as you can see a delete filter (allow list) would be pretty useful.

And then you could filter on “@type” for fields that allow filtering on type (it really should be allowed everywhere)

So to summarise, basically couple requests:

  1. remove special handling of enum objects
  2. remove special handling of an empty matchOn
  3. add delete filter
2

Was addressed in new release, though in a slightly different way