Issue Description
During upgrade to v0.16.5 the second step (after start stalwart in recovery mode) the stalwart-cli apply --file export.json fails with an invalidForeignKey error:
# stalwart-cli apply --file migration/export.json
Plan: 0 destroy, 7 update, 6 create (68 objects)
✓ created Tenant (4)
✓ created Domain (17)
✗ create Account: Account: create failed for `restore-98` (operation #3): error: invalidForeignKey | Object id: Domain#e
Done: 0 destroyed, 0 updated, 21 created (1 failed)
error: Account: create failed for `restore-98` (operation #3): error: invalidForeignKey | Object id: Domain#e
Expected Behavior
All accounts should be created by the apply process.
✓ created Account (…)
Actual Behavior
✗ create Account: Account: create failed for `restore-98` (operation #3): error: invalidForeignKey | Object id: Domain#e
Done: 0 destroyed, 0 updated, 21 created (1 failed)
error: Account: create failed for `restore-98` (operation #3): error: invalidForeignKey | Object id: Domain#e
Reproduction Steps
- Start
stalwart in recovery mode.
Using the following config.json file:
{
"@type": "RocksDb",
"path": "/opt/stalwart-mail/data"
}
- run
stalwart-cli apply
The relevant parts of export.json are (the file is actually much longer):
{
"@type": "create",
"object": "Tenant",
"value": {
"create-1": {
"name": "schabel",
"quotas": {}
},
"create-2": {
"name": "tenant:lernenimaufbruch",
"quotas": {}
}
}
}
{
"@type": "create",
"object": "Domain",
"value": {
"create-7": {
"name": "lernenimaufbruch.at",
"memberTenantId": "#create-2"
}
}
{
"@type": "create",
"object": "Account",
"value": {
"restore-98": {
"@type": "Group",
"name": "letmechatthat",
"domainId": "#create-7",
"aliases": {},
"quotas": {},
"description": "Let Me Chat That",
"memberTenantId": "#create-1"
}
}
Stalwart Version
v0.16.x
Installation Method
Binary (Linux)
Database Backend
RocksDB
Blob Storage
RocksDB
Search Engine
Internal
Directory Backend
Internal
I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.
on
I have searched this support forum (open and closed topics) and confirm this is not a duplicate.
on
I understand that topics in this category are triaged by a bot first but a human reply will follow up. If I’d prefer a human-only reply, I’ll add the no-ai tag to my topic.
on
Let me research this issue with the stalwart-cli apply foreign key error during migration.
The UPGRADING guide addresses this exact scenario in detail. Here’s what’s happening and how to fix it:
The invalidForeignKey error means an Account is referencing a Domain (via #create-7) that was not successfully created in a previous partial apply run, or the Domain block referencing the Tenant via #create-2 has itself failed silently. In your case, looking at your export.json snippet, the Domain block appears to have a missing closing brace (}) – though that may just be a truncation in your post. If the Domain create actually failed or was incomplete, the downstream Account referencing #create-7 would get this error.
Recovery steps (from the upgrade guide’s “Recovering from a partial apply” section):
- While still in recovery mode, wipe the partially-created objects:
$ stalwart-cli query Domain --json | jq -r '.[].id' \
| stalwart-cli delete Domain --stdin
$ stalwart-cli query Account --json | jq -r '.[].id' \
| stalwart-cli delete Account --stdin
$ stalwart-cli query Tenant --json | jq -r '.[].id' \
| stalwart-cli delete Tenant --stdin
$ stalwart-cli query DkimSignature --json | jq -r '.[].id' \
| stalwart-cli delete DkimSignature --stdin
$ stalwart-cli query Certificate --json | jq -r '.[].id' \
| stalwart-cli delete Certificate --stdin
-
Check your export.json for the likely root cause. The guide lists common issues:
- A domain that fails the v0.16 hostname check
- An account whose local-part contains
@
- A stale
/opt/stalwart path embedded by the migration script
In your snippet, note that the Account restore-98 has memberTenantId: "#create-1" (the schabel tenant) but its domainId points to #create-7 (the lernenimaufbruch.at domain, which belongs to the other tenant #create-2). This cross-tenant domain/account mismatch may be what triggers the invalidForeignKey.
-
Also make sure you are running stalwart-cli v1.0.2 or later, as the guide specifically calls that out for the apply command.
-
After cleaning up and fixing export.json, rerun:
$ stalwart-cli apply --file export.json
If you need to proceed despite errors, the --continue-on-error flag is available, though you should still fix the root cause first.
The full troubleshooting section is in the v0.16 upgrade guide under “Recovering from a partial apply”.
This is an automated reply from the Stalwart Help Bot. Other community members may follow up if this answer is incomplete or wrong. If you found it useful, mention @helpbot in any reply to ask follow-up questions and the bot will respond again.
It seems that the migrate_v016.py script generates a faulty export.json file: all accounts with type Group in the export.json have the same domainId foreign key which is definitely not correct.
It seems that the group accounts in the generated principals.json doesn’t contain any domain information, only the tenant.
The faulty call steck in the migrate script is _build_groups → _resolve_name_and_domain → _infer_primary_domain.
The problem seems to be the _infer_primary_domain method of the migrate script. It only returns a domain if an email has a local part and that the local part is equal to the name field of the group.
I have some groups in my principals.json that only have one (or more) catch-all email address in the emails field @domain with no local part. In that case _infer_primary_domain returns None. In that case _resolve_name_and_domain returns default_domain_cid which finally results in a failed stalwart-cli apply because the default domain doesn’t match the group’s tenant (I guess?).
Is there any problem if I patch the _infer_primary_domain method so that it returns one of the domains contained in the emails field of the group even if it doesn’t have a local part?