Context
Three small follow-ups found while doing backup phase 1 (#6072, merged in #6080). Grouped here because each is a few hours of work on its own and none needs a design discussion - filing separately would be more overhead than signal.
1. ExportDatabaseStatement.copy() and ImportDatabaseStatement.copy() drop their settings map
BackupDatabaseStatement had this exact bug and it is fixed (#6080): copy() only copied url, silently dropping the settings map a WITH ... clause populates, so a copy taken from the statement cache lost every setting.
ExportDatabaseStatement.copy() (engine/src/main/java/com/arcadedb/query/sql/parser/ExportDatabaseStatement.java:134) and ImportDatabaseStatement.copy() (engine/src/main/java/com/arcadedb/query/sql/parser/ImportDatabaseStatement.java:150) have the identical shape:
public Statement copy() {
final ExportDatabaseStatement result = new ExportDatabaseStatement(-1);
result.url = this.url;
return result;
}
Checked before filing: this is latent, not live. Whole-statement copy() for an admin statement like EXPORT DATABASE / IMPORT DATABASE is not currently on the execution path - the planners that call .copy() do so on sub-expressions (WHERE clauses, projections), not on top-level admin statements, and neither of these two goes through the statement cache the way a query does. So today, no WITH ... setting is actually being dropped in production. It is worth fixing anyway because it is the same defect class as the BACKUP one, in the same file family, and it stops being latent the moment either statement's execution path changes.
Fix: mirror the one applied to BackupDatabaseStatement:
result.settings.putAll(this.settings);
Scope: both classes, plus a regression test per class in the shape of BackupDatabaseStatementTestParserTest - build a statement with a WITH setting, copy() it, assert the copy's settings map is non-empty and equal to the original's.
2. PageManager.getDeferredRAMBytes() is not wired into metrics
Added in #6080 (engine/src/main/java/com/arcadedb/engine/PageManager.java) so the backup benchmark could report the deferred-RAM high-water mark - the direct signal that arcadedb.flushSuspendMaxDeferredRAM is throttling committers, and by how much. Its only caller today is BackupCompressionBenchmark.
The value is not backup-specific: it reflects any active flush suspension - full backup, HA snapshot ship (SnapshotHttpHandler), HA verify (PostVerifyDatabaseHandler). An operator has no visibility into it today short of running a benchmark.
Proposed: a gauge in PoolMetrics (server/src/main/java/com/arcadedb/server/monitor/PoolMetrics.java) alongside the existing executor-pool gauges, e.g. arcadedb.pagemanager.deferred_ram_bytes, tagged by database. Surfacing this is also a prerequisite for anyone doing the Phase 2(b) work in docs/optimize-backup.md (#6075), since that phase's whole point is shrinking or eliminating this suspension - having the metric land now gives before/after numbers for free once that lands.
Scope: one gauge registration, no engine change (the accessor already exists), a short test that a suspended flush moves the gauge.
3. Server scheduled backups cannot override the phase-1 compression settings per database
AutoBackupConfig / DatabaseBackupConfig (server/src/main/java/com/arcadedb/server/backup/) configure when and where a scheduled backup runs, but not how - there is no field carrying compressionLevel / compressionThreads / maxMBPerSecond per database, so every scheduled backup uses the GlobalConfiguration default for the whole server.
This is a real gap for a mixed fleet: a large, rarely-restored database might want compressionLevel=9 overnight when nobody is watching the backup window, while a small, frequently-restored one wants compressionLevel=1 to minimize its impact on live traffic. Today both get the same setting.
Proposed: extend DatabaseBackupConfig with the three optional settings (mirroring BackupSettings's null = "defer to global" convention already used by the Backup API), and thread them through BackupTask (server/src/main/java/com/arcadedb/server/backup/BackupTask.java) into the reflective Backup construction the same way setEncryptionKey/setVerboseLevel are already threaded through.
Scope: config schema addition (with a version bump / backward-compat default of "unset" for existing backup.json files), BackupTask wiring, and a test asserting a per-database override actually reaches the Backup instance - in the shape of the existing DatabaseBackupConfigTest.
Out of scope for all three
None of these touch the archive format, the compression path, or anything covered by docs/optimize-backup.md's Phase 2/3. They are independent of each other and can be picked up in any order or split further at implementation time if that turns out to be more convenient.
Follow-up to #6072 / #6080.
Context
Three small follow-ups found while doing backup phase 1 (#6072, merged in #6080). Grouped here because each is a few hours of work on its own and none needs a design discussion - filing separately would be more overhead than signal.
1.
ExportDatabaseStatement.copy()andImportDatabaseStatement.copy()drop their settings mapBackupDatabaseStatementhad this exact bug and it is fixed (#6080):copy()only copiedurl, silently dropping thesettingsmap aWITH ...clause populates, so a copy taken from the statement cache lost every setting.ExportDatabaseStatement.copy()(engine/src/main/java/com/arcadedb/query/sql/parser/ExportDatabaseStatement.java:134) andImportDatabaseStatement.copy()(engine/src/main/java/com/arcadedb/query/sql/parser/ImportDatabaseStatement.java:150) have the identical shape:Checked before filing: this is latent, not live. Whole-statement
copy()for an admin statement likeEXPORT DATABASE/IMPORT DATABASEis not currently on the execution path - the planners that call.copy()do so on sub-expressions (WHEREclauses, projections), not on top-level admin statements, and neither of these two goes through the statement cache the way a query does. So today, noWITH ...setting is actually being dropped in production. It is worth fixing anyway because it is the same defect class as the BACKUP one, in the same file family, and it stops being latent the moment either statement's execution path changes.Fix: mirror the one applied to
BackupDatabaseStatement:Scope: both classes, plus a regression test per class in the shape of
BackupDatabaseStatementTestParserTest- build a statement with aWITHsetting,copy()it, assert the copy'ssettingsmap is non-empty and equal to the original's.2.
PageManager.getDeferredRAMBytes()is not wired into metricsAdded in #6080 (
engine/src/main/java/com/arcadedb/engine/PageManager.java) so the backup benchmark could report the deferred-RAM high-water mark - the direct signal thatarcadedb.flushSuspendMaxDeferredRAMis throttling committers, and by how much. Its only caller today isBackupCompressionBenchmark.The value is not backup-specific: it reflects any active flush suspension - full backup, HA snapshot ship (
SnapshotHttpHandler), HA verify (PostVerifyDatabaseHandler). An operator has no visibility into it today short of running a benchmark.Proposed: a gauge in
PoolMetrics(server/src/main/java/com/arcadedb/server/monitor/PoolMetrics.java) alongside the existing executor-pool gauges, e.g.arcadedb.pagemanager.deferred_ram_bytes, tagged by database. Surfacing this is also a prerequisite for anyone doing the Phase 2(b) work indocs/optimize-backup.md(#6075), since that phase's whole point is shrinking or eliminating this suspension - having the metric land now gives before/after numbers for free once that lands.Scope: one gauge registration, no engine change (the accessor already exists), a short test that a suspended flush moves the gauge.
3. Server scheduled backups cannot override the phase-1 compression settings per database
AutoBackupConfig/DatabaseBackupConfig(server/src/main/java/com/arcadedb/server/backup/) configure when and where a scheduled backup runs, but not how - there is no field carryingcompressionLevel/compressionThreads/maxMBPerSecondper database, so every scheduled backup uses theGlobalConfigurationdefault for the whole server.This is a real gap for a mixed fleet: a large, rarely-restored database might want
compressionLevel=9overnight when nobody is watching the backup window, while a small, frequently-restored one wantscompressionLevel=1to minimize its impact on live traffic. Today both get the same setting.Proposed: extend
DatabaseBackupConfigwith the three optional settings (mirroringBackupSettings'snull= "defer to global" convention already used by theBackupAPI), and thread them throughBackupTask(server/src/main/java/com/arcadedb/server/backup/BackupTask.java) into the reflectiveBackupconstruction the same waysetEncryptionKey/setVerboseLevelare already threaded through.Scope: config schema addition (with a version bump / backward-compat default of "unset" for existing
backup.jsonfiles),BackupTaskwiring, and a test asserting a per-database override actually reaches theBackupinstance - in the shape of the existingDatabaseBackupConfigTest.Out of scope for all three
None of these touch the archive format, the compression path, or anything covered by
docs/optimize-backup.md's Phase 2/3. They are independent of each other and can be picked up in any order or split further at implementation time if that turns out to be more convenient.Follow-up to #6072 / #6080.