Repository navigation
Support calendar-month/year (mo/y) intervals for Continuous Query EVERY and RANGE clauses #18428
Description
Is your feature request related to a problem?
IoTDB's Continuous Query (CQ) does not support calendar-month (or calendar-year) alignment for the EVERY execution interval and the RANGE time offsets. This makes it impossible to schedule a CQ that fires exactly at month boundaries — e.g. "run at the end of every month and aggregate over the whole natural month" — which is a common downsampling requirement. Because months have 28/29/30/31 days, the window length must be handled dynamically, so a fixed-day interval is not an adequate substitute.
Current behavior
Documentation
The CQ documentation states that every_interval, start_time_offset, and end_time_offset support only the units ns, us, ms, s, m, h, d, w — mo (month) and y (year) are not listed:
<every_interval>specifies the query execution time interval. We currently support the units of ns, us, ms, s, m, h, d, w ...
Source: https://iotdb.apache.org/UserGuide/latest/User-Manual/Database-Programming.html
Implementation
However, the implementation does not reject mo/y in these clauses — it silently accepts them and converts them to a fixed duration (1 month → 30 days, 1 year → 365 days):
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/DataNodeDateTimeUtils.java— the single-argument overloadconvertDurationStrToLong(String)is called withcurrentTime = -1, so themo/monthbranch flattens to30 * 86_400_000ms andy/yearto365 * 86_400_000ms.- The CQ parser uses exactly this overload for
EVERY/RANGE:ASTVisitor.parseResampleClause(...)iniotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/parser/ASTVisitor.java. - The scheduler then treats the result as a plain
longand does pure integer arithmetic (executionTime += everyInterval,getFirstExecutionTime(...)) iniotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/cq/CQScheduleTask.java— there is no calendar logic at runtime.
As a result, EVERY 1mo does not mean "at the end of every calendar month"; it means "every fixed 30 days", which drifts relative to real month boundaries. This is arguably worse than a clean rejection, because the statement appears to be accepted but is silently mis-scheduled.
Note that GROUP BY(1mo) in the CQ body does work as a natural calendar month, because it goes through a different converter (constructTimeDuration, which keeps a separate monthDuration part and can be evaluated against the calendar). So within a single CQ the EVERY/RANGE cadence (fixed 30 days) and the GROUP BY window (natural month) can drift apart.
Desired behavior
Support mo/month and y/year as first-class units for the CQ EVERY interval and RANGE offsets, evaluated against the calendar (the way GROUP BY already handles months), so a CQ can be aligned to true month/year boundaries.
CREATE CONTINUOUS QUERY cq_monthly_spread
RESAMPLE EVERY 1mo RANGE 1mo
BEGIN
SELECT max_value(s), min_value(s)
INTO root.db.device(monthly_max, monthly_min)
FROM root.db.device
GROUP BY(1mo)
ENDThis should execute at each month boundary and aggregate over the just-finished natural month (correctly handling 28/29/30/31-day months).
Alternatives considered
- External scheduler (e.g. cron at month-end) running a one-shot query — works today, but moves scheduling logic outside IoTDB and loses CQ's built-in result-sink semantics.
- Daily CQ with
EVERY 1d RANGE 31d+GROUP BY(1mo)— partially works, but the monthly result is recomputed daily and only "finalizes" after month-end; andRANGE 31dis still a fixed window rather than a true calendar month.
Additional context
As a smaller, complementary fix (or interim step), the silent 30/365-day flattening of mo/y in EVERY/RANGE could be turned into an explicit error, so that the runtime behavior matches the documentation and users are not misled into thinking EVERY 1mo produces a calendar-month cadence.
/ccraised by a user who wants month-end downsampling (spread = max − min over a natural month).
Activity
DaZuiZui commentedon Aug 10, 2026
i work it
henjoe commentedon Aug 10, 2026
+1 for this feature
DaZuiZui commentedon Aug 10, 2026
Our implementation approach for this requirement
Below is our implementation approach for this requirement. We first define the user-visible behavior—including syntax, duration semantics, component meanings, supported units, and examples—and then describe the corresponding parser, RPC, persistence, scheduling, recovery, and execution changes.
Maintainer alignment: This version incorporates the maintainer feedback on the previously open semantic questions. Section 10 records the aligned decisions so future readers can distinguish the implementation baseline from alternatives that were considered.
1. Goal and scope
Continuous Query should support real calendar-month and calendar-year semantics in EVERY and RANGE:
This feature changes only CQ scheduling and the generated query time range. It does not add new source measurement types, result types, aggregate functions, or SELECT INTO behavior. The CQ query body continues to follow the existing tree-model query and type rules.
2. Syntax
The existing CQ grammar remains:
Duration syntax:
Supported duration units:
Examples of valid duration values:
Syntax rules:
BOUNDARY keeps the existing timeValue inputs:
3. Duration data model and component meanings
Logically, a duration is a pair:
Component meanings:
Additional rules:
4. Meaning of each CQ component
EVERY
EVERY defines the recurrence duration.
Let:
The nth scheduled timestamp is:
Every occurrence must be calculated from the original boundary. It must not be calculated by repeatedly adding one month to the previously clamped timestamp.
For example, with BOUNDARY 2024-01-31 and EVERY 1mo:
The sequence must not drift to March 29 after February clamps the date.
If EVERY is omitted:
BOUNDARY
BOUNDARY defines the stable recurrence anchor.
When explicitly supplied, the parsed timestamp is the exact anchor.
For calendar EVERY with BOUNDARY omitted, our implementation will use 1970-01-01 00:00:00 in the persisted CQ time zone as the canonical local calendar anchor. This allows a CQ without an explicit boundary to run at local month/year boundaries.
An explicit BOUNDARY 0 would continue to mean the Unix epoch instant. Therefore, the implementation must retain whether BOUNDARY was explicitly specified; a numeric value of zero alone is insufficient.
For fixed-only EVERY with BOUNDARY omitted, the current BOUNDARY 0 behavior remains unchanged.
This is the selected omitted-BOUNDARY behavior for this implementation.
RANGE
RANGE defines the left-closed, right-open query interval:
Defaults remain:
To preserve the currently documented boundary formula and avoid non-invertible calendar subtraction, our implementation will compute the endpoints in duration-vector space before applying them to the boundary:
All endpoints are derived from the original boundary in duration-vector space. In particular, when RANGE equals EVERY, startTime(n) equals executionTime(n - 1) for n greater than zero, so adjacent windows remain contiguous even for month-end and leap-day anchors.
Example with BOUNDARY 2024-01-31, EVERY 1mo, RANGE 1mo:
This is the selected RANGE formula. Offsets must not be subtracted from an already-clamped execution timestamp because calendar-month arithmetic is not reversible and would produce different, potentially non-contiguous results.
TIMEOUT POLICY
BLOCKED:
DISCARD:
Calendar occurrence lookup should use an estimate plus correction or binary search. It should not iterate month by month from 1970.
BEGIN/END query body
The query body remains an existing tree-model SELECT INTO statement.
The existing CQ restrictions remain, including the automatic insertion of the time filter and GROUP BY TIME range. The DataNode ultimately receives concrete startTime and endTime values, so the main query execution engine does not need calendar-duration awareness.
5. Calendar arithmetic rules
calendarApply(base, duration, zone) is defined as:
Therefore, calendarApply always applies monthPart using calendar arithmetic in the persisted CQ ZoneId first, and then applies fixedPart as elapsed ticks. The order must not be reversed.
Month-end and leap-year examples:
Because each occurrence uses B plus n times E, the original day can reappear in a later valid month or leap year.
The persisted CQ zone must be used during creation, normal scheduling, leader recovery, and snapshot recovery. ConfigNode system-default time zones must not affect the result.
6. Semantic validation
The current basic constraints remain:
Calendar and fixed durations do not form a permanent total order:
Therefore, semantic checks must never flatten a month to 30 days.
Duration comparison rules:
Overflow and resource validation should also cover excessive month counts, duration multiplication, timestamps near Long.MIN_VALUE/Long.MAX_VALUE, and non-monotonic or non-terminating occurrence searches.
7. Examples
Natural monthly aggregation
With CQ zone Asia/Shanghai and the omitted-BOUNDARY rule defined above, an execution at 2024-03-01 00:00 covers:
The February window is 28 or 29 days depending on the year.
Explicit month-end anchor
Occurrence sequence:
Calendar-year aggregation
The schedule clamps to February 28 in non-leap years and returns to February 29 when n times one year lands in a leap year.
Two RANGE offsets
At an execution timestamp of 2024-04-01 on a first-of-month anchor, the query interval is:
8. Persistence, protocol, and upgrade behavior
Compatibility and upgrade handling:
The timeout sent in TExecuteCQ should be based on the actual distance from the current occurrence to the next occurrence, rather than a fixed 30-day approximation.
9. Implementation outline
The expected data flow is:
Main implementation areas:
10. Maintainer-aligned implementation decisions
Based on maintainer feedback, the implementation will:
The test plan will cover explicit and inherited 1mo/1y, rejection of CQ-only month/year full aliases, omitted RANGE, Jan-31, Feb-29, 28/29/30/31-day months, UTC/Asia-Shanghai/America-New-York, DST transitions, provable comparisons such as 1mo3d versus 1mo, rejection of ambiguous comparisons such as 1mo versus 30d, ms/us/ns precision, BLOCKED/DISCARD catch-up, fixed-duration regression, procedure/plan serialization, occurrence-index recovery, leader changes, new snapshot round trips, legacy multi-entry snapshot fixtures, and preservation of legacy fixed behavior for existing mo/y CQs.
DaZuiZui commentedon Aug 10, 2026
我们实现这个需求的思路（中文）
下面说明我们准备如何实现这个需求。我们先明确用户可见的功能语义，包括语法、duration 语义、各 component 含义、支持的单位和示例，再说明解析、RPC、持久化、调度、恢复和执行链路的实现思路。
Maintainer 对齐状态： 本版本已经同步 maintainer 对此前待确认语义的回复。第 10 节集中记录对齐后的实现结论，方便后续开发者和 AI 区分最终实现基线与曾讨论过的备选方案。
1. 功能目标与范围
Continuous Query 的 EVERY 和 RANGE 应支持真正的日历月、日历年语义：
本功能只改变 CQ 调度和自动生成的查询时间范围，不新增 source measurement 类型、结果类型、聚合函数或 SELECT INTO 行为。CQ 查询正文继续遵循现有 tree-model 查询和类型规则。
2. 语法定义
CQ 现有语法保持不变：
Duration 语法：
本实现支持的单位：
合法 duration 示例：
语法规则：
BOUNDARY 保留现有 timeValue 输入类型：
3. Duration 数据类型与各 component 含义
逻辑上，一个 duration 是一个二元结构：
各 component 含义：
补充规则：
4. 每个 CQ component 的具体语义
EVERY
EVERY 定义 CQ 的重复调度周期。
设：
第 n 个计划执行时间为：
每个 occurrence 必须从原始 BOUNDARY 直接计算，不能在上一次已经发生月末钳制的 executionTime 上连续加月。
例如 BOUNDARY 2024-01-31、EVERY 1mo：
二月钳制到 29 日后，后续不能漂移到 3 月 29 日。
如果省略 EVERY：
BOUNDARY
BOUNDARY 定义整个 recurrence 序列的稳定锚点。
用户显式指定时，使用解析后的准确时间戳作为锚点。
对于包含日历月的 EVERY，如果省略 BOUNDARY，我们的实现会使用 CQ 持久化时区中的 1970-01-01 00:00:00 作为规范化的本地日历锚点。这样不显式写 BOUNDARY 的 CQ 也可以在本地自然月界或自然年界运行。
显式 BOUNDARY 0 仍表示 Unix epoch instant。因此实现中必须保存用户是否显式指定了 BOUNDARY，不能只通过数值是否为 0 来判断。
对于不包含日历月的 fixed-only EVERY，省略 BOUNDARY 时继续保持现有 BOUNDARY 0 行为。
这是本实现确定采用的省略 BOUNDARY 规则。
RANGE
RANGE 定义每次 CQ 查询的左闭右开时间范围：
默认值保持不变：
为了保持现有文档中的 BOUNDARY 公式，并避免日历月加减法不可逆，我们的实现会先在 duration 向量上完成加减，再把结果一次性作用于原始 BOUNDARY：
所有端点都从原始 BOUNDARY 在 duration 向量空间中计算。特别地，当 RANGE 等于 EVERY 时，对于 n 大于 0，startTime(n) 严格等于 executionTime(n - 1)，因此月末和闰日锚点下的相邻窗口仍然连续。
例如 BOUNDARY 2024-01-31、EVERY 1mo、RANGE 1mo：
这是本实现确定采用的 RANGE 公式。不得从已经发生月末钳制的 executionTime 直接减 offset，因为日历月运算不可逆，并会产生不同且可能不连续的结果。
TIMEOUT POLICY
BLOCKED：
DISCARD：
Calendar occurrence 的查找应使用估算加修正或二分，不能从 1970 年开始逐月循环。
BEGIN/END 查询正文
查询正文继续使用现有 tree-model SELECT INTO statement。
现有 CQ 限制保持不变，包括自动加入 time filter 和 GROUP BY TIME range。DataNode 最终仍接收具体的 startTime 和 endTime，因此主查询执行引擎不需要感知 calendar duration。
5. 日历运算规则
calendarApply(base, duration, zone) 定义为：
因此，calendarApply 的顺序固定为：先在持久化 CQ ZoneId 中按日历语义应用 monthPart，再应用 fixedPart elapsed ticks；两部分不得交换顺序。
月末和闰年示例：
因为每次 occurrence 都使用 B + n * E 计算，所以原始日期可以在后续合法月份或闰年重新出现。
CQ 持久化的 ZoneId 必须用于创建、正常调度、leader 恢复和 snapshot 恢复。ConfigNode 所在机器的 system default time zone 不能影响结果。
6. 语义校验
保留现有基础约束：
日历时长和固定时长不存在永久全序：
因此语义校验绝不能继续把一个月压成 30 天。
比较与校验规则：
还需要校验超大月份数量、duration 乘法、接近 Long.MIN_VALUE/Long.MAX_VALUE 的时间戳，以及可能导致 occurrence 搜索不单调或不终止的输入。
7. 使用示例
自然月聚合
在 CQ 时区为 Asia/Shanghai，并按上述省略 BOUNDARY 规则，2024-03-01 00:00 的执行覆盖：
二月窗口会根据年份自动包含 28 或 29 天。
显式月末锚点
Occurrence 序列：
自然年聚合
非闰年会钳制到 2 月 28 日；当 n * 1y 再次落在闰年时，日期会恢复为 2 月 29 日。
两个 RANGE offset
如果锚点为每月 1 日，在 2024-04-01 执行时，查询范围为：
8. 持久化、协议和升级行为
兼容与升级实现：
TExecuteCQ 中的 timeout 应使用本次 occurrence 到下一次 occurrence 的实际距离，不能继续使用固定 30 天近似。
9. 实现链路
预期的数据流：
主要实现点：
10. 基于 maintainer 反馈确定的实现语义
本实现按以下结论落地：
测试计划覆盖：显式及从 GROUP BY 继承的 1mo/1y、CQ 不接受 month/year 全称、省略 RANGE、Jan-31、Feb-29、28/29/30/31 天月份、UTC/Asia-Shanghai/America-New-York、DST 切换、1mo3d 与 1mo 等可证明比较、拒绝 1mo 与 30d 等不确定比较、ms/us/ns 精度、BLOCKED/DISCARD 追赶、fixed duration 回归、procedure/plan 序列化、occurrence-index 恢复、leader 切换、新 snapshot round trip、包含多条 CQ 的 legacy snapshot fixture，以及现有 mo/y CQ 保持 legacy fixed 行为。
DaZuiZui commentedon Aug 13, 2026
Scope-limited addendum: required implementation clarifications
This addendum only closes implementation ambiguities in the proposal above. It is limited to the calendar-duration behavior requested by #18428 and is pending maintainer confirmation. Where it conflicts with an earlier point, this addendum supersedes only that point; the rest of the proposal remains unchanged.
1. CQ duration aliases
For the
EVERYandRANGEduration positions only, support the following case-insensitive calendar units:mo,monthy,year1yand1yearnormalize to 12 calendar months. Plural aliases, decimals, signed components, and whitespace inside a duration remain unsupported. A CQ-specific duration rule must consume and validate the complete value, so addingmonthandyearhere does not changeGROUP BY TIME, date arithmetic,FILL,SESSION, or general identifier tokenization.This replaces the earlier statement that this issue supports only the abbreviated spellings.
2. Structured request and explicit BOUNDARY
Use a bounded structured value such as:
TCreateCQReqadds optional structured values forevery,startOffset, andendOffset, plus an optionalboundaryExplicitflag. For a new structured request, all three effective durations andboundaryExplicitmust be present; a partial structured representation is rejected. All arithmetic and narrowing conversions are checked.The existing
i64duration fields and field IDs remain for legacy fixed-only requests. A request with no structured fields is interpreted as legacy fixed-only. For a structured request, the structured values are authoritative; the legacyi64fields must never be used by new code to approximate a non-zero calendar month part. When every structured month part is zero, the legacy and structured fixed values must agree or the request is rejected.boundaryExplicit=falsemeans that ConfigNode applies the selected omitted-BOUNDARY rule.boundaryExplicit=truepreserves an explicitly writtenBOUNDARY 0as the Unix epoch instant. The flag, normalized boundary, durations, ZoneId, and progress index must pass through procedure/plan serialization and CQ metadata snapshots.3. Mixed-version creation policy
This issue does not claim support for creating a calendar CQ during a rolling mixed-version window. Calendar CREATE is supported only after all registered ConfigNodes and all DataNodes that can accept client SQL have been upgraded to code that understands the structured representation. This condition covers DataNodes because
CREATE CQis parsed there beforeTCreateCQReqis built.Before forwarding a structured calendar request, an upgraded SQL-ingress DataNode must positively verify this condition using the existing cluster node-version information; the upgraded ConfigNode repeats the check before accepting the request. An unknown or unsupported registered version causes a clear rejection. During an upgrade, old DataNodes must be removed from client routing before calendar CREATE is used. Fixed-only CQ creation and historical CQs retain their legacy behavior.
This is a narrow fail-closed compatibility check for #18428, not a new feature-activation, membership-attestation, or upgrade framework.
4. Deterministic calendar and DST resolution
Let
Bbe the exact anchor instant in persisted timestamp-precision ticks andZthe persisted CQZoneId. For a duration vector(M, F):M == 0,calendarApply(B, (M, F), Z)is checked elapsed-tick additionB + F. In particular,calendarApply(B, ZERO, Z) == B, and fixed-only CQ behavior is unchanged.targetLocal = anchorLocal.plusMonths(M), including end-of-month clamping. Resolve it with the rules ofZ, preferringanchorOffsetwhen that offset is valid attargetLocal, and then addFas checked elapsed ticks.anchorOffsetwhen valid; otherwise use the earlier valid offset. This is the behavior ofZonedDateTime.ofLocal(targetLocal, Z, anchorOffset)and must not depend on a host default time zone.An integer or explicitly-offset BOUNDARY is first parsed as the exact instant
B. A written offset selects that instant but does not replace the persisted CQ zoneZ; recurrence observesBinZ. An offset-less BOUNDARY is resolved inZusing the same gap rule and the earlier offset for an overlap when no preferred anchor offset exists.This issue persists
ZoneId, not a private copy of TZDB rules. Scheduling nodes are expected to use compatible TZDB data; installing a newer TZDB may legitimately affect future civil-time transitions.5. Anchor-relative compound durations and progress
Duration arithmetic is component-wise and anchor-relative:
Multiplication, vector addition/subtraction, and timestamp conversion are checked. A compound duration is not repeatedly applied to the previous occurrence. For example, with a UTC anchor of
2024-01-30 00:00andEVERY 1mo2d,n=1is2024-03-02 00:00andn=2is2024-04-03 00:00. RANGE vectors use the same component-wise arithmetic before onecalendarApplycall.ConfigNode captures the CREATE reference instant
Conce. The initial index is exactly:Equality selects that occurrence. CQ metadata persists
nextOccurrenceIndex, defined as the first occurrence not yet durably completed. Failure retries the same index; after success, progress is persisted before the next occurrence is scheduled, and recovery resumes the stored index.BLOCKEDadvances in order;DISCARDuses the same lower-bound rule against one captured callback time when skipping missed indices. This preserves the existing at-least-once execution boundary and does not introduce an exactly-once protocol.6. Timeout and focused tests
TExecuteCQ.timeoutremains milliseconds. For occurrencen, derive it from the actual adjacent scheduled instants:The subtraction and conversion are checked, and a positive sub-millisecond interval becomes 1 ms. Timestamp-precision ticks must not be passed directly as milliseconds, and no 30-day approximation is used.
In addition to the tests already listed, focused tests will cover:
EVERY/RANGE;boundaryExplicitround trips;calendarApply(B, ZERO, Z) == B, DST gaps, both overlap offsets, and explicit-offset BOUNDARY with a different CQ ZoneId;nextOccurrenceIndex, and recovery without an off-by-one shift;Explicitly outside this addendum are new V2 RPC families, PREPARE/ACTIVATE commands, reader-floor markers, node attestation, general mutation exact-retry protocols, execution-admission redesign, and a custom ZoneRules snapshot format.
DaZuiZui commentedon Aug 13, 2026
中文对照：范围受限的实现补充说明（对应英文 addendum）
以下内容仅为上一条英文 “Scope-limited addendum: required implementation clarifications” 的中文对照，方便中文读者阅读。它不引入任何新的要求、语义或实现范围，也不单独构成规范；如中英文表述存在歧义或不一致，以上一条英文 addendum 为准。其适用范围、待 maintainer 确认状态及与此前方案的优先关系，均与英文原文完全相同：仅在发生冲突的具体点上替代此前表述，其余内容保持不变。
本补充说明只用于消除上述方案中的实现歧义。它仅限于 #18428 所要求的日历时长行为，并等待 maintainer 确认。如果本补充说明与之前的某一点冲突，则只取代发生冲突的那一点；方案的其余部分保持不变。
1. CQ 时长单位别名
仅在 CQ 的
EVERY和RANGE时长参数位置，支持以下大小写不敏感的日历单位：mo、monthy、year1y和1year统一归一化为 12 个日历月。复数形式的别名、小数、带正负号的输入分量以及时长内部的空白仍不受支持。CQ 专用的时长规则必须完整消费并校验整个值，因此在这里增加month和year不会改变GROUP BY TIME、日期运算、FILL、SESSION或一般标识符的词法切分方式。这一点取代此前“本 issue 只支持缩写形式”的表述。
2. 结构化请求与显式 BOUNDARY
使用一个数值有界的结构化值，形式例如：
TCreateCQReq为every、startOffset和endOffset增加 optional 结构化值，并增加 optional 的boundaryExplicit标志。对于新的结构化请求，三个最终生效的时长和boundaryExplicit必须全部存在；部分提供的结构化表示必须被拒绝。所有算术运算和窄化转换都必须检查溢出与越界。为兼容 legacy fixed-only 请求，保留现有的
i64时长字段及其 field ID。完全没有结构化字段的请求按 legacy fixed-only 请求解释。对于结构化请求，结构化值是权威值；新代码绝不能使用 legacyi64字段近似任何非零的日历月分量。当所有结构化时长的monthPart均为 0 时，legacy fixed 值与 structured fixed 值必须一致，否则拒绝该请求。boundaryExplicit=false表示 ConfigNode 应用已经选定的“省略 BOUNDARY”规则。boundaryExplicit=true则将用户显式写出的BOUNDARY 0保留为 Unix epoch instant。该标志、归一化后的 boundary、各时长、ZoneId和进度索引必须完整贯穿 procedure/plan 序列化以及 CQ metadata snapshot。3. 混合版本下的创建策略
本 issue 不声称支持在滚动升级的混合版本窗口中创建日历型 CQ。只有当所有已注册的 ConfigNode，以及所有能够接收客户端 SQL 的 DataNode，都已经升级到能够理解结构化表示的代码版本后，才支持日历型 CQ 的 CREATE。之所以该条件必须覆盖 DataNode，是因为
CREATE CQ会先在 DataNode 上解析，然后才构造TCreateCQReq。在转发结构化日历请求前，升级后的 SQL-ingress DataNode 必须使用现有的集群节点版本信息，明确验证上述条件已经满足；升级后的 ConfigNode 在接受请求前再次执行该检查。任何已注册节点的版本未知或不受支持，都必须导致清晰的拒绝。在升级期间，使用日历型 CREATE 之前，必须先将旧 DataNode 从客户端路由中移除。fixed-only CQ 的创建以及历史 CQ 继续保持 legacy 行为。
这只是针对 #18428 的狭义、fail-closed 兼容性检查，并不是新的 feature-activation、membership-attestation 或升级框架。
4. 确定性的日历与 DST 解析
令
B为使用持久化 timestamp precision tick 表示的精确锚点 instant，Z为持久化的 CQZoneId。对于时长向量(M, F)：M == 0，则calendarApply(B, (M, F), Z)为经过检查的 elapsed-tick 加法B + F。特别地，calendarApply(B, ZERO, Z) == B，并且 fixed-only CQ 的行为保持不变。targetLocal = anchorLocal.plusMonths(M)，其中包括月末钳制。按照Z的规则解析该本地时间；当anchorOffset在targetLocal有效时优先使用它，之后再把F作为经过检查的 elapsed ticks 加上。anchorOffset有效则使用它，否则使用较早的有效 offset。这与ZonedDateTime.ofLocal(targetLocal, Z, anchorOffset)的行为一致，并且不得依赖宿主机的默认时区。整数形式或显式带 offset 的 BOUNDARY，首先解析为精确的 instant
B。写出的 offset 只用于选定该 instant，不会替换持久化的 CQ 时区Z；recurrence 会在Z中观察并计算B。不带 offset 的 BOUNDARY 使用Z解析：gap 使用同样的规则；overlap 在不存在 preferred anchor offset 时使用较早的 offset。本 issue 持久化的是
ZoneId，而不是 TZDB 规则的私有副本。各调度节点应使用兼容的 TZDB 数据；安装较新的 TZDB 可能会合理地影响未来的 civil-time transition。5. 相对锚点的复合时长与进度
时长运算按分量执行，并且相对于原始锚点：
乘法、向量加减以及 timestamp 转换都必须经过检查。复合时长不得反复作用于前一个 occurrence。例如，以 UTC 的
2024-01-30 00:00为锚点，EVERY 1mo2d时，n=1为2024-03-02 00:00，n=2为2024-04-03 00:00。RANGE 向量同样先进行按分量运算，然后只调用一次calendarApply。ConfigNode 只捕获一次 CREATE reference instant
C。初始索引严格定义为：相等时选择该 occurrence。CQ metadata 持久化
nextOccurrenceIndex，其定义为第一个尚未持久完成的 occurrence。执行失败时重试同一个索引；执行成功后，必须先持久化进度，再调度下一个 occurrence；恢复时从已存储的索引继续。BLOCKED按顺序推进；DISCARD在跳过错过的索引时，针对一次捕获的 callback time 使用同一个 lower-bound 规则。这样保持现有的 at-least-once 执行边界，并且不会引入 exactly-once 协议。6. Timeout 与定向测试
TExecuteCQ.timeout仍以毫秒表示。对于 occurrencen，根据相邻两个实际计划执行 instant 之间的距离计算：减法和转换都必须经过检查，任何正的亚毫秒间隔转换为 1 ms。不得将 timestamp-precision ticks 直接作为毫秒传递，也不得使用固定 30 天的近似值。
除此前已经列出的测试外，还将通过定向测试覆盖：
EVERY/RANGE之外使用这些别名时的拒绝行为；boundaryExplicit的 round trip；calendarApply(B, ZERO, Z) == B、DST gap、overlap 的两种 anchor offset，以及显式带 offset 的 BOUNDARY 与不同 CQZoneId的组合；nextOccurrenceIndex，以及恢复时不发生 off-by-one 偏移；本补充说明明确不包括：新的 V2 RPC families、PREPARE/ACTIVATE 命令、reader-floor markers、node attestation、通用 mutation exact-retry 协议、execution-admission redesign，以及自定义 ZoneRules snapshot 格式。
JackieTien97 commentedon Aug 21, 2026
Thanks for the detailed proposal and addendum. I reviewed them against the current
masterimplementation (a110cc0). The duration-pair model and anchor-relative arithmetic are good directions, but I think several points need to be resolved before this becomes the implementation baseline. I consider the first three correctness/compatibility blockers.1. The proposed DST-overlap rule is inconsistent with the existing
GROUP BY TIMEcalendar behaviorThe addendum resolves an overlap with
ZonedDateTime.ofLocal(targetLocal, zone, anchorOffset), preferring the original anchor offset when it is valid. However, the currentGROUP BY TIMEmonth path goes throughcalcPositiveIntervalByMonth, whose underlying calendar conversion resolves the resulting local time with the normalatZonerule (the earlier offset in an overlap).For example, in
America/New_York:The addendum's CQ rule selects
-05:00(06:30Z), while the current group-by rule selects-04:00(05:30Z). A CQ RANGE ending at06:30Zcombined withGROUP BY(10mo)can therefore create a one-hour partial/misaligned bucket. That conflicts with the stated goal that CQ cadence/RANGE andGROUP BY TIMEuse consistent calendar semantics.CQ should reuse exactly the same calendar helper and overlap rule as
GROUP BY TIME. If the shared rule is intentionally changed, that is a broader behavior change and should be made explicit and tested for both paths.2. The mixed-version policy is not actually fail-closed for requests entering through an old DataNode
An old DataNode already accepts
EVERY 1mo, flattens it to 30 days, and sends only the legacyi64fields. Under the proposed rules, a new ConfigNode interprets a request with no structured fields as a valid legacy fixed-only request. Consequently:1mo;Removing old DataNodes from client routing is an operational prerequisite, not an enforceable protocol guarantee.
A new ConfigNode needs an enforceable way to identify this case, for example by parsing the original
req.sqland rejecting legacy requests whose CQEVERY/RANGEcontains calendar units, rejecting all legacy-protocol CQ creation during the mixed-version window, or requiring an explicit source-capability/protocol marker.Also, the legacy duration fields remain required, but the proposal does not define which values a new DataNode sends there when a structured duration has a nonzero month part. A 30-day approximation is unsafe, while zero may break an accidental old reader. The wire contract and the hard reader barrier should both be specified.
3.
nextOccurrenceIndexneeds an exact CAS/fencing transitionPersisting
nextOccurrenceIndexis the right model, but "persist progress before scheduling the next occurrence" is not sufficient by itself during a ConfigNode leader change. The current implementation accepts any greater execution time for the same CQ token (CQInfo.updateCQLastExecutionTime), and the callback can continue scheduling after most progress-write failures (CQScheduleTask). Old- and new-leader callbacks can therefore race and derive competing schedule chains.The progress write should carry the CQ generation/token and expected index
n, and atomically perform:only if both still match. If the stored index is already greater than
n, the callback is stale/duplicate and must not schedule another task. If the write fails, the samenshould be retried. ForDISCARD, advancement after a successfulnshould be at leastmax(n + 1, lowerBound(callbackTime)), including clock rollback/equality cases.4. Duration comparison still needs a deterministic, implementable definition
"Provable for every applicable calendar position, time zone, and DST transition" is a useful principle, but it does not define the actual comparator or the validation against
continuous_query_minimum_every_interval.In particular, an existing 28-days-per-month minimum must not be reused as a conservative lower bound: a calendar month crossing a spring-forward transition can be shorter than 28 elapsed days (for example,
27d23h). Historical zone transitions can be larger as well.Please specify the exact partial order. A simple safe rule for duration-vs-duration checks is component-wise dominance, with calendar-vs-fixed cases rejected when ordering cannot be proven. Validation against the fixed minimum-EVERY configuration needs its own explicit rule so that different implementations or TZDB data do not produce inconsistent acceptance.
5. I do not recommend adding CQ-only
month/yearaliases in this issueCurrent Tree SQL duration literals support the existing abbreviations such as
yandmo(lexer rule). Table SQL hasYEAR/MONTH, but Table SQL does not currently provide CQ syntax.Allowing
EVERY 1monthwhile rejectingGROUP BY(1month), date arithmetic,FILL, andSESSIONcreates an unnecessary CQ-private dialect. A single CQ could have to write:RESAMPLE EVERY 1month RANGE 1month ... GROUP BY(1mo)I recommend keeping this change aligned with existing Tree SQL and supporting only
y/mo. Fullyear/monthaliases should be a separate, consistent Tree SQL grammar enhancement covering all duration positions.The following parts of the proposal look sound and should be retained:
(monthPart, fixedPart)durations withy -> 12mo;With the DST alignment, mixed-version ingress hole, and progress CAS semantics addressed, the core design should be in good shape.
DaZuiZui commentedon Aug 24, 2026
Revised addendum addressing the review blockers
Thanks for taking the time to review the proposal so carefully against the current implementation and for walking through the issues one by one. I agree with the points you raised, especially that the first three need to be addressed first as correctness and compatibility issues.
Where this revision conflicts with the previous proposal or addendum, this revision takes precedence. The remaining parts of the previous proposal stay unchanged.
1. Use exactly the same calendar and DST resolution as
GROUP BY TIMEThe CQ implementation will not introduce the previously proposed preferred-anchor-offset rule for DST overlaps.
CQ occurrence and RANGE calculations will reuse the same calendar conversion used by the current
GROUP BY TIMEmonth path, including the behavior of:The implementation will reuse
DateTimeUtils.calcPositiveIntervalByMonthand its underlyingTimeDuration.calcPositiveIntervalByMonthbehavior, or extract that behavior into a shared helper used by both paths.The resulting rules are:
atZonerule.The shared helper may be extended to support checked arithmetic and signed internal duration vectors needed by RANGE calculation, but the existing local-time resolution behavior must not change.
For the example from the review:
both CQ and
GROUP BY TIMEwill select the earlier overlap offset:Tests will assert exact instant equality between CQ-generated boundaries and
GROUP BY TIMEboundaries for ordinary dates, 28/29/30/31-day month transitions, month-end clamping, leap-day anchors, DST gaps, and DST overlaps.This replaces the preferred-
anchorOffsetrule from the previous addendum.2. Make the structured wire format and mixed-version rejection enforceable
The request will carry an explicit duration encoding version together with the structured duration values. For example:
For
durationEncodingVersion = 1:everyDuration,startOffsetDuration,endOffsetDuration, andboundaryExplicitmust all be present.A request without
durationEncodingVersionis a legacy-protocol request.A new ConfigNode will reject a markerless request at the CREATE ingress instead of automatically interpreting it as fixed-only. This deliberately prevents an old DataNode from sending a calendar duration that has already been flattened to a legacy
i64value.During a rolling upgrade, this means an old DataNode cannot create a CQ through a new ConfigNode. Existing persisted CQs are unaffected and continue to run with their existing semantics.
Legacy representations encountered while loading existing procedures, plans, metadata, or snapshots remain supported and are normalized as legacy fixed-duration CQs. The rejection applies to new CQ creation at the RPC ingress, not to recovery of existing state.
Calendar-bearing version-1 requests additionally require a hard capability barrier:
The existing required legacy
i64fields remain unchanged for wire compatibility and are populated as follows:monthPart == 0, the legacy fields contain the exact fixed-duration values.monthPart, there is no valid legacy representation.0.The following cases are rejected:
This closes the old-DataNode-to-new-ConfigNode ingress hole and defines both the required legacy values and the hard reader barrier.
3. Advance
nextOccurrenceIndexusing an exact consensus CAS and fencing tokennextOccurrenceIndexis defined as the first occurrence that has not yet been durably completed or discarded.Progress advancement will use a consensus plan containing at least:
The CQ token acts as the CQ generation and fencing token. The state machine atomically applies:
The possible results are:
cqTokendoes not match, the callback is fenced and stops.expectedIndex, the state machine updates it totargetIndexand returnsADVANCED.expectedIndex, the callback is stale or duplicated. The state machine returnsSTALE, and that callback must not schedule another task.expectedIndex, the state machine returns a consistency error, and the callback must not schedule another task.For
BLOCKED:For
DISCARD, the callback time is captured exactly once after successful execution of occurrencen:This definition covers callback time exactly equal to an occurrence, delayed callbacks, multiple missed occurrences, clock rollback, and stale callbacks from a previous leader.
A callback may schedule the target occurrence only after its exact CAS returns
ADVANCED. It must not increment an in-memory occurrence index or schedule another task after a failed progress write, token mismatch,STALEresult, or consistency error.If a consensus write fails or its result is ambiguous, the callback retries the same transition using the same
cqToken,expectedIndex, andtargetIndex. It does not calculate a different target and does not rebase the transition on local state.If the first write committed but its acknowledgement was lost, the retry observes that the stored index is already greater than
expectedIndex, receivesSTALE, and stops without creating a competing schedule chain.Leader recovery and scheduler reconciliation always read the persisted
(cqToken, nextOccurrenceIndex)and install at most one task for that persisted pair. A stale callback never derives a new chain from its local execution time.The recovery flow is:
Execution remains at-least-once. If query execution succeeds but the CAS is not durably committed before a failure, recovery may execute the same occurrence again. This prevents competing schedule chains but does not introduce an exactly-once query execution protocol.
Tests will cover old- and new-leader callbacks racing on the same index, duplicated callbacks, lost or ambiguous consensus responses, token mismatch after DROP and re-CREATE with the same CQ ID, CAS retry with the same expected index,
DISCARDwith multiple missed occurrences, equality and clock rollback cases, and recovery without an off-by-one shift.4. Define a deterministic and implementable duration partial order
After normalization, a user-provided duration is represented as:
where
M >= 0is the calendar-month component,F >= 0is the fixed-tick component, and at least one component must be positive when the clause requires a positive duration.Duration-to-duration validation uses component-wise dominance:
Examples:
When a required ordering is incomparable, the statement is rejected with a clear semantic error. This partial order is used for duration-to-duration constraints such as:
No month-to-day approximation is used for these comparisons.
Validation against the fixed configuration value
continuous_query_minimum_every_intervaluses a separate deterministic elapsed-time lower bound.For an EVERY duration
(M, F):The calculation uses checked arithmetic in the configured timestamp precision. The bound is based on these platform-level limits:
Mpositive Gregorian calendar months spans at leastM * 28din local date/time.ZoneOffsetis bounded between-18:00and+18:00.The CQ is accepted only when:
Otherwise it is conservatively rejected because the minimum interval cannot be proven.
This does not reuse 28 days per month as if it were an elapsed duration. The additional 36-hour offset bound accounts for DST and larger historical offset transitions representable by the Java time model. CREATE acceptance is therefore independent of the host default time zone and of TZDB-version-specific transition data. Overflow while calculating the bound is a semantic error.
Tests will cover equal duration vectors, strict and non-strict component-wise dominance, incomparable calendar-versus-fixed durations, compound durations, minimum-EVERY values around the lower-bound threshold, and checked overflow under
ms,us, andnsprecision.5. Keep CQ aligned with the existing Tree SQL duration aliases
CQ will not introduce private
monthoryearaliases in this issue.The supported calendar units are limited to the existing Tree SQL abbreviations:
Unit matching remains case-insensitive, and
yis normalized to twelve calendar months.The following aliases are outside the scope of this issue:
If full unit aliases are added later, they should be introduced through a separate Tree SQL grammar change covering all duration positions consistently, including
GROUP BY TIME, date arithmetic,FILL,SESSION, and CQEVERY/RANGE.Therefore, a CQ will consistently use:
RESAMPLE EVERY 1mo RANGE 1mo ... GROUP BY(1mo)This replaces the previous addendum section that proposed CQ-only
monthandyearaliases.Retained implementation decisions
The following parts of the previous proposal remain unchanged:
(monthPart, nonMonthDuration)representation.1yis normalized to12mo.boundaryExplicit, zone, normalized durations, CQ token, and occurrence index are persisted.mo/yCQs are not silently migrated; recreating a CQ is the explicit opt-in to calendar semantics.TExecuteCQ.timeoutis calculated from adjacent actual occurrence instants:Updated focused test plan
The implementation tests will include:
EVERY 1mo,EVERY 1y, and inheritedGROUP BY(1mo)/GROUP BY(1y).month,months,year, andyears.GROUP BY TIMEinstant equality across ordinary dates, month ends, leap days, DST gaps, and DST overlaps.BLOCKEDandDISCARDrecovery, including equality and clock rollback.ms/us/nstimeout conversion.With these changes, DST alignment, mixed-version ingress protection, exact progress transitions, deterministic comparison rules, and Tree SQL alias consistency are implementation requirements.
DaZuiZui commentedon Aug 24, 2026
中文对照：针对评审阻塞问题的修订补充说明
感谢你这么仔细地对照当前实现审阅方案，也谢谢你把这些问题逐项指出来。你提到的几点我都认可，尤其前三项确实需要作为正确性和兼容性问题优先解决。
如果本修订与此前的方案或补充说明存在冲突，以本修订为准；此前方案中未发生冲突的部分保持不变。
1. CQ 与
GROUP BY TIME使用完全相同的日历及 DST 解析规则CQ 实现将不再引入此前提出的“DST overlap 时优先使用原始 anchor offset”的规则。
CQ occurrence 和 RANGE 的计算将复用当前
GROUP BY TIME自然月路径使用的日历转换逻辑，包括以下行为：实现将复用
DateTimeUtils.calcPositiveIntervalByMonth及其底层的TimeDuration.calcPositiveIntervalByMonth行为，或者将该行为提取为 CQ 和GROUP BY TIME共同使用的 helper。最终规则如下：
atZone规则保持一致。可以扩展公共 helper，以支持 RANGE 计算需要的溢出检查和内部有符号 duration vector，但不能改变现有的本地时间解析行为。
对于评审中给出的示例：
CQ 和
GROUP BY TIME都选择 overlap 中较早的 offset：测试将覆盖普通日期、28/29/30/31 天月份转换、月末钳制、闰日 anchor、DST gap 和 DST overlap，并断言 CQ 生成的边界 instant 与
GROUP BY TIME生成的边界 instant 完全相同。本节取代此前补充说明中的 preferred-
anchorOffset规则。2. 使结构化 wire format 和混合版本拒绝机制可以被强制执行
请求将同时携带显式的 duration 编码版本和结构化 duration。例如：
当
durationEncodingVersion = 1时：everyDuration、startOffsetDuration、endOffsetDuration和boundaryExplicit必须全部存在。没有
durationEncodingVersion的请求视为 legacy protocol 请求。新 ConfigNode 会在 CREATE 入口拒绝没有 marker 的请求，而不是自动将其解释为 fixed-only 请求。这样可以阻止旧 DataNode 把已经压平为 legacy
i64的日历 duration 发送给新 ConfigNode。因此在滚动升级期间，旧 DataNode 不能通过新 ConfigNode 创建 CQ。已经持久化的 CQ 不受影响，继续按照原有语义执行。
加载已有 procedure、plan、metadata 或 snapshot 时遇到的 legacy 表示仍然受支持，并归一化为 legacy fixed-duration CQ。上述拒绝只适用于 RPC 入口的新 CQ 创建，不适用于已有状态的恢复。
包含日历分量的 version-1 请求还必须通过强制能力屏障：
现有必填 legacy
i64字段保持不变，并按以下规则赋值：monthPart都为0时，legacy 字段填写完全相同的固定时长值。monthPart，则不存在合法的 legacy 表示。0。以下请求必须被拒绝：
这样可以堵住旧 DataNode 到新 ConfigNode 的请求入口漏洞，同时明确必填 legacy 字段的值和强制 reader barrier。
3. 使用精确的 consensus CAS 和 fencing token 推进
nextOccurrenceIndexnextOccurrenceIndex定义为第一个尚未被持久化标记为完成或丢弃的 occurrence。进度推进使用一个至少包含以下字段的 consensus plan：
CQ token 同时作为 CQ generation 和 fencing token。状态机原子地应用以下转换：
结果定义如下：
cqToken不匹配时，callback 被 fencing 并停止。expectedIndex时，状态机将其更新为targetIndex并返回ADVANCED。expectedIndex时，callback 已经过期或重复。状态机返回STALE，该 callback 不得再调度任务。expectedIndex时，状态机返回一致性错误，该 callback 不得再调度任务。对于
BLOCKED：对于
DISCARD，在 occurrencen成功执行后，只捕获一次 callback time：该定义覆盖 callback time 恰好等于某个 occurrence、callback 延迟、错过多个 occurrence、系统时钟回拨，以及前任 leader 遗留的过期 callback。
callback 只有在自己的精确 CAS 返回
ADVANCED后，才允许调度目标 occurrence。在进度写入失败、token 不匹配、返回STALE或出现一致性错误后，不得递增内存中的 occurrence index，也不得调度其他任务。如果 consensus write 失败或提交结果不明确，callback 使用相同的
cqToken、expectedIndex和targetIndex重试同一个转换。它不能计算另一个 target，也不能根据本地状态重新建立转换基准。如果第一次写入实际已经提交，只是 acknowledgement 丢失，那么重试会发现已存储 index 已大于
expectedIndex，收到STALE后停止，不会创建竞争调度链。leader 恢复和 scheduler reconciliation 始终读取持久化的
(cqToken, nextOccurrenceIndex)，并且最多为该持久化 pair 安装一个任务。过期 callback 永远不能根据自己的本地 execution time 派生新调度链。恢复流程如下：
执行语义仍然是 at-least-once。如果查询已经成功执行，但 CAS 在故障前没有被持久提交，恢复后可能再次执行同一个 occurrence。该设计用于防止竞争调度链，不引入 exactly-once 查询执行协议。
测试将覆盖：旧、新 leader callback 对同一个 index 的竞争；重复 callback；consensus 响应丢失或结果不明确；使用同一 CQ ID 执行 DROP 后重新 CREATE 引发的 token 不匹配；使用相同 expected index 的 CAS 重试；
DISCARD跳过多个 occurrence；相等和时钟回拨情况；以及无 off-by-one 偏移的恢复。4. 定义确定且可以直接实现的 duration 偏序
归一化后，用户提供的 duration 表示为：
其中
M >= 0是日历月分量，F >= 0是固定 tick 分量；当所在 clause 要求 duration 为正时，至少一个分量必须为正。duration 之间的校验使用分量支配关系：
示例：
如果某项约束要求确定的顺序，而两个 duration 不可比较，则拒绝语句并返回清晰的语义错误。该偏序用于以下 duration-to-duration 约束：
这些比较不使用任何“月转天”的近似值。
针对固定配置项
continuous_query_minimum_every_interval的校验使用一套独立且确定的 elapsed-time 下界。对于 EVERY duration
(M, F)：该计算在配置的 timestamp precision 下使用 checked arithmetic。下界基于以下平台级限制：
M个正 Gregorian 日历月，在本地日期时间上至少跨越M * 28d。ZoneOffset的范围是-18:00到+18:00。只有满足以下条件时才接受 CQ：
否则因为无法证明满足最小间隔而保守拒绝。
该规则不会把“每月 28 天”直接当成 elapsed duration 使用；额外的 36 小时 offset 界限覆盖 DST 以及 Java time model 能表示的更大历史 offset transition。因此 CREATE 的接受结果不依赖宿主机默认时区，也不依赖特定 TZDB 版本的 transition 数据。计算下界时发生溢出属于语义错误。
测试将覆盖：相等 duration vector；严格和非严格的分量支配；不可比较的日历与固定 duration；复合 duration；minimum-EVERY 下界临界值；以及
ms、us、nsprecision 下的 checked overflow。5. CQ 与现有 Tree SQL duration 别名保持一致
本 issue 不会为 CQ 引入私有的
month或year别名。支持的日历单位仅限现有 Tree SQL 缩写：
单位匹配仍然大小写不敏感，
y归一化为十二个日历月。以下别名不属于本 issue 的范围：
如果将来增加完整单位别名，应通过单独的 Tree SQL grammar 改动，在所有 duration 位置一致地引入，包括
GROUP BY TIME、日期运算、FILL、SESSION和 CQ 的EVERY/RANGE。因此一条 CQ 将统一写成：
RESAMPLE EVERY 1mo RANGE 1mo ... GROUP BY(1mo)本节取代此前补充说明中提出 CQ 私有
month和year别名的部分。保留的实现决定
此前方案中的以下内容保持不变：
(monthPart, nonMonthDuration)表示。1y归一化为12mo。boundaryExplicit、zone、归一化 durations、CQ token 和 occurrence index。mo/yCQ 不会被静默迁移；重新创建 CQ 是明确选择日历语义的方式。TExecuteCQ.timeout根据相邻两个实际 occurrence instant 计算：更新后的定向测试计划
实现测试将包括：
EVERY 1mo、EVERY 1y，以及从GROUP BY(1mo)/GROUP BY(1y)继承 EVERY。month、months、year和years。GROUP BY TIME的 instant 完全相同。BLOCKED和DISCARD恢复，包括相等和时钟回拨情况。ms/us/nstimeout 转换。完成这些修改后，DST 规则对齐、混合版本请求入口保护、精确进度转换、确定性比较规则以及 Tree SQL 别名一致性都属于实现要求。
feat: support calendar month and year CQ intervals
Remove out-of-scope mixed-version cluster capability negotiation from…
test: add P0 coverage for calendar CQ ser/de and BOUNDARY 0 semantics
refactor: eliminate duplication and FQN pollution in CQ calendar code