Skip to content

Memory jobs inherit the interactive thread's Fast Mode / priority service tier #32429

Open

Description

@lattwood

What issue are you seeing?

Background memory extraction and consolidation jobs can inherit the foreground thread's Fast Mode / priority service tier.

Memory maintenance is background work and should use the default service tier unless a memory-specific tier has been explicitly configured. Inheriting priority unexpectedly increases usage for requests the user did not intentionally accelerate.

The two memory stages currently inherit the tier through different configuration paths:

  • Phase 1 reads the live thread configuration snapshot and copies its service_tier into StageOneRequestContext.
  • Phase 2 clones the thread's original Config, including its service_tier, when creating the consolidation agent.

The phase-1 request builder filters the tier against the selected extraction model. The default extraction model, gpt-5.4-mini, currently advertises no accelerated service tier, so priority is normally omitted for that model.

However, memories.extract_model is configurable. If it is changed to gpt-5.4 or another model that supports priority, phase 1 can also send the inherited priority tier.

Phase 2 uses gpt-5.4 by default, and that model advertises priority, so phase-2 consolidation can use the foreground thread's priority tier under the default model configuration.

Relevant code:

  • Phase 1 copies the live tier:

    pub(crate) async fn stage_one_request_context(
    &self,
    config: &Config,
    model_name: &str,
    reasoning_effort: ReasoningEffort,
    ) -> StageOneRequestContext {
    let config_snapshot = self.thread.config_snapshot().await;
    let model_info = self
    .thread_manager
    .get_models_manager()
    .get_model_info(model_name, &config.to_models_manager_config())
    .await;
    let reasoning_summary = config
    .model_reasoning_summary
    .unwrap_or(model_info.default_reasoning_summary);
    StageOneRequestContext {
    model_info,
    session_telemetry: build_session_telemetry(
    &self.auth_manager,
    self.thread_id,
    config,
    config_snapshot.session_source,
    model_name,
    config_snapshot.originator,
    ),
    reasoning_effort: Some(reasoning_effort),
    reasoning_summary,
    service_tier: config_snapshot.service_tier,
    }

  • Phase 1 passes it to the model client:

    let mut stream = client_session
    .stream(
    prompt,
    &context.model_info,
    &context.session_telemetry,
    context.reasoning_effort.clone(),
    context.reasoning_summary,
    context.service_tier.clone(),
    &responses_metadata,
    &InferenceTraceContext::disabled(),
    )

  • The request builder retains a tier when the selected model advertises support:

    pub fn service_tier_for_request(&self, service_tier: Option<String>) -> Option<String> {
    service_tier.filter(|service_tier| {
    service_tier != SERVICE_TIER_DEFAULT_REQUEST_VALUE
    && self.supports_service_tier(service_tier)
    })
    }

    let service_tier = model_info.service_tier_for_request(service_tier);
    let request = ResponsesApiRequest {
    model: model_info.slug.clone(),
    instructions,
    input,
    tools,
    tool_choice: "auto".to_string(),
    parallel_tool_calls: prompt.parallel_tool_calls && !model_info.use_responses_lite,
    reasoning,
    store: provider.is_azure_responses_endpoint(),
    stream: true,
    stream_options,
    include,
    service_tier,
    prompt_cache_key,
    text,
    client_metadata: Some(responses_metadata.client_metadata()),
    };

  • Phase 2 clones the parent config without clearing service_tier:

    pub(super) fn get_config(config: &Config, provider: &dyn ModelProvider) -> Option<Config> {
    let root = memory_root(&config.codex_home);
    let mut agent_config = config.clone();
    agent_config.cwd = root.clone();
    // Consolidation threads must never feed back into phase-1 memory generation.
    agent_config.ephemeral = true;
    agent_config.memories.generate_memories = false;
    agent_config.memories.use_memories = false;
    agent_config.include_apps_instructions = false;
    agent_config.mcp_servers = Constrained::allow_only(HashMap::new());
    // Approval policy
    agent_config.permissions.approval_policy = Constrained::allow_only(AskForApproval::Never);
    // Consolidation runs as an internal worker and must not recursively delegate.
    let _ = agent_config.features.disable(Feature::SpawnCsv);
    let _ = agent_config.features.disable(Feature::Collab);
    let _ = agent_config.features.disable(Feature::MemoryTool);
    let _ = agent_config.features.disable(Feature::Apps);
    let _ = agent_config.features.disable(Feature::Plugins);
    let _ = agent_config
    .features
    .disable(Feature::SkillMcpDependencyInstall);
    // Sandbox policy
    let writable_roots = vec![root];
    // The consolidation agent only needs local memory-root write access and no network.
    let consolidation_sandbox_policy = SandboxPolicy::WorkspaceWrite {
    writable_roots,
    network_access: false,
    exclude_tmpdir_env_var: true,
    exclude_slash_tmp: true,
    };
    agent_config
    .set_legacy_sandbox_policy(consolidation_sandbox_policy)
    .ok()?;
    agent_config.model = Some(
    config
    .memories
    .consolidation_model
    .clone()
    .unwrap_or_else(|| provider.memory_consolidation_preferred_model().to_string()),
    );
    agent_config.model_reasoning_effort = Some(crate::stage_two::REASONING_EFFORT);
    Some(agent_config)

  • The default consolidation model is gpt-5.4:

    /// Returns the preferred model used for memory consolidation.
    ///
    /// Providers that require backend-specific model IDs should override this.
    fn memory_consolidation_preferred_model(&self) -> &'static str {
    DEFAULT_MEMORY_CONSOLIDATION_PREFERRED_MODEL
    }

  • gpt-5.4 advertises priority:

    ],
    "supports_search_tool": true,
    "default_service_tier": null,
    "service_tiers": [
    {
    "id": "priority",
    "name": "Fast",
    "description": "1.5x speed, increased usage"
    }
    ],

What steps can reproduce the bug?

  1. Enable memory generation.

  2. Enable Fast Mode so the foreground thread's effective service tier is priority.

  3. Configure both memory stages to use a model that supports priority:

service_tier = "fast"

[memories]
extract_model = "gpt-5.4"
consolidation_model = "gpt-5.4"
  1. Create enough eligible thread history for memory extraction and consolidation to run.

  2. Start another foreground turn, which launches the memory startup task.

  3. Inspect the background Responses requests or associated usage. The memory jobs can carry service_tier: "priority" even though no memory-specific accelerated tier was selected.

With the default extraction model, the phase-1 tier is filtered because gpt-5.4-mini does not currently advertise priority support. Changing memories.extract_model to a priority-capable model exposes the same behavior in phase 1. Phase 2 is affected with its default gpt-5.4 model.

What is the expected behavior?

Background memory jobs should not implicitly inherit the interactive thread's accelerated service tier.

Unless a memory-specific service tier is explicitly configured, both memory extraction and memory consolidation should use the default service tier. On the wire, this should mean omitting service_tier from the Responses request.

The behavior should be consistent across both phases and independent of whether the selected memory model happens to support priority.

Additional information

There is also an inconsistency in where the two phases obtain their configuration:

  • Phase 1 reads thread.config_snapshot() and therefore sees live thread-level service-tier changes.
  • Phase 2 receives thread.config(), which resolves to original_config_do_not_use, and therefore follows the original configuration rather than necessarily following the current live thread override.

Relevant paths:

  • Memory startup passes thread.config():

    if turn_has_input {
    let config_snapshot = thread.config_snapshot().await;
    codex_memories_write::start_memories_startup_task(
    Arc::clone(&self.thread_manager),
    Arc::clone(&self.auth_manager),
    thread_id,
    Arc::clone(&thread),
    thread.config().await,
    &config_snapshot.session_source,
    );

  • thread.config() returns the original session config:

    pub async fn config(&self) -> Arc<crate::config::Config> {
    self.codex.session.get_config().await
    }

    pub(crate) async fn get_config(&self) -> std::sync::Arc<Config> {
    let state = self.state.lock().await;
    state
    .session_configuration
    .original_config_do_not_use
    .clone()
    }

The existing phase-1 test asserts that the request context inherits the live tier, but it does not assert the final serialized service_tier in the request body:

async fn memories_startup_phase1_uses_live_thread_service_tier_and_detached_metadata()
-> anyhow::Result<()> {
let server = start_mock_server().await;
let home = Arc::new(TempDir::new()?);
let test = build_test_codex(&server, home).await?;
assert_eq!(test.config.service_tier, None);
reset_git_repository(&test.config.cwd).await?;
core_test_support::submit_thread_settings(
&test.codex,
codex_protocol::protocol::ThreadSettingsOverrides {
service_tier: Some(Some(ServiceTier::Fast.request_value().to_string())),
..Default::default()
},
)
.await?;
let config_snapshot =
wait_for_service_tier(&test, Some(ServiceTier::Fast.request_value().to_string())).await?;
assert_eq!(
config_snapshot.service_tier,
Some(ServiceTier::Fast.request_value().to_string())
);
let context = crate::runtime::MemoryStartupContext::new(
Arc::clone(&test.thread_manager),
test.thread_manager.auth_manager(),
test.session_configured.thread_id,
Arc::clone(&test.codex),
&test.config,
config_snapshot.session_source.clone(),
);
let request_context = context
.stage_one_request_context(
&test.config,
test.config.model.as_deref().unwrap_or("gpt-5.4-mini"),
ReasoningEffort::Low,
)
.await;
assert_eq!(
request_context.service_tier,
Some(ServiceTier::Fast.request_value().to_string())
);
let stage_one = mount_sse_once(
&server,
sse(vec![
ev_response_created("resp-phase1"),
ev_assistant_message("msg-phase1", "phase1 complete"),
ev_completed("resp-phase1"),
]),
)
.await;
context
.stream_stage_one_prompt(
&test.config,
&codex_core::Prompt::default(),
&request_context,
)
.await?;
let request = wait_for_single_request(&stage_one).await;
let metadata_header = request
.header("x-codex-turn-metadata")
.expect("detached memory request should include workspace metadata");
let metadata: serde_json::Value =
serde_json::from_str(&metadata_header).expect("turn metadata json");
assert_eq!(metadata["request_kind"].as_str(), Some("memory"));
assert!(metadata.get("session_id").is_none());
assert!(metadata.get("thread_id").is_none());
assert!(metadata.get("turn_id").is_none());
assert!(metadata.get("window_id").is_none());
assert!(metadata.get("workspaces").is_some());
shutdown_test_codex(&test).await?;
Ok(())

The phase-2 request tests assert the selected model but do not cover service-tier behavior:

#[tokio::test]
async fn memories_startup_phase2_provider_default_drives_request_model() -> anyhow::Result<()> {
let server = start_mock_server().await;
let home = Arc::new(TempDir::new()?);
let request =
run_memory_phase_two_model_request_test(&server, home, startup_test_memories_config())
.await?;
assert_eq!(
request.body_json()["model"].as_str(),
Some(MOCK_PROVIDER_PHASE_TWO_MODEL)
);
Ok(())
}

It would be useful to add request-body regression coverage for both phases.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app-serverIssues involving app server protocol or interfacesbugSomething isn't workingmemory

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions