Skip to content

[Feature] Support bucket commit in paimon #2641

Open

Description

@FangYongs

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

Currently Paimon will create a global commit operator for flink job which makes all operators in the job as one region. When a task failed, it causes global failover. I'd like to support bucket commit in paimon, then we can build commit operator for each sink and support region failover in flink jobs

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

  1. FangYongs commented on Jan 5, 2024

    @FangYongs
    ContributorAuthor

    @JingsongLi What do you think of this issue

  2. microbearz commented on Jul 23, 2024

    @microbearz

    @FangYongs any update here?

  3. FangYongs commented on Jan 23, 2025

    @FangYongs
    ContributorAuthor

    @microbearz I have created a pip https://cwiki.apache.org/confluence/display/PAIMON/PIP-30%3A+Improvement+For+Paimon+Committer+In+Flink for this issue and started a thread, please feel free to discuss there, THX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions