Skip to content

duckdb: table functions such as read_parquet, read_csv and read_json are unknown #4659

Open

Description

@xen0l

What do you want to change?

A query that reads a file with a DuckDB table function fails with unknown function.
This is a very common way to use DuckDB.
The function list in internal/engine/duckdb/dialect/functions.jsonl has no entry for these table functions.
Their columns depend on the file, so sqlc cannot know them.

Request, one of these or another idea:

  • a way to declare the columns of a table function, in the config or in the schema file, or
  • documentation that shows the recommended pattern (for example, a stand-in table with the same columns).

Output:

== FROM read_parquet('items.parquet')
# package repro
query.sql:1:1: unknown function "read_parquet"
sqlc exit code: 1
== FROM read_csv('items.csv')
# package repro
query.sql:1:1: unknown function "read_csv"
sqlc exit code: 1
== FROM read_json('items.json')
# package repro
query.sql:1:1: unknown function "read_json"
sqlc exit code: 1
repro5.sh
#!/usr/bin/env bash
rm -rf repro5 && mkdir repro5 && cd repro5
cat > sqlc.yaml <<'EOF'
version: "2"
sql:
  - engine: "duckdb"
    schema: "schema.sql"
    queries: "query.sql"
    gen:
      go:
        package: "repro"
        out: "out"
EOF
cat > schema.sql <<'EOF'
CREATE TABLE items (id INTEGER NOT NULL, name VARCHAR NOT NULL);
EOF
for fn in "read_parquet('items.parquet')" "read_csv('items.csv')" "read_json('items.json')"; do
  printf -- "-- name: Q :many\nSELECT id, name FROM %s;\n" "$fn" > query.sql
  echo "== FROM $fn"
  sqlc generate 2>&1 | head -2; echo "sqlc exit code: ${PIPESTATUS[0]}"
done

No patch: the right design is a decision for the maintainers.

What database engines need to be changed?

(none of the listed options)

What programming language backends need to be changed?

Go

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions