chdb-go with --path file did not persist the table data and it messed up display on my MacBook for some reason

[ns-gzhang]

Thank you folks for the great chdb.
I'm playing with the chdb-go CLI first to get familiar with the functionality. However, I found persistence not working no matter how I gave the file path (absolute, relative) or whether I created chdb.db file ahead of time or not. Not sure why.

Also the display was messed up after exit and can only be recovered by reset. I'm using MacOS's Terminal.

Any ideas to fix these or work around it? Thanks!

[auxten]

Just make sure, you are using the v1.8.0, right? @ns-gzhang

[ns-gzhang]

@auxten Thanks for your attention. I followed the install instructions in the repo and assume it's the latest release. I tried to build from main also, but same behavior for my test. When chdb-go exits, it seems that it didn't clean up enough from what I could see. Thanks!

Another strange thing is when I built the sample program in the README, and go fetched a very old non-released version:

go: found github.com/chdb-io/chdb-go/chdb in github.com/chdb-io/chdb-go/chdb v0.0.0-20231112193600-9a45e041749f

I'm using go version go1.22.4 darwin/amd64. I have to manually edit go.mod to get v1.8.0.
And I played with code. It never persisted the data - no files created.

One problem I encountered related to types: the CH column type Date becomes uint16 when selecting back using database/sql driver. I saw driverType is either ARROW or PARQUET so some types are lost in translation... Thanks.

Edit: I played with clickhouse-local, it has the same type changes... so seems chDB inherited the clickhouse upstream behavior. Thanks.

Edit 2: Since the MacOS became unstable after running chdb-go, I tried it on Ubuntu docker container (on two different systems, MacOS and Ubuntu 22.04). I followed the instructions to install, and turned out it was worse inside containers. It hit fatal errors always for a simple select from a parquet file:

 [ 5622 ] {} <Fatal> ClientBase:  ########## Short fault info ############
 [ 5622 ] {} <Fatal> ClientBase: (version 24.8.4.1, build id: , git hash: ) (from thread 5618) Received signal 11
 [ 5622 ] {} <Fatal> ClientBase: Signal description: Segmentation fault
 [ 5622 ] {} <Fatal> ClientBase: Address: 0x180210. Access: read. Address not mapped to object.
...

Any idea what's wrong with my test?

[chamot1111]

Same problem for me on mac os. The db file is never saved when using 1.8 golang lib and set the path on new connection.

By the way great Job.

[agoncear-mwb]

Hi @ns-gzhang @chamot1111
could you please provide a simple code so we can reproduce the error?

As for the path, i think i know where the problem is, i'll fix it in the next pr. But if you can provide me some code, i'll try to address also your issues

[auxten]

I'm working on this issue. see branch libchdb3 @agoncear-mwb

[ns-gzhang]

Thanks folks. Really appreciated!
To recreate fatal error on Ubuntu, it was very simple using the chdb-go cli with any parquet file (I copied it into a running ubuntu docker container after installing chdb and chdb-go binary ):

select * from file('students.parquet');

For non-persistence, pretty much any newly created table would reproduce:

package main

import (
	"fmt"
	"path/filepath"

	"github.com/chdb-io/chdb-go/chdb"
)

const basePath = "/Users/<user>/chdb/data" // replace <user> here

func main() {
	dataPath := filepath.Join(basePath, "mychdb_test")
	session, _ := chdb.NewSession(dataPath)
	defer session.Close()

	_, err := session.Query("CREATE DATABASE IF NOT EXISTS testdb; " +
		"CREATE TABLE IF NOT EXISTS testdb.testtable (id Int32, name String, day Date) ENGINE = MergeTree() ORDER BY id;")
	if err != nil {
		fmt.Println(err)
		return
	}

	_, err = session.Query("INSERT INTO testdb.testtable VALUES (1, 'aaaaaa', '2025-03-01'), (2, 'bbbbbbb', '2025-03-02'), (3, 'cccccc', '2025-03-03');")
	if err != nil {
		fmt.Println(err)
		return
	}
	_, err = session.Query("INSERT INTO testdb.testtable VALUES (4, 'aaaaaa', '2025-03-04'), (5, 'bbbbbbb', '2025-03-05'), (6, 'cccccc', '2025-03-06');")
	if err != nil {
		fmt.Println(err)
		return
	}
}

After running, check the data/ directory, nothing there. Thanks. (same result for using database/sql interface)