Column with List(Struct) causes failed to decode level data for struct array

[valkum]

Describe the bug

I have a file generated using polars which polars, duckdb and other tools read fine. arrow-rs fails with a Failed to decode level data for struct array error.

To Reproduce

Try to read the following file: https://storage.googleapis.com/ids-next/arrow-bug-dremel-encoding.parquet e.g. via

pub(crate) fn bug() -> () {
    let src = "https://storage.googleapis.com/ids-next/arrow-bug-dremel-encoding.parquet";
    let mut reader = File::open(src.strip_prefix("file://").unwrap()).unwrap();

    let metadata = ArrowReaderMetadata::load(&mut reader, ArrowReaderOptions::default()).unwrap();

    let mut market_reader =
        ParquetRecordBatchReaderBuilder::new_with_metadata(reader, metadata.clone())
            .build()
            .unwrap();

    let mut count = 0;

    while let Some(batch) = market_reader.next() {
        let batch = batch.unwrap();
        count += batch.num_rows();
    }

    println!("Read {} rows", count);

    ()
}

Expected behavior

The file should read or provide a better error if the file contains something unsupported.

Additional context

I tried to come up with a unit test for struct_array.rs but ran out of time.
I tried this with a struct of only ints, but that didn't trigger the bug; thus, I assume the dict might be part of the cause.

[alamb]

Here is the reproducer:

arrow-bug-dremel-encoding.parquet.zip

DataFusion CLI indeed has an error:

DataFusion CLI v50.0.0
> select * from 'arrow-bug-dremel-encoding.parquet';
Parquet error: Arrow: Parquet argument error: Parquet error: Failed to decode level data for struct array

DuckDB can read this file

D select * from read_parquet('arrow-bug-dremel-encoding.parquet');
┌─────────────┬─────────┬───────────┬─────────────┬─────────┬─────────────────┬──────────────────────┬────────────┬─────────────────────────────────────────────────────┐
│ registrable │  etld   │ is_market │ is_expiring │  zone   │ security_trails │       markets        │   domain   │                      embedding                      │
│   varchar   │ varchar │  boolean  │   boolean   │ boolean │     boolean     │ struct(market varc…  │  varchar   │                       float[]                       │
├─────────────┼─────────┼───────────┼─────────────┼─────────┼─────────────────┼──────────────────────┼────────────┼─────────────────────────────────────────────────────┤
│ en13549     │ eu      │ true      │ false       │ false   │ true            │ [{'market': sedo.c…  │ en13549.eu │ [-0.056253448, -0.026599279, -0.05077901, -0.0911…  │
└─────────────┴─────────┴───────────┴─────────────┴─────────┴─────────────────┴──────────────────────┴────────────┴─────────────────────────────────────────────────────┘
D