Extended fs_driver.py with support for directories (patch attached) + fixed lv_fs.c + fixed file_explorer

[ThomasFarstrike]

I added support for directories to your fs_driver.py so that the file_explorer widget is able to browse the filesystem.

Here's the proposed patch:

--- ./api_drivers/py_api_drivers/fs_driver.py   2025-07-05 01:29:19.923626386 +0200
+++ ../MicroPythonOS/internal_filesystem/lib/mpos/fs_driver.py  2025-07-05 01:53:28.199800772 +0200
@@ -74,6 +74,41 @@
 
     return lv.FS_RES.OK
 
+def _fs_dir_open_cb(drv, path):
+    #print(f"_fs_dir_open_cb for path '{path}'")
+    try:
+        import os # for ilistdir()
+        return {'iterator' : os.ilistdir(path)}
+    except Exception as e:
+        print(f"_fs_dir_open_cb exception: {e}")
+        return None
+
+def _fs_dir_read_cb(drv, lv_fs_dir_t, buf, btr):
+    try:
+        iterator = lv_fs_dir_t.__cast__()['iterator']
+        nextfile = iterator.__next__()
+        #print(f"nextfile: {nextfile}")
+        filename = nextfile[0]
+        entry_type = nextfile[1]  # Type field
+        if entry_type == 0x4000:
+            #print(f"{filename} is a directory")
+            filename = f"/{filename}"
+        # Convert filename to bytes with null terminator
+        tmp_data_bytes = filename.encode() + b'\x00'
+        buf.__dereference__(btr)[0:len(tmp_data_bytes)] = tmp_data_bytes
+        return lv.FS_RES.OK
+    except StopIteration:
+        # Clear buffer and return FS_ERR when iteration ends
+        buf.__dereference__(btr)[0:1] = b'\x00'  # Empty string (null byte)
+        return lv.FS_RES.NOT_EX  # Next entry "does not exist"
+    except Exception as e:
+        print(f"_fs_dir_read_cb exception: {e}")
+        return lv.FS_RES.UNKNOWN
+
+def _fs_dir_close_cb(drv, lv_fs_dir_t):
+    #print(f"_fs_dir_close_cb called")
+    # No need to cleanup the iterator so nothing to do
+    return lv.FS_RES.OK
 
 def fs_register(fs_drv, letter, cache_size=500):
 
@@ -85,6 +120,9 @@
     fs_drv.seek_cb = _fs_seek_cb
     fs_drv.tell_cb = _fs_tell_cb
     fs_drv.close_cb = _fs_close_cb
+    fs_drv.dir_open_cb = _fs_dir_open_cb
+    fs_drv.dir_read_cb = _fs_dir_read_cb
+    #fs_drv.dir_close_cb = _fs_dir_close_cb
 
     if cache_size >= 0:
         fs_drv.cache_size = cache_size

I validated it on the unix and esp32 targets with something like this:

import fs_driver
fs_drv = lv.fs_drv_t()
fs_driver.fs_register(fs_drv, 'M')
file_explorer = lv.file_explorer(lv.screen_active())
file_explorer.explorer_open_dir('M:/')
def file_explorer_event_cb(event):
    path = file_explorer.explorer_get_current_path()
    file = file_explorer.explorer_get_selected_file_name()
    print(f"Selected: {path}{file}")
file_explorer.add_event_cb(file_explorer_event_cb, lv.EVENT.VALUE_CHANGED, None)

One major gotcha is that the current LVGL dir_read_cb's "fn" argument (an output buffer for the next filename) is defined as a char * instead of void *, which results in a UnicodeError when a C char * containing non-printable characters like 0x00 is converted to a MicroPython str by the binding. Also, str doesn't have the __dereference__() function etc...

Here's my fix for that issue:

diff --git a/src/misc/lv_fs.c b/src/misc/lv_fs.c
index f713698..9d07b45 100644
--- a/src/misc/lv_fs.c
+++ b/src/misc/lv_fs.c
@@ -341,19 +341,19 @@ lv_fs_res_t lv_fs_dir_open(lv_fs_dir_t * rddir_p, const char * path)
     return LV_FS_RES_OK;
 }
 
-lv_fs_res_t lv_fs_dir_read(lv_fs_dir_t * rddir_p, char * fn, uint32_t fn_len)
+lv_fs_res_t lv_fs_dir_read(lv_fs_dir_t * rddir_p, void * fn, uint32_t fn_len)
 {
     if(fn_len == 0) {
         return LV_FS_RES_INV_PARAM;
     }
 
     if(rddir_p->drv == NULL || rddir_p->dir_d == NULL) {
-        fn[0] = '\0';
+        *(char *)fn = 0x00;
         return LV_FS_RES_INV_PARAM;
     }
 
     if(rddir_p->drv->dir_read_cb == NULL) {
-        fn[0] = '\0';
+        *(char *)fn = 0x00;
         return LV_FS_RES_NOT_IMP;
     }
 
diff --git a/src/misc/lv_fs.h b/src/misc/lv_fs.h
index c6f938b..500e405 100644
--- a/src/misc/lv_fs.h
+++ b/src/misc/lv_fs.h
@@ -79,7 +79,7 @@ struct lv_fs_drv_t {
     lv_fs_res_t (*tell_cb)(lv_fs_drv_t * drv, void * file_p, uint32_t * pos_p);
 
     void * (*dir_open_cb)(lv_fs_drv_t * drv, const char * path);
-    lv_fs_res_t (*dir_read_cb)(lv_fs_drv_t * drv, void * rddir_p, char * fn, uint32_t fn_len);
+    lv_fs_res_t (*dir_read_cb)(lv_fs_drv_t * drv, void * rddir_p, void * fn, uint32_t fn_len);
     lv_fs_res_t (*dir_close_cb)(lv_fs_drv_t * drv, void * rddir_p);
 
     void * user_data; /**< Custom file user data*/
@@ -210,7 +210,7 @@ lv_fs_res_t lv_fs_dir_open(lv_fs_dir_t * rddir_p, const char * path);
  * @param fn_len    length of the buffer to store the filename
  * @return          LV_FS_RES_OK or any error from lv_fs_res_t enum
  */
-lv_fs_res_t lv_fs_dir_read(lv_fs_dir_t * rddir_p, char * fn, uint32_t fn_len);
+lv_fs_res_t lv_fs_dir_read(lv_fs_dir_t * rddir_p, void * fn, uint32_t fn_len);
 
 /**
  * Close the directory reading

I wanted to get a second opinion before proposing this change to upstream LVGL, so don't hold back ;-) Maybe there's a way to make it work without changing the function signature?

Another issue I encountered is that the file_explorer's navigation was buggy when entering/leaving directories repeatedly, and to fix that I cherry-picked the fix from upstream.

Anyway, the file_explorer seems to work well now, so that's going to make a nice addition to MicroPythonOS ;-) Ty!

[kdschlosser]

Do me a favor, open up a pr for all of the changes made to this repo to get it work work up to the point of needing to alter things in LVGL. I have a feeling that we may need to make an adjustment in the binding code generator and not in LVGL itself to handle the string issue.

All dereference does is it converts a pointer to a memoryview object. and I believe the issue is the char* is being interperted as a string instead of it being a buffer that needs to be filled. a str in python is immutable and is not something that is able to be changed. What I mean by that is the following...

def some_function(some_string: str):
    some_string = 'some new string text'


string_data = 'string data'

print(string_data)
some_function(string_data)
print(string_data)

That has the output of

string data
string data

BUT if you do this...

def some_function(some_string: bytearray):
    some_string[:20] = bytearray(b'some new string text')


string_data = bytearray(20)
string_data[:11] = bytearray(b'string data')

print(string_data.decode('utf-8'))
some_function(string_data)
print(string_data.decode('utf-8'))

the output ends up being

string data
some new string text

the latter is mutable.

The binding code generator sees all char * types as python str types instead of it simply being a pointer to a location in memory that is able to have the contents of that memory address altered.

what needs to be done is we need to look at specifically are the function prototypes for callback functions and see how many of them in LVGL take a char* as one of the parameter types. We need to look at what the impact would be to the binding to change the char* type handling in a callback function to a "Blob" type would effect other callback functions.

I don't foresee there being a large number of function prototypes in LVGL taking a char * parameter so the impact might be minimal. we have to look before we leap so to speak just to make sure.

[kdschlosser]

Now here some the fun part tho. if lvgl is expecting the callback to fill the char* pointer then chances are the memory is not going to be allocated for that pointer. that would mean that there is also a length parameter that must also be filled. This is where we have a little bit of a stumble. Allocating the memory is easy but we will need to hold a reference to that memory after the callback function exits otherwise the garbage collection in MicroPython will release it. But how long do we need to hold the reference for that memory? How are we able to determine when it is safe to remove that reference?

[ThomasFarstrike]

Sure! Please check out #399

About the memory allocation you touched on, I'll post the the relevant code from lib/lvgl/src/others/file_explorer/lv_file_explorer.c below.

It allocates the memory (char fn[LV_FILE_EXPLORER_PATH_MAX_LEN]) on the stack, the way I see it:

static void show_dir(lv_obj_t * obj, const char * path)
{
    lv_file_explorer_t * explorer = (lv_file_explorer_t *)obj;

    char fn[LV_FILE_EXPLORER_PATH_MAX_LEN];
    uint16_t index = 0;
    lv_fs_dir_t dir;
    lv_fs_res_t res;

    res = lv_fs_dir_open(&dir, path);
    if(res != LV_FS_RES_OK) {
        LV_LOG_USER("Open dir error %d!", res);
        return;
    }

    lv_table_set_cell_value_fmt(explorer->file_table, index++, 0, LV_SYMBOL_DIRECTORY "  %s", ".");
    lv_table_set_cell_value_fmt(explorer->file_table, index++, 0, LV_SYMBOL_DIRECTORY "  %s", "..");
    lv_table_set_cell_value(explorer->file_table, 0, 1, "0");
    lv_table_set_cell_value(explorer->file_table, 1, 1, "0");

    while(1) {
        res = lv_fs_dir_read(&dir, fn, sizeof(fn));
        if(res != LV_FS_RES_OK) {
            LV_LOG_USER("Driver, file or directory is not exists %d!", res);
            break;
        }

        /*fn is empty, if not more files to read*/
        if(lv_strlen(fn) == 0) {
            LV_LOG_USER("Not more files to read!");
            break;
        }

        if((is_end_with(fn, ".png") == true)  || (is_end_with(fn, ".PNG") == true)  || \
           (is_end_with(fn, ".jpg") == true) || (is_end_with(fn, ".JPG") == true) || \
           (is_end_with(fn, ".bmp") == true) || (is_end_with(fn, ".BMP") == true) || \
           (is_end_with(fn, ".gif") == true) || (is_end_with(fn, ".GIF") == true)) {
            lv_table_set_cell_value_fmt(explorer->file_table, index, 0, LV_SYMBOL_IMAGE "  %s", fn);
            lv_table_set_cell_value(explorer->file_table, index, 1, "1");
        } // ...removed a few cases for brevity...
        else {
            lv_table_set_cell_value_fmt(explorer->file_table, index, 0, LV_SYMBOL_FILE "  %s", fn);
            lv_table_set_cell_value(explorer->file_table, index, 1, "4");
        }

        index++;
    }

    lv_fs_dir_close(&dir);

    lv_table_set_row_count(explorer->file_table, index);
    file_explorer_sort(obj);
    lv_obj_send_event(obj, LV_EVENT_READY, NULL);

    /*Move the table to the top*/
    lv_obj_scroll_to_y(explorer->file_table, 0, LV_ANIM_OFF);

    lv_strncpy(explorer->current_path, path, sizeof(explorer->current_path));
    lv_label_set_text_fmt(explorer->path_label, LV_SYMBOL_EYE_OPEN" %s", path);

    size_t current_path_len = lv_strlen(explorer->current_path);
    if((explorer->current_path[current_path_len - 1] != '/') && (current_path_len < LV_FILE_EXPLORER_PATH_MAX_LEN)) {
        *((explorer->current_path) + current_path_len) = '/';
    }
}

[kdschlosser]

yes it is allocating it on the stack. so that takes care of handling the allocation aspect of it which is good.

now we need to check and see how many function prototypes for callbacks have char * as any of the parameter types.

we also know that the buffer need to contain the string data and if that string data doesn't fill the entire buffer then the remaining bytes need to be written as null characters. That is how LVGL determines where the end of the string is. You can see that call being made here...

if(lv_strlen(fn) == 0) {

There is no pointer passed to the callback where we would need to populate it with the length of the string.

OK so what you would need to do is this...

in the file gen/python_api_gen.mpy.py change the build_callback_func_arg function so it reads

def build_callback_func_arg(arg, index, func, func_name = None):
    arg_type = get_type(arg.type, remove_quals = True)
    cast = '(void*)' if isinstance(arg.type, c_ast.PtrDecl) else '' # needed when field is const. casting to void overrides it

    if arg_type not in lv_to_mp or not lv_to_mp[arg_type]:
        try_generate_type(arg.type)
        if arg_type not in lv_to_mp or not lv_to_mp[arg_type]:
            raise MissingConversionException("Callback: Missing conversion to %s" % arg_type)
    
    arg_metadata = {'c_type': arg_type, 'py_type': get_py_type(arg_type)}

    if arg.name:
        arg_metadata['name'] = arg.name
    else:
        arg_metadata['name'] = None

    callback_metadata[func_name]['args'].append(arg_metadata)

    if arg_type == 'char *':
        converter = 'ptr_to_mp'
    else:
        converter = lv_to_mp[arg_type]
        
    return 'mp_args[{i}] = {convertor}({cast}arg{i});'.format(
                convertor = converter,
                i = index, cast = cast)

You will have to dereference the fn parameter passing the length that is supplied via the fn_len parameter
You will need to convert the "file name" into a bytes object and then pad the end of the bytes object with null characters so the bytes object has the length that was passed in the fn_len parameter. The filename cannot be longer than fn_len - 1 because you must have at least one null at the end.

Then convert that bytes object into a bytearray and set it into the returned memoryview when you dereferenced the fn parameter.

That should solve the problem. It may cause issues if char * is a type for any other callback parameters. I am pretty confident in saying that I do not think that there is going to be an issue. I believe that strings passed to callback functions would have the type const char * and not char * easy way to find out is by trying it!!!

[ThomasFarstrike]

Thanks for spoon-feeding it to me! ;-) That really helps. I didn't realize you were doing so much of the binding generation code yourself - I assumed it had already been created and lvgl_micropython was just making it all work, making it easy to compile, fixing stuff. Super impressed!

I quickly tried it as you proposed, but indeed, it seems to affect const char * from things like lv_fs_res_t lv_fs_open(lv_fs_file_t * file_p, const char * path, lv_fs_mode_t mode) and lv_fs_res_t lv_fs_dir_open(lv_fs_dir_t * rddir_p, const char * path) because now those are throwing a "can't convert 'Blob' object to str implicitly" exception, whereas this Just Worked (tm) before.

I haven't made time to study the full chain yet but wanted to give this first feedback already, maybe it rings a bell :-)

Here's the resulting output from lib/micropython/ports/unix/build-standard/lv_mpy.c:

/* Debugging I added:

arg_type: lv_fs_drv_t *
cast: (void*)
converter: mp_read_ptr_lv_fs_drv_t
arg_type: char *
cast: (void*)
converter: ptr_to_mp
arg_type: lv_fs_mode_t
cast:
converter: mp_obj_new_int
*/

/*
 * Callback function lv_fs_drv_t_open_cb
 * void *open_cb(lv_fs_drv_t *drv, const char *path, lv_fs_mode_t mode)
 */

GENMPY_UNUSED static void * lv_fs_drv_t_open_cb_callback(lv_fs_drv_t *arg0, const char *arg1, lv_fs_mode_t arg2)
{
    mp_obj_t mp_args[3];
    mp_args[0] = mp_read_ptr_lv_fs_drv_t((void*)arg0);
    mp_args[1] = ptr_to_mp((void*)arg1);
    mp_args[2] = mp_obj_new_int(arg2);
    mp_obj_t callbacks = get_callback_dict_from_user_data(arg0->user_data);
    _nesting++;
    mp_obj_t callback_result = mp_call_function_n_kw(mp_obj_dict_get(callbacks, MP_OBJ_NEW_QSTR(MP_QSTR_lv_fs_drv_t_open_cb)) , 3, 0, mp_args);
    _nesting--;
    return mp_to_ptr(callback_result);
}

...

#define funcptr_open_cb NULL


/*
 * lvgl extension definition for:
 * void *open_cb(lv_fs_drv_t *drv, const char *path, lv_fs_mode_t mode)
 */

static mp_obj_t mp_funcptr_open_cb(size_t mp_n_args, const mp_obj_t *mp_args, void *lv_func_ptr)
{
    lv_fs_drv_t *drv = mp_to_ptr(mp_args[0]);
    const char *path = (const char *)(char*)convert_from_str(mp_args[1]);
    lv_fs_mode_t mode = (int)mp_obj_get_int(mp_args[2]);
    void * _res = ((void *(*)(lv_fs_drv_t *, const char *, lv_fs_mode_t))lv_func_ptr)(drv, path, mode);
    return ptr_to_mp((void*)_res);
}



static MP_DEFINE_CONST_LV_FUN_OBJ_STATIC_VAR(mp_funcptr_open_cb_mpobj, 3, mp_funcptr_open_cb, funcptr_open_cb);

static inline mp_obj_t mp_lv_funcptr_open_cb(void *func){ return mp_lv_funcptr(&mp_funcptr_open_cb_mpobj, func, NULL, MP_QSTR_, NULL); }

static void * lv_fs_drv_t_open_cb_callback(lv_fs_drv_t *drv, const char *path, lv_fs_mode_t mode);

[kdschlosser]

OK I will need to make a special case so the const doesn't get stripped off this way we can tell the difference between the 2. I believe that is going to be the ticket to knowing if it is a string being passed to a callback or if it is a string buffer that needs to be filled that is being passed to a callback.

change that function in the code generation script to the following..

def build_callback_func_arg(arg, index, func, func_name = None):
    arg_type = get_type(arg.type, remove_quals = False)
    cast = '(void*)' if isinstance(arg.type, c_ast.PtrDecl) else '' # needed when field is const. casting to void overrides it

    if arg_type == 'char *':
        converter = 'ptr_to_mp'
    else:
        arg_type = get_type(arg.type, remove_quals = True)
        
        if arg_type not in lv_to_mp or not lv_to_mp[arg_type]:
            try_generate_type(arg.type)
            if arg_type not in lv_to_mp or not lv_to_mp[arg_type]:
                raise MissingConversionException("Callback: Missing conversion to %s" % arg_type)

        converter = lv_to_mp[arg_type]
    
    arg_metadata = {'c_type': arg_type, 'py_type': get_py_type(arg_type)}

    if arg.name:
        arg_metadata['name'] = arg.name
    else:
        arg_metadata['name'] = None

    callback_metadata[func_name]['args'].append(arg_metadata)

    return 'mp_args[{i}] = {convertor}({cast}arg{i});'.format(
                convertor = converter,
                i = index, cast = cast)

[kdschlosser]

I am pretty sure we are on the right track with how to make this work. Just going to have to dink about with the code a little bit to get it figured out.

[ThomasFarstrike]

Works like a charm! And all the other code and apps look okay, no regressions, and it has pretty good coverage, so I think this is resolved.

I pushed the additional commit to the pull request.

Thanks so much for your help - I learned a lot from this!

[kdschlosser]

It can get tricky working with converting from C data type to Python types. There are all kinds of tricks that can be used to get the job done. The C API for MicroPython is pretty forgiving and it can be manipulated in different ways to get an outcome that is desired.

The code generator script I personally did not write. I have added to it, fixed bugs in it and did some small cleanup. It really needs to be redone as it is a mess that is really hard to follow. It was written by someone that is a C programmer and that is able to be easily seen by looking at the way the code is written.

The code generator uses a python library called pycparser to read the preprocessed output from a compiler. This is basically all of the header files merged into a single header file. The issue with pycparser is it doesn't allow different objects to look backwards through the objects that are created it is only able to look forward. That is the reason why you see a crazy number of iterations being done in the code generator.

What I have discovered as a better way is to use pycparser to read the preprocessor output and then use it to generate a normalized header file where each declaration whether it be a function prototype, structure, union or enum it outputs that object on a single line.

as an example..

this is copied from an LVGL header file..

typedef struct {
    int32_t x;
    int32_t y;
} lv_point_t;

typedef struct {
    lv_value_precise_t x;
    lv_value_precise_t y;
} lv_point_precise_t;

/** Represents an area of the screen.*/
typedef struct {
    int32_t x1;
    int32_t y1;
    int32_t x2;
    int32_t y2;
} lv_area_t;

This is far more complicated to read programmatically than this...

typedef struct {int32_t x;int32_t y;} lv_point_t;
typedef struct {lv_value_precise_t x;lv_value_precise_t y;} lv_point_precise_t;
typedef struct {int32_t x1;int32_t y1;int32_t x2;int32_t y2;} lv_area_t;

I am not a giant fan of using regex because it's hard to understand what is happening. The code to read the normalized output seen above would be pretty simple.