Copy large file from Assets Folder to Application Documents Folder

[iKK001]

Steps to Reproduce

Hi,
I try to use a large db-file. And therefore, my code copies this file from the Assets folder to the Application-Documents-Folder as can be seen in the code-example below.

For small files, my code works perfectly fine and I can work with my DB perfectly fine.

However, if I try it with a large db-File (>665MB), things start to break as for my Android-part of my Flutter app. (in fact, iOS works perfectly fine, no matter how large the db-File really is....). Only for Android, a large db-File breaks things.

The following Dart code does not work for a large File (>665MB) - (however it works for small files):

  Future<Database> initializeDatabase() async {

    Directory directory = await getApplicationDocumentsDirectory();
    var dbPath = join(directory.path, "ch_departures.db");

    // copy db file from Assets folder to Documents folder (only if not already there...)    
    if (FileSystemEntity.typeSync(dbPath) == FileSystemEntityType.notFound) {
     ByteData data = await rootBundle.load("assets/ch_departures.db");
     writeToFile(data, dbPath);
    }

    // Open/create the database at a given path
    var departuresDatabase = await openDatabase(dbPath);
    return departuresDatabase;
  }

  // HERE IS WHERE THE CODE CRASHES (WHEN TRYING TO WRITE THE LOADED BYTES)
  void writeToFile(ByteData data, String path) {
    final buffer = data.buffer;
    return new File(path).writeAsBytesSync(
      buffer.asUint8List(data.offsetInBytes, data.lengthInBytes));
  }

I tried to play around with the gradle.properties File of the Android part of my Flutter App.

Inside gradle.properties, I tried:

org.gradle.jvmargs=-Xmx3g -XX:MaxPermSize=2048m

But unfortunately, no success.

Do you have any idea on how to make all this work with a large (>665MB) File ????

[zoechi]

Also asked on https://stackoverflow.com/questions/54160771/inside-flutter-copy-large-file-from-assets-folder-to-application-documents-fold

[kangwang1988]

@iKK001

But unfortunately, no success.

Any detail or log, how it fails. It fails when building? Or it builds successfully but fails to run the logic properly?

Besides, flutter doctor -v is always required to help to diagnose it.

[zoechi]

I think there should be an

Stream<List<int>> openRead ([
int start,
int end
])

on AssetBundle like
https://api.dartlang.org/stable/2.1.0/dart-io/File/openRead.html
so that it is not necessary to load the whole file into memory for such a purpose.

A better way for this concrete use case would be to use a native file copy method
like https://stackoverflow.com/questions/9292954/how-to-make-a-copy-of-a-file-in-android, https://stackoverflow.com/questions/6545180/ios-copy-a-file-in-documents-folder

[iKK001]

I finally made (what @zoechi suggested also) - I created a Flutter plugin that does the desired large-File copy natively in iOS and Android respectively. My large_file_copy Flutter plugin can be found here.

[dblqt]

I came across the same use case.
Having an SQLite database to deploy on first run.
Now iKK001 solution does work, kudos, but a native Flutter solution would be nice in the long run.

Thanks,
Stefan

[EArminjon]

We got the same issue on our app. It was for the database. We decided to split this database into small file of 50mb each (db_part01, db_part02, etc). Our database size was around 400mb. App stop crashing and this system is not very slow.

We are still looking for a better solution...

[Soap-141]

Any update?

[jwinarske]

@iKK001 Why not put the kennel to work and implement using memory mapped files in Dart?

dart-lang/sdk#35070 (comment)