-
Notifications
You must be signed in to change notification settings - Fork 12.9k
Open
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedNeeds help from the communityNeeds help from the communitysplitGGUF split model shardingGGUF split model sharding
Description
Motivation
we support --split-max-tensors
since:
As mentionned by @Artefact2 in this comment:
allowing to split by file size would be more intuitive (and usually more appropriate since file size is usually the limiting factor, eg 4G for FAT or 50G for HF)
Proposition:
Introduce --split-max-size N(M|G)
split strategy to split files in file with a max size of N Megabytes or Gigabytes.
As it is not possible to have less than 1 tensor per GGUF, this size is a soft limit.
Artefact2
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedNeeds help from the communityNeeds help from the communitysplitGGUF split model shardingGGUF split model sharding