-
Notifications
You must be signed in to change notification settings - Fork 96
Add Model.save(); Let loadModel() support IOHandlers #161
Conversation
* Model.save() uses IOHandler.save() to save model as artifacts. * Model.load() uses IOHandler.load() to get artifacts and construct a Model object. * Using dummy implementations of IOHandler in unit tests. Towards: tensorflow/tfjs#13
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commented several places regarding the IOHandler|string conventions, since it didn't match what I was expecting from the design doc.
After this PR I feel we're in a weird place and we probably want to push more into it, or pull saving out.
As I understand it now we
a) still accept the old load string format, with no changes to behavior (even though I think we require changes to match the design doc)
b) don't accept strings on save (explicit error)
@@ -155,8 +155,8 @@ export class ModelExports { | |||
subheading: 'Loading', | |||
useDocsFrom: 'loadModelInternal' | |||
}) | |||
static loadModel(modelConfigPath: string): Promise<Model> { | |||
return loadModelInternal(modelConfigPath); | |||
static loadModel(pathOrIOHandler: string|io.IOHandler): Promise<Model> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the design settled on is not backward compatible, so we'll need to make sure we bump the middle version number, right? (While we still accept strings like before, we require a prefix that we didn't in the past)
src/models.ts
Outdated
* See the Python converter function `save_model()` for more details. | ||
* | ||
* It is also assumed that model weights can be accessed from relative | ||
* paths described by the `paths` fields in weights manifest. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
formatting
src/models.ts
Outdated
@@ -125,7 +190,7 @@ export interface ModelAndWeightsConfig { | |||
* @returns A `Promise` of `Model`, with the topology and weights loaded. | |||
*/ | |||
// TODO(cais): Add link to the core's documentation of `WeightManifestConfig`. | |||
export async function loadModelInternal(modelConfigPath: string): | |||
export async function loadModelFromPath(modelConfigPath: string): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at my earlier API comment, I think we're missing something here to support the scheme prefixed string sugaring. Or is this an interim solution that's accepting raw strings without scheme information?
src/models.ts
Outdated
* | ||
* // Load the model back and use it to do prediction, which should yield | ||
* // the same result as the prediction result above. | ||
* model2 = await tf.loadModel('localstorage://SavedByModelSaveCodeSnippet'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This shows the scheme I was expecting, but I don't think the string load side handles it yet?
Review status: 0 of 7 files reviewed at latest revision, 4 unresolved discussions. src/exports.ts, line 158 at r1 (raw file): Previously, ericdnielsen wrote…
This is backward compatible. Currently we support Also, yes, the release will involve a minor (middle) version bump. src/models.ts, line 193 at r1 (raw file): Previously, ericdnielsen wrote…
This is interim. Scheme-based URL-like string will be supported. The error thrown below is just a temporary src/models.ts, line 665 at r1 (raw file): Previously, ericdnielsen wrote…
Correct. I just opted to write the code snippet according the final state we want to be in at the release, even though this is not supported yet. Comments from Reviewable |
Review status: 0 of 7 files reviewed at latest revision, 4 unresolved discussions. src/models.ts, line 126 at r1 (raw file): Previously, ericdnielsen wrote…
Done. Comments from Reviewable |
Reviewed 2 of 7 files at r1. src/models.ts, line 121 at r2 (raw file):
When referring to objects in other repositories, do we typically offer links? src/models.ts, line 132 at r2 (raw file):
Just a question here. I thought TypeScript handled this type of inference automatically. Do these have any function beyond clarity to the reader (which is good enough purpose for this reviewer). src/models.ts, line 673 at r2 (raw file):
Great job on the clarity of the documentation!!! src/models_test.ts, line 387 at r2 (raw file):
nit sp. Comments from Reviewable |
Review status: 2 of 7 files reviewed at latest revision, 8 unresolved discussions, all commit checks successful. src/model_save_test.ts, line 21 at r2 (raw file):
remove this line src/model_save_test.ts, line 29 at r2 (raw file):
Handler src/model_save_test.ts, line 30 at r2 (raw file):
remove this line src/models.ts, line 665 at r1 (raw file): Previously, caisq (Shanqing Cai) wrote…
I wouldn't add this until we actually support it. src/models.ts, line 132 at r2 (raw file): Previously, bileschi (Stanley Bileschi) wrote…
+1 I think this should be inferred src/models.ts, line 161 at r2 (raw file):
pull false / true out to variables for readability. src/models.ts, line 630 at r2 (raw file):
it's not really saving it to an IOHandler, it's using an IOHandler. What about something more human like:
And then go into detail in the next lines. src/models.ts, line 690 at r2 (raw file):
name these params src/engine/topology.ts, line 993 at r2 (raw file):
Comments from Reviewable |
Review status: 2 of 7 files reviewed at latest revision, 15 unresolved discussions, all commit checks successful. src/model_save_test.ts, line 21 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Done. src/model_save_test.ts, line 30 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Done. src/models.ts, line 665 at r1 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
OK. I saved this on my computer. And I'll add it back later. src/models.ts, line 121 at r2 (raw file): Previously, bileschi (Stanley Bileschi) wrote…
Good point. Unfortunately we don't have Python doc up on js.tensorflow.org yet. I added a link to a tutorial page on js.tensorflow.org and a link to keras doc instead. src/models.ts, line 132 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Done. src/models.ts, line 161 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Done. src/models.ts, line 630 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Good point. Done. Comments from Reviewable |
Review status: 2 of 7 files reviewed at latest revision, 15 unresolved discussions, all commit checks successful. src/models.ts, line 690 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Done. src/models_test.ts, line 387 at r2 (raw file): Previously, bileschi (Stanley Bileschi) wrote…
Done. src/engine/topology.ts, line 993 at r2 (raw file): Previously, nsthorat (Nikhil Thorat) wrote…
Good point. Done. Comments from Reviewable |
a Model object.
Also in this change:
Towards: tensorflow/tfjs#13
This change is