TensorFlow 2.3 SavedModel: Operation Not Found

[justnoxx]

Hello!

After 0.16 version has been released we decided to switch to TF2.

The issue is that it can't find the op by name even if this OP is present in the graph (checked with tensorboard and dumped model).

Could someone please help me or show me where I am wrong? Or it is better for now go back to TF1?

My system:

1. macOS 10.15.7
2. cargo 1.45.0
3. tensorflow: 2.3.0
4. tensorflow-rust: 1.16.1

To reproduce it, please, use the examples.

This code produces the simple Sequential model using Keras:

import tensorflow as tf;
import tensorboard;
from tensorflow.python import keras
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

import json;
import numpy as np;


tensorboard_callback = keras.callbacks.TensorBoard(log_dir="/tmp/logs")

classifier = Sequential()
classifier.add(Dense(5, activation='relu', kernel_initializer='random_normal', name="test_in", input_dim=5))
classifier.add(Dense(5, activation='relu'))
classifier.add(Dense(1, activation='sigmoid', name="test_out"))



classifier.compile(optimizer ='adam',loss='binary_crossentropy', metrics = ['accuracy'])


classifier.fit([[0.1, 0.2, 0.3, 0.4, 0.5]], [[1]], batch_size=1, epochs=1, callbacks=[tensorboard_callback]);
result = classifier.predict([[0.1, 0.2, 0.3, 0.4, 0.5]])

print(result);
classifier.summary();

for layer in classifier.layers:
    print(layer.name)

classifier.save('/tmp/saved_model', save_format='tf')

This rust code tries to load this model and calculate something (please, note that I also tried test_in as input name, the result is the same):

use tensorflow::{SessionRunArgs, Graph, Session, SessionOptions, Tensor};

fn main() {
    let network_path = "/tmp/saved_model";
    let v: Vec<f32> = vec![0.1, 0.2, 0.3, 0.4, 0.5];
    let input_layer = "test_in_input";
    let output_layer = "test_out_output";


    let tensor = vector_to_tensor(&v);
    let mut graph = Graph::new();
    let session = Session::from_saved_model(
        &SessionOptions::new(),
        &["serve"],
        &mut graph,
        network_path,
    ).unwrap();


    let mut args = SessionRunArgs::new();
    let input_op = &graph.operation_by_name_required(input_layer).unwrap();
    args.add_feed(&input_op, 0, &tensor);
    let out = args.request_fetch(&graph.operation_by_name_required(output_layer).unwrap(), 0);

    let result = session.run(&mut args);
    if result.is_err() {
        panic!("Error occured during calculations: {:?}", result);
    }
    let out_res:f32 = args.fetch(out).unwrap()[0];
    println!("Results: {:?}", out_res);
}

pub fn vector_to_tensor(v: &Vec<f32>) -> Tensor<f32> {
    let dimension = v.len();
    let tensor = Tensor::new(&[1, dimension as u64]).with_values(&v[..]).unwrap();
    return tensor;
}

But it returns the following error:

2020-11-10 23:27:02.348764: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /tmp/saved_model
2020-11-10 23:27:02.351523: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2020-11-10 23:27:02.353879: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-11-10 23:27:02.370771: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe9b469d6f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-11-10 23:27:02.370812: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-11-10 23:27:02.389267: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
2020-11-10 23:27:02.466215: I tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /tmp/saved_model
2020-11-10 23:27:02.477538: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 128781 microseconds.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: {inner:0x7fe9d5b08c00, Unavailable: Operation "test_in_input" not found}', src/main.rs:21:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Thanks.

[justnoxx]

saved_model.tar.gz

There is a screenshot from tensorboard (it shows that my input exists) and archive with saved model.

Please, let me know what information I need to provide or how can I help.

Thanks.

[adamcrume]

I think what TensorBoard is displaying is not the operation names (or at least not always), but the names from the signature, instead. The way to fix the Rust code is to load the signature and get the operation names from it. Take a look at how the eval function in https://github.com/tensorflow/rust/blob/master/examples/xor.rs works. In your case, though, it looks like REGRESS_METHOD_NAME, REGRESS_INPUTS, and REGRESS_OUTPUTS should be replaced with "serving_default", "test_in_input", and "test_out". I'll mark Session::from_saved_model as deprecated, because SavedModelBundle::load is strictly more powerful.

…

On Tue, Nov 10, 2020 at 1:42 PM Dmitriy Shamatrin ***@***.***> wrote: [image: Screenshot 2020-11-10 at 23 38 32] <https://user-images.githubusercontent.com/2946069/98736993-44c56180-23ae-11eb-9683-6c2bcf4fa0f7.png> [saved_model.tar.gz]( https://github.com/tensorflow/rust/files/5520332/saved_model.tar.gz) There is a screenshot from tensorboard and archive with saved model. Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#282 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIU7RLESKBN5REJMQECCNDSPGXUPANCNFSM4TRGBEEA> .

[adamcrume]

By the way, the mailing list is a good place to ask questions like this.

[justnoxx]

@adamcrume thanks! Next time I will use the mailing list for questions, but this particular looked like a bug for me, but it works now.

Can I create an example with Keras and raise a pull request to have it in the examples list to avoid questions like this in the future?

[adamcrume]

Yes, a Keras example would definitely be welcome!

…

On Thu, Nov 12, 2020, 12:21 AM Dmitriy Shamatrin ***@***.***> wrote: @adamcrume <https://github.com/adamcrume> thanks! Next time I will use the mailing list for questions, but this particular looked like a bug for me, but it works now. Can I create an example with Keras and raise a pull request to have it in the examples list to avoid questions like this in the future? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#282 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIU7RKDOFGY375DRAAEWNDSPOLI3ANCNFSM4TRGBEEA> .