Reuse TensorFlow protobuf messages

[wangkuiyi]

We are considering reusing the TensorProto message defined in TensorFlow. This requires us to be able to build TensorFlow's .proto files into Go source code. This guide explains how to do this.

Install Protobuf Tools

To convert .proto files into .go and .py files, we need protoc, the protobuf compilers, and its Go code generator.

• To install protoc, we can use this shell script https://github.com/sql-machine-learning/sqlflow/blob/develop/scripts/docker/install-protobuf.bash.

• To install Go plugin, we can take this script as a reference and run the following command:

  go get github.com/golang/protobuf/protoc-gen-go

Compile Protobuf Messages

1. We can check out the TensorFlow source tree to any local directory, say, /tmp/tensorflow, by running the following command:

   git clone https://github.com/tensorflow/tensorflow /tmp/tensorflow

2. Make sure that we set the environment variable $GOPATH, say, ~/go.

3. To compiler all .proto files, which reside in tensorflow/core/framework, please type the following command.

   for i in tensorflow/core/framework/*.proto; do 
       protoc --go_out=$GOPATH/src -I $PWD $i; 
   done

   The generated Go source files are in the directory $GOPATH/src/github.com/tensorflow/tensorflow/tensorflow/go/core/framework/, because of in all these .proto files, TensorFlow authors put lines like the following:

   option go_package = "github.com/tensorflow/tensorflow/tensorflow/go/core/framework/tensor_go_proto";

Use the Protobuf Messages in Go Code

Let us create an example project in $GOPATH/src/github.com/wangkuiyi/aaa or anywhere in the $GOPATH/src directory, and put in there the following Go source file a.go.

package main

import (
	"fmt"

	"github.com/tensorflow/tensorflow/tensorflow/go/core/framework/tensor_shape_go_proto"
)

func main() {
	var a tensor_shape_go_proto.TensorShapeProto
	fmt.Println(a.GetDim())
}

We can now install dependencies and build this program.

cd $GOPATH/src/github.com/wangkuiyi/aaa
go get ./...
go install
$GOPATH/bin/aaa