Update capture profile contract to allow FE start and stop

[qqfish]

Right now to capture profile in TensorBoard, user is asked set an duration in prior. But a better way is to have a start and stop button that allow users to start or stop profiling at any time they want. It can also avoid holding the request open for the duration of profiling.

[stephanwlee]

@qiuminxu What changes do you think needs to be made to enable this feature? Thanks!

[qiuminxu]

We need to add a GRPC call between GCE vm and TPU to cancel/stop a profile. For GPU, I guess a similar GPRC interface needed to be added in GPU profiler server. We decided not to support this feature for TPU because TPU could generate hundreds of MBs of traces in just one second, and when the profile getting too large, the rest of the trace has to be dropped. We assume the traces around the point where the user stops the traces are more interesting, and those will most likely be dropped. We also had an continuous tracing mode, where the last few seconds' trace will be kept, but that is currently broken due to TPU specific symbolization requirement.