Skip to content

Removing some dependency edges from Blob to other caffe2 #11923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 90 commits into from

Conversation

smessmer
Copy link
Contributor

@smessmer smessmer commented Sep 21, 2018

Stack:
    :black_circle:  #11923 Removing some dependency edges from Blob to other caffe2  💚
    :white_circle:  #11924 Move Blob to ATen/core  💛
    :white_circle:  #11414 IValue can store Blob  💚
    :white_circle:  #11548 Blob doesn't allow access to destroyCall anymore  💚
    :white_circle:  #11500 Use TypeMeta::dtor() instead of Blob::DestroyCall  💛
    :white_circle:  #11925 Serialization takes pointers instead of Blob  💛
    :white_circle:  #11926 Remove Blob::ShareExternal from serialization  💛

This is pre-work to allow moving Blob to ATen/core, which cannot depend on caffe2 anymore.
(1) Removing the Blob -> Tensor dependency allows us to move Blob to ATen/core and use it inside IValue without having to wait for the Tensor merge to be complete.
(2) In the final Blob design, we want it to be a very small class that doesn't have any special treatment for Tensor (or to be more correct, doesn't allow storing Tensor anymore), , so this is anyhow the direction we want to go.

This changes call sites that will have to be moved to IValue later, but they cannot be moved to IValue directly, because for that, IValue first needs to be able to store Blob, which in turn first needs this diff and some other changes coming up in future diffs.

Codemods:
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)\.IsTensorType\(" "BlobIsTensorType(\1, "
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)->IsTensorType\(" "BlobIsTensorType(\1, "
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)\.GetMutableTensor\(" "BlobGetMutableTensor(\1, "
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)->GetMutableTensor\(" "BlobGetMutableTensor(
\1, "

It is, however, not only these codemods because regex based refactoring was only able to match a small amount of the call sites. To catch more, I wouldn've needed a AST aware tool like clangr, which I didn't figure out how to use.

Differential Revision: D9979976

Differential Revision: D9623916
Differential Version: 56705898
Differential Revision: D9623916
Differential Version: 56858422
Differential Revision: D9644700
Differential Version: 56870160
Differential Revision: D9623916
Differential Version: 56900314
Differential Revision: D9644700
Differential Version: 56900331
Differential Revision: D9652088
Differential Version: 56917655
Differential Revision: D9652088
Differential Version: 56918713
Differential Revision: D9652089
Differential Version: 56918714
Differential Revision: D9623916
Differential Version: 56982469
Differential Revision: D9644700
Differential Version: 56982494
Differential Revision: D9652088
Differential Version: 56982465
Differential Revision: D9652089
Differential Version: 56982470
Differential Revision: D9663476
Differential Version: 56982467
Differential Revision: D9623916
Differential Version: 57101548
Differential Revision: D9644700
Differential Version: 57101652
Differential Revision: D9652088
Differential Version: 57101545
Differential Revision: D9652089
Differential Version: 57101550
Differential Revision: D9663476
Differential Version: 57101556
Differential Revision: D9623916
Differential Version: 57135223
Differential Revision: D9644700
Differential Version: 57135224
Differential Revision: D9652088
Differential Version: 57135218
Differential Revision: D9652089
Differential Version: 57135219
Differential Revision: D9663476
Differential Version: 57135226
Differential Revision: D9694327
Differential Version: 57135227
Differential Revision: D9694326
Differential Version: 57135216
Differential Revision: D9694918
Differential Version: 57140103
Differential Revision: D9623916
Differential Version: 57214033
Differential Revision: D9644700
Differential Version: 57214023
Differential Revision: D9652088
Differential Version: 57214005
Differential Revision: D9652089
Differential Version: 57214030
@ezyang
Copy link
Contributor

ezyang commented Sep 22, 2018

I'm a little confused. Why would having these methods on Blob prevent movement to ATen/core? Tensor is going to live in ATen/core, and I can't think of anything else these methods could refer to that you can't reference in core?

@smessmer
Copy link
Contributor Author

It's two things. (1) it unblocks me from having to wait until Tensor merge is complete (because before that, blob references c2 tensor), and (2) we want to do this anyhow because the final blob design doesn't let it store tensor anymore.

@ezyang
Copy link
Contributor

ezyang commented Sep 24, 2018

ghexport has messed this one up too. What it looks like is that you updated export-D9882726 without also reexporting this one. A gh from the top of the stack should fix it.

Differential Revision: D9979976
Differential Version: 58691284
@smessmer smessmer changed the title [wip] Removing some dependency edges from Blob to other caffe2 Removing some dependency edges from Blob to other caffe2 Sep 24, 2018
@smessmer smessmer changed the base branch from export-D9882726 to master September 24, 2018 15:38
@smessmer
Copy link
Contributor Author

I think this might have been broken by landing the diff below. I re-exported it.

Differential Revision: D9979976
Differential Version: 58749277
@soumith soumith deleted the export-D9979976 branch February 21, 2019 23:24
@ezyang ezyang added the merged label Jun 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants