Add linalg.lu_factor #66933

lezcano · 2021-10-20T11:02:11Z

Stack from ghstack:

This PR exposes torch.lu as torch.linalg.lu_factor and
torch.linalg.lu_factor_ex.

This PR also adds support for matrices with zero elements both in
the size of the matrix and the batch. Note that this function simply
returns empty tensors of the correct size in this case.

We add a test and an OpInfo for the new function.

This PR also adds documentation for this new function in line of
the documentation in the rest of torch.linalg.

Fixes #56590
Fixes #64014

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @lezcano

Differential Revision: D32834069

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. [ghstack-poisoned]

pytorch-probot · 2021-10-20T11:02:14Z

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/60e23935476eb2eaa9b79ef5fe6c7b07d61f8c92/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default,ciflow/all

Workflows	Labels (bold enabled)	Status
Triggered Workflows
caffe2-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
docker-builds	`ciflow/all`, `ciflow/trunk`	✅ triggered
ios-12-5-1-arm64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-arm64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-arm64-custom-ops	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-arm64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-arm64-metal	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-x86-64	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-x86-64-coreml	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
ios-12-5-1-x86-64-full-jit	`ciflow/all`, `ciflow/ios`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
libtorch-linux-xenial-cuda10.2-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
libtorch-linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-bionic-cuda10.2-py3.9-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/slow`, `ciflow/trunk`	✅ triggered
linux-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/noarch`, `ciflow/trunk`	✅ triggered
linux-docs	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/docs`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-docs-push	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/scheduled`	✅ triggered
linux-vulkan-bionic-py3.7-clang9	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`, `ciflow/vulkan`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test	`ciflow/all`, `ciflow/bazel`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-build	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static	`ciflow/all`, `ciflow/default`, `ciflow/linux`, `ciflow/mobile`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-asan	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/sanitizers`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-clang7-onnx	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/onnx`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
linux-xenial-py3.7-gcc7	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
macos-10-15-py3-arm64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
macos-10-15-py3-lite-interpreter-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
macos-11-py3-x86-64	`ciflow/all`, `ciflow/macos`, `ciflow/trunk`	✅ triggered
parallelnative-linux-xenial-py3.7-gcc5.4	`ciflow/all`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	✅ triggered
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/libtorch`, `ciflow/linux`, `ciflow/scheduled`	✅ triggered
periodic-linux-bionic-cuda11.5-py3.7-gcc7	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	✅ triggered
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`, `ciflow/slow`, `ciflow/slow-gradcheck`	✅ triggered
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug	`ciflow/all`, `ciflow/cuda`, `ciflow/linux`, `ciflow/scheduled`	✅ triggered
periodic-win-vs2019-cuda11.1-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	✅ triggered
periodic-win-vs2019-cuda11.5-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/scheduled`, `ciflow/win`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit	`ciflow/all`, `ciflow/android`, `ciflow/cpu`, `ciflow/default`, `ciflow/linux`, `ciflow/trunk`	✅ triggered
win-vs2019-cpu-py3	`ciflow/all`, `ciflow/cpu`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
win-vs2019-cuda11.3-py3	`ciflow/all`, `ciflow/cuda`, `ciflow/default`, `ciflow/trunk`, `ciflow/win`	✅ triggered
Skipped Workflows

You can add a comment to the PR and tag @pytorchbot with the following commands:

# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

facebook-github-bot · 2021-10-20T11:02:22Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/66933
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit 60e2393 (more details on the Dr. CI page):

1/1 failures possibly* introduced in this PR
- 1/1 non-scanned failure(s)

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm4.3.1-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. [ghstack-poisoned]

@IvanYashchuk

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. cc jianyuh nikitaved pearu mruberry walterddr @IvanYashchuk xwang233 @lezcano [ghstack-poisoned]

@IvanYashchuk

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. cc jianyuh nikitaved pearu mruberry walterddr @IvanYashchuk xwang233 @lezcano [ghstack-poisoned]

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano [ghstack-poisoned]

torch/linalg/__init__.py

aten/src/ATen/native/cuda/BatchLinearAlgebraLib.cpp

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano [ghstack-poisoned]

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #56590 Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano Differential Revision: [D32834069](https://our.internmc.facebook.com/intern/diff/D32834069) [ghstack-poisoned]

mruberry · 2021-12-17T19:08:00Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

mruberry · 2021-12-18T02:35:38Z

Still getting:

/aten/src/ATen/native/cuda/BatchLinearAlgebra.cpp:1858:77: error: expected ';' after do/while statement
  AT_ERROR("linalg.lu_factor: PyTorch was not compiled with MAGMA support.")

internally. We should also rebase and retest the stack (at least up to maybe_multiply) because later parts are hitting a merge conflict, too.

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #56590 Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano Differential Revision: [D32834069](https://our.internmc.facebook.com/intern/diff/D32834069) [ghstack-poisoned]

mruberry · 2021-12-19T21:56:41Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #56590 Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano Differential Revision: [D32834069](https://our.internmc.facebook.com/intern/diff/D32834069) [ghstack-poisoned]

mruberry · 2021-12-27T22:23:04Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

mruberry · 2021-12-28T23:42:07Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #56590 Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano Differential Revision: [D32834069](https://our.internmc.facebook.com/intern/diff/D32834069) [ghstack-poisoned]

mruberry · 2022-01-04T02:36:33Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #56590 Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano Differential Revision: [D32834069](https://our.internmc.facebook.com/intern/diff/D32834069) [ghstack-poisoned]

mruberry · 2022-01-04T15:07:35Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This PR exposes `torch.lu` as `torch.linalg.lu_factor` and `torch.linalg.lu_factor_ex`. This PR also adds support for matrices with zero elements both in the size of the matrix and the batch. Note that this function simply returns empty tensors of the correct size in this case. We add a test and an OpInfo for the new function. This PR also adds documentation for this new function in line of the documentation in the rest of `torch.linalg`. Fixes #56590 Fixes #64014 cc jianyuh nikitaved pearu mruberry walterddr IvanYashchuk xwang233 Lezcano Differential Revision: [D32834069](https://our.internmc.facebook.com/intern/diff/D32834069) [ghstack-poisoned]

mruberry · 2022-01-05T18:19:45Z

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@jeffdaily

- This move later magma commit, fixes many memory leaks which were found while running linalg UT on ROCm 5.0 release. These memory issues were exposed by the PR #66933 cc @jeffdaily @jithunnair-amd Pull Request resolved: #73906 Approved by: https://github.com/jithunnair-amd, https://github.com/seemethere, https://github.com/malfet, https://github.com/osalpekar

@jeffdaily

- This move later magma commit, fixes many memory leaks which were found while running linalg UT on ROCm 5.0 release. These memory issues were exposed by the PR #66933 cc @jeffdaily @jithunnair-amd Pull Request resolved: #73906 Approved by: https://github.com/jithunnair-amd, https://github.com/seemethere, https://github.com/malfet, https://github.com/osalpekar

lezcano requested review from albanD, ezyang, IvanYashchuk, nikitaved and soulitzer as code owners October 20, 2021 11:02

pytorch-probot bot added the ciflow/default label Oct 20, 2021

This was referenced Oct 20, 2021

Remove check_errors from LU #66932

Closed

Make linalg.lu_factor structured #66934

Closed

facebook-github-bot added the cla signed label Oct 20, 2021

pytorchbot added the open source label Oct 20, 2021

lezcano added 2 commits October 20, 2021 13:56

albanD removed their request for review October 20, 2021 16:15

lezcano removed request for ezyang and soulitzer October 20, 2021 18:24

lezcano added the module: linear algebra label Oct 20, 2021

lezcano added 3 commits October 20, 2021 18:36

lezcano mentioned this pull request Oct 21, 2021

Remove unnecessary sync in linalg.det #67014

Closed

IvanYashchuk reviewed Oct 21, 2021

View reviewed changes

torch/linalg/__init__.py Show resolved Hide resolved

IvanYashchuk reviewed Oct 21, 2021

View reviewed changes

torch/linalg/__init__.py Outdated Show resolved Hide resolved

lezcano commented Oct 21, 2021

View reviewed changes

aten/src/ATen/native/cuda/BatchLinearAlgebraLib.cpp Show resolved Hide resolved

lezcano added 4 commits October 21, 2021 15:57

lezcano added 3 commits December 14, 2021 22:42

lezcano added 4 commits December 21, 2021 12:21

lezcano mentioned this pull request Dec 24, 2021

lu_solve will frequently crash #70292

Closed

lezcano mentioned this pull request Jan 5, 2022

torch.inverse and torch.lu_solve give wrong results for singular matrices #48572

Closed

facebook-github-bot closed this in a35b4b4 Jan 6, 2022

lezcano mentioned this pull request Jan 7, 2022

Actually actionable list of batching rules to write pytorch/functorch#240

Open

16 tasks

facebook-github-bot deleted the gh/Lezcano/20/head branch January 9, 2022 15:16

lezcano mentioned this pull request Jan 13, 2022

Cuda related autograd tests fail with RuntimeError: Jacobian mismatch for output 0 with respect to input 0 #53446

Closed

nikitaved mentioned this pull request Feb 9, 2022

linalg_det_singular: modify samples such that CUDA IMA dissapears. #72585

Closed

jithunnair-amd mentioned this pull request Mar 15, 2022

[ROCm] Update the magma commit #73906

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add linalg.lu_factor #66933

Add linalg.lu_factor #66933

lezcano commented Oct 20, 2021 •

edited

Loading

pytorch-probot bot commented Oct 20, 2021 •

edited

Loading

⚛️ CI Flow

facebook-github-bot commented Oct 20, 2021 •

edited

Loading

mruberry commented Dec 17, 2021

mruberry commented Dec 18, 2021

mruberry commented Dec 19, 2021

mruberry commented Dec 27, 2021

mruberry commented Dec 28, 2021

mruberry commented Jan 4, 2022

mruberry commented Jan 4, 2022

mruberry commented Jan 5, 2022

Add linalg.lu_factor #66933

Add linalg.lu_factor #66933

Conversation

lezcano commented Oct 20, 2021 • edited Loading

pytorch-probot bot commented Oct 20, 2021 • edited Loading

⚛️ CI Flow

facebook-github-bot commented Oct 20, 2021 • edited Loading

🔗 Helpful links

💊 CI failures summary and remediations

ci.pytorch.org: 1 failed

mruberry commented Dec 17, 2021

mruberry commented Dec 18, 2021

mruberry commented Dec 19, 2021

mruberry commented Dec 27, 2021

mruberry commented Dec 28, 2021

mruberry commented Jan 4, 2022

mruberry commented Jan 4, 2022

mruberry commented Jan 5, 2022

lezcano commented Oct 20, 2021 •

edited

Loading

pytorch-probot bot commented Oct 20, 2021 •

edited

Loading

facebook-github-bot commented Oct 20, 2021 •

edited

Loading