Skip to content

[camerax] Implement lockCaptureOrientation & unlockCaptureOrientation #5285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 75 commits into from
Jan 2, 2024

Conversation

camsim99
Copy link
Contributor

@camsim99 camsim99 commented Oct 31, 2023

Implements lockCaptureOrientation & unlockCaptureOrientation for all camera UseCases.

Also fixes small bug concerning not initially setting the target rotation of the UseCases to the requested sensor orientation when createCamera is called.

Fixes flutter/flutter#125915.

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
  • I signed the CLA.
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I listed at least one issue that this PR fixes in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
  • I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

camsim99 added 30 commits May 1, 2023 16:46
@@ -708,7 +779,7 @@ class AndroidCameraCameraX extends CameraPlatform {
width: imageProxy.width);

weakThis.target!.cameraImageDataStreamController!.add(cameraImageData);
unawaited(imageProxy.close());
await imageProxy.close();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future<void> _onFrameStreamListen(int cameraId) async {
await _configureImageAnalysis(cameraId);
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just changed _onFrameStreamListen reference to _configureImageAnalysis.

@camsim99 camsim99 requested a review from gmackall December 11, 2023 22:43
/// Should be specified in terms of one of the [Surface]
/// rotation constants that represents the counter-clockwise degrees of
/// rotation relative to [DeviceOrientation.portraitUp].
final int? initialTargetRotation;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reflecting on this now and wondering if we should even save the target rotation for UseCases and instead expose methods like https://developer.android.com/reference/androidx/camera/core/ImageCapture#getTargetRotation() that allow us to retrieve the target rotation for each of them. (this is a side effect of these wrapper classes needing to be immutable)

@gmackall lmk what you think. I would probably do this in a follow up PR because this has a lot going on already. Running into a similar thing in the last wrappings I need to do and want to take a similar path.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I support replacing with the native method, but am also fine with it being a follow up!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok great! Filed flutter/flutter#140664 to follow up.

Copy link
Member

@gmackall gmackall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments! My main question is making sure we are managing the "unlocked" state for a given UseCase in the same way that it behaves if it is created and never set/unset.

final int targetLockedRotation =
_getRotationConstantFromDeviceOrientation(orientation);

// Update UseCases to use target device orientation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Preview usecase also has a method to set the target rotation, should we be setting that as well?
https://developer.android.com/reference/kotlin/androidx/camera/core/Preview#setTargetRotation(int)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe so since this is only supposed to impact the capture orientation. The preview rotation should be managed by the Flutter app using the plugin like, for example, our example app.

} else {
return Configuration.ORIENTATION_PORTRAIT;
}
int getDefaultRotation() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I believe this method is named getDefaultRotation because it is the rotation that CameraX use cases (at least ImageCapture) default to right? I think we might want to name it something else, though, because without that context from the comment here, it mostly gets used in places where it looks like we are getting a default rotation of the camera proxy, (i.e. proxy.getDefaultRotation()) which at first glance doesn't seem like something that would be a function of the current rotation of the device, but rather a persistent default rotation.

Basically it just feels more like a getCurrentRotation than a getDefaultRotation, to me at least. Not a big deal, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To address this, I renamed the wrapped method to getDefaultDisplayRotation() and left this since this really a part of the DeviceOrientationManager and would like to keep it consistent with camera_android for now.

// Set target rotation to current photo orientation only if capture
// orientation not locked.
if (!captureOrientationLocked && shouldSetDefaultRotation) {
await imageCapture!.setTargetRotation(await proxy.getDefaultRotation());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In testing, does this achieve the same behavior between "plugin-managed" unlocked case (i.e. lock->unluck), and the "camerax" managed unlocked case? I'm curious because after reading the documentation on ImageCapture's setTargetRotation, particularly:

If no target rotation is set by the application, it is set to the value of getRotation of the default display at the time the use case is bound.

To me this sounds like we would be doing "rotation of default display at time of photo", while camerax is doing "rotation of default display at time of use case creation", which sounds like it would be different if the phone was rotated after creation. Am I misinterpreting the docs, or do the docs not line up with what happens when we test the use cases?

(this comment applies to how we manage the rotation for all use cases, I just put it here as an example)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that the default display does not change anywhere in the plugin and the rotation of that display always will return the rotation needed to return to its "natural" orientation (https://developer.android.com/reference/android/view/Display.html#getRotation()). From my testing, this yields the desired result of matching camera_android behavior.

This makes sense to me because my interpretation of "To return to the default value, set the value to context.getSystemService(WindowManager.class).getDefaultDisplay().getRotation();" was that you should be able to do that at anytime to have the UseCase return to its default rotation by CameraX standards. I think there would be a concern here if the default display were changing between binding UseCases and setting the target rotation, but like I mentioned, I don't believe it should be a concern of ours.

/// Should be specified in terms of one of the [Surface]
/// rotation constants that represents the counter-clockwise degrees of
/// rotation relative to [DeviceOrientation.portraitUp].
final int? initialTargetRotation;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I support replacing with the native method, but am also fine with it being a follow up!

Copy link
Member

@gmackall gmackall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@camsim99 camsim99 added autosubmit Merge PR when tree becomes green via auto submit App and removed autosubmit Merge PR when tree becomes green via auto submit App labels Jan 2, 2024
@camsim99 camsim99 added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 2, 2024
@auto-submit auto-submit bot merged commit bbb4134 into flutter:main Jan 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 3, 2024
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Jan 3, 2024
flutter/packages@fa74d3e...bbb4134

2024-01-02 [email protected] [camerax] Implement `lockCaptureOrientation` & `unlockCaptureOrientation` (flutter/packages#5285)
2024-01-02 49699333+dependabot[bot]@users.noreply.github.com Bump github/codeql-action from 3.22.11 to 3.22.12 (flutter/packages#5738)
2024-01-02 [email protected] Add Flutter CI Status badge to README (flutter/packages#5733)
2023-12-26 49699333+dependabot[bot]@users.noreply.github.com [path_provider]: Bump androidx.annotation:annotation from 1.7.0 to 1.7.1 in /packages/path_provider/path_provider_android/android (flutter/packages#5709)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
@reidbaker reidbaker mentioned this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosubmit Merge PR when tree becomes green via auto submit App p: camera platform-android
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[camerax] Implement locking/unlocking capture orientation
2 participants