Skip to content

Adds Img2Img2 Variant for HunyuanDiT. #9041

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

MnCSSJ4x
Copy link
Contributor

@MnCSSJ4x MnCSSJ4x commented Aug 1, 2024

What does this PR do?

Adds Img2Img variant for HunyuanaDiT as per discussion in #8992.

Before submitting

How to reproduce:

import torch
from diffusers import HunyuanDiTImg2ImgPipeline
from diffusers.utils import load_image

pipe = HunyuanDiTImg2ImgPipeline.from_pretrained(
    "Tencent-Hunyuan/HunyuanDiT-Diffusers", torch_dtype=torch.float16
)
pipe.to("cuda")

url = (
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kolors/bunny_source.png"
)
init_image = load_image(url)

# You may also use English prompt as HunyuanDiT supports both English and Chinese
# prompt = "An astronaut riding a horse"
prompt = "一个宇航员在骑马"

image = pipe(prompt, image=init_image).images[0]

Output:
image

A colab notebook demonstrating all results can be found here.

Who can review?

@a-r-r-o-w @DN6

@MnCSSJ4x MnCSSJ4x marked this pull request as ready for review August 3, 2024 13:14
@DN6
Copy link
Collaborator

DN6 commented Aug 16, 2024

cc: @yiyixuxu

Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks really good!
thanks for the PR!

can we add a test (similar to https://github.com/huggingface/diffusers/blob/main/tests/pipelines/hunyuan_dit/test_hunyuan_dit.py) and doc too?

)

# 1. Check inputs. Raise error if not correct
self.check_inputs(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we check strength too?

Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Sep 14, 2024
@yiyixuxu yiyixuxu added close-to-merge contributions-welcome and removed stale Issues that haven't received updates labels Dec 3, 2024
@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Dec 3, 2024

@MnCSSJ4x
The PR is pretty close to merge, I added a "contribution welcome" label so someone in the community can pick it up. Let me know if it is ok. If you have time and would like to work on this, we will remove the label :)

The work left:

  1. remove the changes in the example folder, as the community pipeline has already added
  2. add a test
  3. provide a code example and show us some outputs:)

@MnCSSJ4x
Copy link
Contributor Author

MnCSSJ4x commented Dec 3, 2024

@yiyixuxu
Hey. I think there is an example notebook link already mentioned in the PR. I would need to update some commands there once the PR is merged. Sorry for not wrapping this up as my semester picked up. Regarding test(s) sure I can try to wrap everything up by end of this week. You can keep the tag up just in case I am unable to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants