Skip to content

Add code to do color compositing on inpainting #598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Feb 24, 2025

Conversation

willhsmit
Copy link
Contributor

This is intended as a fix for #597, which might take some further iteration.

Adds a UI parameter under 'Init Image' called 'Color Correction Behavior' with values 'None', 'Uniform', and 'Linear'.

  • None is the default and is intended as a noop from current
  • Uniform implements the behavior from https://discord.com/channels/1243166023859961988/1243185862234210389/1327390619068530729: find the mean difference between source and dest for each of HSV and apply it to source.
  • Linear attempts to model the dest as a linear function of source and apply the function to source. Experimentally, I found that Hue is rarely improved and sometimes made a lot worse by correction, and decided to leave Hue uncorrected. Typically, the derived function will increase the saturation and contrast of the image somewhat.

You mentioned putting it behind a parameter; I assume you meant a UI parameter and not a server/user parameter that hides the UI parameter - I didn't see a lot of prior art for that but am open to fixing it.

I left the Uniform method as an option since I think the Linear option is not always better than it and people may want to experiment. I mostly left the code for Uniform as it was in the discord, but am open to:

  • Trying the same removal of Hue as Linear
  • Removing the .remainder() from the clamp at the end. I took this out of Linear because it sometimes produces black spots in the middle of a bright white. This happens occasionally but less often in Uniform; I wasn't sure if the remainder() was in there for an important reason.

@mcmonkey4eva
Copy link
Member

The parameter should be... somewhere else. Honestly not sure where, maybe shoved under Advanced Sampling?

Because it's needed for <segment: usage especially, not just init image.
That or both an init image one, and a Regional Prompting one.

willhsmit and others added 3 commits February 19, 2025 12:59
…ed sampling so it can be used for segment, per suggestions.

(2) Switch toggleable to IgnoreIf: "None" per suggestions.
(3) Remove output remainder and Hue adjustment from the Uniform sampler, to match Linear. I think these create more artifacts than they solve.
@willhsmit
Copy link
Contributor Author

The parameter should be... somewhere else. Honestly not sure where, maybe shoved under Advanced Sampling?

Because it's needed for <segment: usage especially, not just init image. That or both an init image one, and a Regional Prompting one.

I can't think of a better spot for the parameter than Advanced Sampling; I put it under some VAE-related parameters.

I tried doing some segment tests, the seam and the correction effect are pretty subtle. I'll add a couple to the bug thread.

--

I ended up removing remainder() and also Hue from the 'Uniform' algorithm; there are some pretty clear examples from testing where those made things worse.

@mcmonkey4eva mcmonkey4eva merged commit 4e2f4ac into mcmonkeyprojects:master Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants