MyStyle: A Personalized Generative Prior
Supplementary Material
 |
 |
 |
Random walks in our personalized latent sub-space of generators tuned for Emilia Clarke, Joe Biden, and Oprah Winfrey. |
- Synthesis: our results
- Inpainting: our results
- Super-resolution: our results
- Pose & smile editing: our results
- Synthesis: comparisons
- Inpainting: comparisons
- Super-resolution: comparisons
- Pose & smile editing: comparisons
Synthesis: our results
Images sampled from our personalized generator for each subject. Note how our method is able to robustly produce high-quality images that preserve the subject's identity.
Adele
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Angela Merkel
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Barack Obama
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Dwayne Johnson
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Emilia Clarke
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Jeff Bezos
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Joe Biden
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Kamala Harris
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Lady Gaga
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Michelle Obama
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Oprah Winfrey
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Xi Jinping
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Inpainting: our results
Results of applying the personalized prior to the task of image inpainting. Although key parts of the face are occluded, our method is able to recover the individual's true likeliness.
Barack Obama
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Dwayne Johnson
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Joe Biden
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Lady Gaga
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Xi Jinping
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Super-resolution: our results
Results of applying the personalized prior to the task of image super-resolution. Given an extremely low-resolution image (32 x 32) of a known individual, our method is able to generate a high-resolution image (1024 x 1024) that best fits the input while preserving their personal identity.
Note that our method is not designed to recover the non-face areas. Hence, as a post-processing step, the raw model output is segmented into face and non-face regions, and the non-face region is replaced by a Lanczos-upsampled version of the input. That's why the background may look "blocky" in some results. See the paper for more details.
Adele
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
.jpg) |
.jpg) |
 |
 |
.jpg) |
.jpg) |
Angela Merkel
| Input | Our result | Input | Our result |
|---|---|---|---|
.jpg) |
.jpg) |
 |
 |
 |
 |
 |
 |
Barack Obama
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Dwayne Johnson
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
.jpg) |
.jpg) |
 |
 |
Emilia Clarke
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
.jpg) |
.jpg) |
 |
 |
Jeff Bezos
| Input | Our result | Input | Our result |
|---|---|---|---|
.jpg) |
.jpg) |
 |
 |
 |
 |
Joe Biden
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
.jpg) |
.jpg) |
.jpg) |
.jpg) |
 |
 |
Kamala Harris
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Lady Gaga
| Input | Our result | Input | Our result |
|---|---|---|---|
.jpg) |
.jpg) |
 |
 |
.jpg) |
.jpg) |
Michelle Obama
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
_(cropped).jpg) |
_(cropped).jpg) |
 |
 |
.jpg) |
.jpg) |
Oprah Winfrey
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Taylor Swift
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Xi Jinping
| Input | Our result | Input | Our result |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
.jpg) |
.jpg) |
Pose & smile editing: our results
Our personalized prior can also be applied to the semantic editing of portrait images. Here, we show two examples: editing the subject's pose (rotating left/right), and their smile (less/more smile). As shown below, each individual's identity is perserved throughout the editing range.
Adele
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
 |
 |
|
 |
 |
 |
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
Angela Merkel
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
 |
 |
 |
 |
 |
 |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
 |
 |
|
 |
 |
 |
Barack Obama
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
 |
 |
|
 |
 |
 |
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
Dwayne Johnson
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
 |
 |
|
 |
 |
 |
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
Jeff Bezos
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
 |
 |
 |
 |
 |
 |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
 |
 |
|
 |
 |
 |
Joe Biden
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
_(cropped)_by_rotation/4_-5d556.jpg) |
_(cropped)_by_rotation/6_-3d333.jpg) |
_(cropped).jpg) |
_(cropped)_by_rotation/12_3d333.jpg) |
_(cropped)_by_rotation/14_5d556.jpg) |
_(cropped)_by_rotation/animation.gif) |
 |
 |
 |
 |
 |
 |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
_(cropped)_by_smile/4_-5d556.jpg) |
_(cropped)_by_smile/6_-3d333.jpg) |
|
_(cropped)_by_smile/12_3d333.jpg) |
_(cropped)_by_smile/14_5d556.jpg) |
_(cropped)_by_smile/animation.gif) |
 |
 |
|
 |
 |
 |
Kamala Harris
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
 |
 |
|
 |
 |
 |
 |
 |
|
 |
 |
 |
Lady Gaga
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
-3_(cropped)_by_rotation/4_-5d556.jpg) |
-3_(cropped)_by_rotation/6_-3d333.jpg) |
-3_(cropped).jpg) |
-3_(cropped)_by_rotation/12_3d333.jpg) |
-3_(cropped)_by_rotation/14_5d556.jpg) |
-3_(cropped)_by_rotation/animation.gif) |
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
-3_(cropped)_by_smile/4_-5d556.jpg) |
-3_(cropped)_by_smile/6_-3d333.jpg) |
|
-3_(cropped)_by_smile/12_3d333.jpg) |
-3_(cropped)_by_smile/14_5d556.jpg) |
-3_(cropped)_by_smile/animation.gif) |
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
Michelle Obama
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
_(cropped)_by_rotation/4_-5d556.jpg) |
_(cropped)_by_rotation/6_-3d333.jpg) |
_(cropped).jpg) |
_(cropped)_by_rotation/12_3d333.jpg) |
_(cropped)_by_rotation/14_5d556.jpg) |
_(cropped)_by_rotation/animation.gif) |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
 |
 |
|
 |
 |
 |
_(cropped)_by_smile/4_-5d556.jpg) |
_(cropped)_by_smile/6_-3d333.jpg) |
|
_(cropped)_by_smile/12_3d333.jpg) |
_(cropped)_by_smile/14_5d556.jpg) |
_(cropped)_by_smile/animation.gif) |
Taylor Swift
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
_by_rotation/4_-5d556.jpg) |
_by_rotation/6_-3d333.jpg) |
.jpg) |
_by_rotation/12_3d333.jpg) |
_by_rotation/14_5d556.jpg) |
_by_rotation/animation.gif) |
 |
 |
 |
 |
 |
 |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
_by_smile/4_-5d556.jpg) |
_by_smile/6_-3d333.jpg) |
|
_by_smile/12_3d333.jpg) |
_by_smile/14_5d556.jpg) |
_by_smile/animation.gif) |
 |
 |
|
 |
 |
 |
Xi Jinping
Editing pose
| Rotating left | ← | Original input | → | Rotating right | GIF animation |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Editing smile
| Less smile | ← | Original input | → | More smile | GIF animation |
|---|---|---|---|---|---|
 |
 |
|
 |
 |
 |
 |
 |
|
 |
 |
 |
Synthesis: comparisons
We compare our synthesis results with state-of-the-art methods (see paper for details). Our method consistently produces results that are more natural-looking and identity-preserving.
Adele
DiffAugment
 |
 |
 |
 |
 |
 |
Ojha et al.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Ours
 |
|
|
|
|
 |
 |
|
|
|
|
|
Joe Biden
DiffAugment
 |
 |
 |
 |
 |
 |
Ojha et al.
 |
 |
 |
 |
 |
 |
Ours
|
|
|
|
|
 |
|
 |
|
|
|
|
Kamala Harris
DiffAugment
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Ojha et al.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Ours
|
|
|
|
 |
 |
|
|
|
|
|
|
Inpainting: comparisons
While other generative inpainting methods also produce high image quality, they don't preserve the individual's identity as well as ours, even after fine-tuning on this individual's dataset.
Barack Obama
| Input | co-mod-GAN | co-mod-GAN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
 |
 |
 |
 |
 |
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
 |
 |
 |
 |
 |
Lady Gaga
| Input | co-mod-GAN | co-mod-GAN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
Xi Jinping
| Input | co-mod-GAN | co-mod-GAN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
|
 |
 |
 |
|
 |
 |
 |
 |
 |
|
 |
 |
 |
|
|
 |
 |
 |
|
Super-resolution: comparisons
This section compares our personalized prior against state-of-the-art methods on the application of image super-resolution. The input image is 16 x 16 pixels. The output image is 512 x 512 for GPEN (16x magnification), and 1024 x 1024 for DiffAugment and our method (32x magnification). Since GPEN is a generic model without personalization, we also tried to fine-tune the model using the same personal dataset for each individual until the LPIPS loss converges. However, our method still outperforms the rest in terms of image quality (lack of artifacts) and identity preservation.
Note that none of the methods is designed to recover the non-face areas. We applied the same post-processing to our results (see above), but not to the other methods, since the face detector and segmenter don't always work reliably on their outputs due to low quality.
Emilia Clarke
| Input | GPEN | GPEN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
.jpg) |
.jpg) |
.jpg) |
|
|
 |
 |
 |
|
Kamala Harris
| Input | GPEN | GPEN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
 |
 |
 |
 |
 |
Michelle Obama
| Input | GPEN | GPEN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
|
 |
 |
 |
|
|
_(cropped).jpg) |
_(cropped).jpg) |
_(cropped).jpg) |
|
|
 |
 |
 |
|
 |
 |
 |
 |
 |
|
.jpg) |
.jpg) |
.jpg) |
|
 |
 |
 |
 |
 |
Xi Jinping
| Input | GPEN | GPEN, fine-tuned | DiffAugment | Ours |
|---|---|---|---|---|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
 |
 |
 |
|
|
.jpg) |
.jpg) |
.jpg) |
|
.jpg) |
.jpg) |
.jpg) |
.jpg) |
.jpg) |
 |
 |
 |
 |
 |
Pose & smile editing: comparisons
Comparing with other semantic editing methods, the personalized prior significantly improves identity preservation, especially when the "edit distance" is large. Below, we show one static example and one GIF animation for each editing method.
Barack Obama
Editing pose
| Input | PTI | PTI, DiffAugment | Ours |
|---|---|---|---|
|
 |
 |
|
 |
 |
|
|
|
_by_rotation/34_4d167.jpg) |
_by_rotation/19_3d571.jpg) |
|
_by_rotation/animation.gif) |
_by_rotation/animation.gif) |
|
Editing smile
| Input | PTI | PTI, DiffAugment | Ours |
|---|---|---|---|
|
 |
 |
|
 |
 |
|
|
|
_by_smile/29_2d083.jpg) |
_by_smile/19_3d571.jpg) |
|
_by_smile/animation.gif) |
_by_smile/animation.gif) |
|
Joe Biden
Editing pose
| Input | PTI | PTI, DiffAugment | Ours |
|---|---|---|---|
|
_(cropped)_by_rotation/34_4d167.jpg) |
_(cropped)_by_rotation/19_3d571.jpg) |
|
_(cropped)_by_rotation/animation.gif) |
_(cropped)_by_rotation/animation.gif) |
|
|
|
 |
 |
|
 |
 |
|
Editing smile
| Input | PTI | PTI, DiffAugment | Ours |
|---|---|---|---|
|
_(cropped)_by_smile/29_2d083.jpg) |
_(cropped)_by_smile/19_3d571.jpg) |
|
_(cropped)_by_smile/animation.gif) |
_(cropped)_by_smile/animation.gif) |
|
|
|
 |
 |
|
 |
 |
|
Michelle Obama
Editing pose
| Input | PTI | PTI, DiffAugment | Ours |
|---|---|---|---|
|
 |
 |
|
 |
 |
|
|
|
_(cropped)_by_rotation/18_5d000.jpg) |
_(cropped)_by_rotation/19_3d571.jpg) |
|
_(cropped)_by_rotation/animation.gif) |
_(cropped)_by_rotation/animation.gif) |
|
Editing smile
| Input | PTI | PTI, DiffAugment | Ours |
|---|---|---|---|
|
 |
 |
|
 |
 |
|
|
|
_(cropped)_by_smile/14_2d778.jpg) |
_(cropped)_by_smile/19_3d571.jpg) |
|
_(cropped)_by_smile/animation.gif) |
_(cropped)_by_smile/animation.gif) |
|