yuval-alaluf / sam

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

  • Public
  • 1M runs
  • GitHub
  • Paper
  • License

Run time and cost

This model costs approximately $0.0042 to run on Replicate, or 238 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 19 seconds. The predict time for this model varies significantly based on the inputs.

Readme

The task of age transformation illustrates the change of an individual’s appearance over time. Accurately modeling this complex transformation over an input facial image is extremely challenging as it requires making convincing and possibly large changes to facial features and head shape, while still preserving the input identity. In this work, we present an image-to-image translation method that learns to directly encode real facial images into the latent space of a pre-trained unconditional GAN (e.g., StyleGAN) subject to a given aging shift. We employ a pre-trained age regression network used to explicitly guide the encoder to generate the latent codes corresponding to the desired age. In this formulation, our method approaches the continuous aging process as a regression task between the input age and desired target age, providing fine-grained control on the generated image. Moreover, unlike other approaches that operate solely in the latent space using a prior on the path controlling age, our method learns a more disentangled, non-linear path. We demonstrate that the end-to-end nature of our approach, coupled with the rich semantic latent space of StyleGAN, allows for further editing of the generated images. Qualitative and quantitative evaluations show the advantages of our method compared to state-of-the-art approaches.

Credits

StyleGAN2 model and implementation:
https://github.com/rosinality/stylegan2-pytorch
Copyright (c) 2019 Kim Seonghyeon
License (MIT) https://github.com/rosinality/stylegan2-pytorch/blob/master/LICENSE

IR-SE50 model and implementations:
https://github.com/TreB1eN/InsightFace_Pytorch
Copyright (c) 2018 TreB1eN
License (MIT) https://github.com/TreB1eN/InsightFace_Pytorch/blob/master/LICENSE

Ranger optimizer implementation:
https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer
License (Apache License 2.0) https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer/blob/master/LICENSE

LPIPS model and implementation:
https://github.com/S-aiueo32/lpips-pytorch
Copyright (c) 2020, Sou Uchida
License (BSD 2-Clause) https://github.com/S-aiueo32/lpips-pytorch/blob/master/LICENSE

DEX VGG model and implementation:
https://github.com/InterDigitalInc/HRFAE
Copyright (c) 2020, InterDigital R&D France
https://github.com/InterDigitalInc/HRFAE/blob/master/LICENSE.txt

pSp model and implementation:
https://github.com/eladrich/pixel2style2pixel
Copyright (c) 2020 Elad Richardson, Yuval Alaluf
https://github.com/eladrich/pixel2style2pixel/blob/master/LICENSE

Acknowledgments

This code borrows heavily from pixel2style2pixel

Citation

If you use this code for your research, please cite our paper Only a Matter of Style: Age Transformation Using a Style-Based Regression Model:

@article{alaluf2021matter,
    author = {Alaluf, Yuval and Patashnik, Or and Cohen-Or, Daniel},
    title = {Only a Matter of Style: Age Transformation Using a Style-Based Regression Model},
    journal = {ACM Trans. Graph.},
    issue_date = {August 2021},
    volume = {40},
    number = {4},
    year = {2021},
    articleno = {45},
    publisher = {Association for Computing Machinery},
    url = {https://doi.org/10.1145/3450626.3459805}
}