UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalization
💬 Ask
UniPortrait is a new human image personalization framework that can create images of single or multiple people, preserving their identity and allowing for diverse modifications. The work was conducted by researchers at Alibaba Group and was recently published on arXiv.
UniPortrait leverages two key modules: an ID embedding module and an ID routing module. The ID embedding module extracts versatile high-fidelity facial features for each person, and the ID routing module combines and distributes these embeddings to their respective regions within the synthesized image, achieving the customization of single and multiple people. The authors emphasize that UniPortrait provides several advantages over existing methods:
- High face fidelity: UniPortrait can maintain the identity of a person while enabling extensive facial editability.
- Free-form input description: Users can describe their desired images using free-form text prompts, giving them flexibility in how they envision their creations.
- Diverse layout generation: UniPortrait does not require a predetermined layout, allowing for diverse and unique image compositions.
- Unified single- and multi-ID customization: UniPortrait can handle both single- and multi-person images.
For example, a user could provide a text prompt such as “a woman smiling with her hair down and wearing a necklace” alongside a reference image of a person. UniPortrait would then generate an image of that person, smiling, with her hair down, and wearing a necklace. The user could even provide multiple reference images and text prompts to create a multi-person scene. In this case, UniPortrait would automatically assign each person to their respective location in the scene, resulting in a naturally arranged image that captures the essence of the text prompt.
The authors demonstrate the effectiveness of their method through extensive experiments, showing that UniPortrait outperforms existing approaches in terms of identity preservation, prompt consistency, and the quality and diversity of generated images. Furthermore, they demonstrate that UniPortrait is compatible with existing generative control tools, such as ControlNet, which allows for further control over the generated images, such as pose, style, and lighting.
UniPortrait represents a significant advancement in human image personalization, enabling more creative and realistic image generation. The authors believe that their method could have a wide range of applications, including AI portrait photos, image animation, virtual try-ons, and even creative storytelling.
Chat about this paper
To chat about this paper, you'll need a free Gemini API key from Google AI Studio.
Your API key will be stored securely in your browser's local storage.