Anthony Le Gourriérec

Anthony Le Gourriérec

he/him

PhD Student in XR & Urban Lighting Perception · Digital Imaging Engineer · Passionate about immersive tech, light, and visual storytelling through 3D, XR, and photography.

Training a LoRA on FLUX for Personal Image Generation

Published:

Overview

FLUX is a new open-source model similar to Stable Diffusion, and I was eager to explore its capabilities. Given my computer’s limitations for model training, I decided to use Civit AI to train a LoRA model using 25 captioned photos of myself. This project aimed to evaluate the effectiveness of FLUX in generating both realistic and stylized images of my face by employing two distinct checkpoints with varying levels of training.

Key Skills Demonstrated:

  • LoRA Model Training: Trained a custom LoRA using Civit AI, experimenting with different checkpoints for a tailored results.
  • Model Fine-Tuning: Selected two different checkpoints, one for realism and one for stylization, balancing likeness with flexibility.
  • ComfyUI Usage: Set up and ran the FLUX dev model with the LoRA locally on my computer using ComfyUI.

Technologies Used: FLUX, ComfyUI, Civit AI

Project Highlights

Key Features

  • Realistic Likeness with FLUX: The first LoRA checkpoint was optimized for realistic image generation, focusing on closely replicating my facial features. This was achieved through slightly overfitting the model to prioritize likeness.
  • Stylized Generation: The second LoRA checkpoint was trained with more flexibility, allowing for creative, stylized outputs that still captured the main features of my face but offered more artistic diversity.
  • Efficient Workflow with Limited Resources: Despite not having a powerful computer for training, I effectively utilized Civit AI’s resources to handle the intensive LoRA training, and then ran the results locally using ComfyUI.

Achievements

  • Effective Overfitting: Successfully created a highly realistic version of the LoRA that captured fine details of my face, allowing for accurate representation in image generations.
  • Creative Flexibility: Developed a more flexible LoRA checkpoint, which enabled artistic and stylized variations of my face while still retaining key identifying features.
  • Resourceful Setup: Overcame hardware limitations by using external resources for training while running the final model locally.

Results

The project demonstrated the capabilities of FLUX in both realistic and stylized image generation. By experimenting with overfitting and underfitting, I was able to strike a balance between realism and creativity, with the following outcomes:

  • Increased Likeness: The overfitted model generated highly realistic images, particularly challenging when creating self-images.
  • Enhanced Creativity: The stylized checkpoint provided a broader range of artistic interpretations while still retaining the core features of my face.
Generated Image 11 Generated Image 2 Generated Image 3 Generated Image 4 Generated Image 5 Generated Image 6 Generated Image 7 Generated Image 8 Generated Image 12 Generated Image 9 Generated Image 10 Generated Image 13 Generated Image 14 Generated Image 15 Generated Image 16 Generated Image 17 Generated Image 18 Generated Image 19 Generated Image 20 Generated Image 21 Generated Image 22 Generated Image 23 Generated Image 24 Generated Image 25 Generated Image 26

Closing Thoughts

This project highlights the potential of FLUX as a powerful open-source model for personal image generation. By training a LoRA with 25 captioned photos of myself, I was able to achieve both realistic and creative outputs. The process also demonstrated the effectiveness of leveraging external training resources like Civit AI while managing model execution locally using ComfyUI. This project not only showcased technical skills but also the ability to overcome hardware limitations through resourceful problem-solving.

Interested in learning more about this project or collaborating? Contact me or explore more of my projects.