---
title: qwen-image-2512-lora-advertisement
canonical_url: "https://www.modelscope.ai/models/rayyan9477/qwen-image-2512-lora-advertisement"
md_url: "https://www.modelscope.ai/models/rayyan9477/qwen-image-2512-lora-advertisement.md"
repository: rayyan9477/qwen-image-2512-lora-advertisement
last_updated: 2026-03-17
license: apache-2.0
pipeline_tag: text-to-image-synthesis
tasks:
  - text-to-image-synthesis
base_model:
  - Qwen/Qwen-Image-2512
base_model_relation: adapter
parameters: 428.0M
tensor_type:
  - F32
library_name:
  - pytorch
  - lora
  - peft
  - safetensors
frameworks:
  - pytorch
downloads: 7
stars: 1
tags:
  - lora
  - dora
  - rslora
  - text-to-image
  - advertisement
  - product-photography
  - qwen-image
  - diffusion
  - flow-matching
  - commercial
  - marketing
  - pexels
  - high-resolution
  - peft
---

# qwen-image-2512-lora-advertisement

> qwen-image-2512-lora-advertisement - An open-source model by rayyan9477 on ModelScope. Qwen-Image-2512 LoRA for Professional Advertisement Generation

rayyan9477/qwen-image-2512-lora-advertisement is a 428.0M-parameter text-to-image-synthesis model on ModelScope. licensed under apache-2.0. derived from Qwen/Qwen-Image-2512.

- **Repository**: rayyan9477/qwen-image-2512-lora-advertisement
- **License**: apache-2.0
- **Tasks**: text-to-image-synthesis
- **Parameters**: 428.0M
- **Base model**: Qwen/Qwen-Image-2512
- **Tags**: lora, dora, rslora, text-to-image, advertisement, product-photography, qwen-image, diffusion, flow-matching, commercial, marketing, pexels, high-resolution, peft
- **Downloads**: 7
- **Stars**: 1
- **Last updated**: 2026-03-17

Source: https://www.modelscope.ai/models/rayyan9477/qwen-image-2512-lora-advertisement

---

<div align="center">

# Qwen-Image-2512 LoRA for Professional Advertisement Generation

### 80 Hours. One RTX 3090. Zero Cloud Compute.

<br>

[![Method](https://img.shields.io/badge/Method-DoRA_+_rsLoRA-FF6B6B?style=for-the-badge&logo=pytorch)](https://arxiv.org/abs/2402.09353)
[![Base](https://img.shields.io/badge/Base-Qwen--Image--2512_(20.5B)-4ECDC4?style=for-the-badge)](https://modelscope.ai/models/Qwen/Qwen-Image-2512)
[![GPU](https://img.shields.io/badge/Trained_On-RTX_3090_(24GB_VRAM)-76B900?style=for-the-badge&logo=nvidia)](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090/)
[![Dataset](https://img.shields.io/badge/Dataset-12,917_Pexels_Images-F59E0B?style=for-the-badge)](https://huggingface.co/datasets/RayyanAhmed9477/pexels-advertisement-photography)
[![License](https://img.shields.io/badge/License-Apache_2.0-green?style=for-the-badge)](https://www.apache.org/licenses/LICENSE-2.0)
[![Competition](https://img.shields.io/badge/Competition-Qwen_Image_LoRA_2026-8B5CF6?style=for-the-badge)](https://modelscope.ai/active/qwenimagelora)

<br>

| Metric | Value |
|:------:|:-----:|
| **Training Time** | **80 hours** on a single consumer GPU |
| **Base Model** | Qwen-Image-2512 (20.5B parameters) |
| **Dataset** | 12,917 curated images + 25,834 VLM-enriched captions |
| **GPU Hardware** | NVIDIA RTX 3090 &mdash; 24 GB VRAM, consumer-grade ($1,500) |
| **Adapter Size** | 1.7 GB (428M trainable params, 4.02% of transformer) |
| **Resolution** | 2-stage curriculum: 512px &rarr; 768px |
| **Techniques** | DoRA, rsLoRA, Min-SNR-gamma, 4-bit QLoRA, embedding pre-caching |

<br>

**Rayyan Ahmed** &mdash; [GitHub](https://github.com/Rayyan9477) | [LinkedIn](https://www.linkedin.com/in/rayyan-ahmed9477/) | [ModelScope](https://modelscope.ai/profile/rayyan9477) | [HuggingFace](https://huggingface.co/RayyanAhmed9477)

Part of the [MarketMind](https://github.com/Rayyan9477/MarketMind) AI Marketing Platform

</div>

---

> **Competition Entry:** [2026 Qwen-Image LoRA Training Competition](https://modelscope.ai/active/qwenimagelora) &mdash; **Track 1: AI for Production** (branding, e-commerce, industrial design). Hosted by Tongyi Lab + ModelScope.

---

## TL;DR

A **1.7 GB LoRA adapter** that transforms the 20.5 billion parameter Qwen-Image-2512 into a professional advertisement and product photography engine. Trained on **12,917 curated Pexels images** across 40 advertising categories using a two-stage resolution curriculum (512px &rarr; 768px) on a **single NVIDIA RTX 3090** &mdash; a consumer GPU with just 24 GB of VRAM. No cloud compute, no A100, no H100. Just engineering.

The entire pipeline &mdash; dataset curation, VLM caption enrichment, multi-stage training, evaluation, and deployment &mdash; was designed from scratch to operate within the hard constraints of consumer hardware, accumulating **80+ hours of continuous GPU training**.

---

## Showcase Gallery

All images generated at **768px** with the LoRA adapter using True CFG guidance (scale=4.0, 100 inference steps).

### Product Photography Across Industries

<table>
<tr>
<td align="center"><img src="showcase/v2_seed_2026_luxury_perfume.png" width="256"><br><sub><b>Luxury Fragrance</b><br><i>Crystal perfume bottle, dramatic Rembrandt lighting</i></sub></td>
<td align="center"><img src="showcase/v2_seed_2026_tech_smartphone.png" width="256"><br><sub><b>Technology</b><br><i>Flagship smartphone, clean studio lighting</i></sub></td>
<td align="center"><img src="showcase/v2_seed_42_food_coffee.png" width="256"><br><sub><b>Food & Beverage</b><br><i>Artisan cappuccino, warm morning tones</i></sub></td>
<td align="center"><img src="showcase/v2_seed_42_automotive_suv.png" width="256"><br><sub><b>Automotive</b><br><i>Luxury SUV, golden hour mountain road</i></sub></td>
</tr>
<tr>
<td align="center"><img src="showcase/v2_seed_42_fashion_watch.png" width="256"><br><sub><b>Fashion & Jewelry</b><br><i>Luxury timepiece, moody dramatic lighting</i></sub></td>
<td align="center"><img src="showcase/v2_seed_2026_fitness_shoes.png" width="256"><br><sub><b>Sports & Fitness</b><br><i>Performance running shoe, dynamic floating shot</i></sub></td>
<td align="center"><img src="showcase/v2_seed_42_skincare_serum.png" width="256"><br><sub><b>Skincare & Beauty</b><br><i>Glass serum bottle, marble surface, natural light</i></sub></td>
<td align="center"><img src="showcase/v2_seed_777_realestate_interior.png" width="256"><br><sub><b>Real Estate</b><br><i>Penthouse interior, city skyline view</i></sub></td>
</tr>
</table>

### Seed Diversity &mdash; Same Prompt, Three Different Seeds

Demonstrating the model's ability to generate diverse, high-quality variations from the same prompt.

<table>
<tr>
<th></th>
<th align="center">Seed 42</th>
<th align="center">Seed 2026</th>
<th align="center">Seed 777</th>
</tr>
<tr>
<td><b>Luxury Perfume</b></td>
<td align="center"><img src="showcase/v2_seed_42_luxury_perfume.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_2026_luxury_perfume.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_777_luxury_perfume.png" width="200"></td>
</tr>
<tr>
<td><b>Technology</b></td>
<td align="center"><img src="showcase/v2_seed_42_tech_smartphone.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_2026_tech_smartphone.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_777_tech_smartphone.png" width="200"></td>
</tr>
<tr>
<td><b>Fitness</b></td>
<td align="center"><img src="showcase/v2_seed_42_fitness_shoes.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_2026_fitness_shoes.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_777_fitness_shoes.png" width="200"></td>
</tr>
<tr>
<td><b>Real Estate</b></td>
<td align="center"><img src="showcase/v2_seed_42_realestate_interior.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_2026_realestate_interior.png" width="200"></td>
<td align="center"><img src="showcase/v2_seed_777_realestate_interior.png" width="200"></td>
</tr>
</table>

### Ad Overlays &mdash; Production-Ready Output

<table>
<tr>
<td align="center"><img src="showcase/tech_smartphone_ad.png" width="200"><br><sub>Tech Ad</sub></td>
<td align="center"><img src="showcase/luxury_perfume_ad.png" width="200"><br><sub>Luxury Ad</sub></td>
<td align="center"><img src="showcase/food_coffee_ad.png" width="200"><br><sub>Food Ad</sub></td>
<td align="center"><img src="showcase/automotive_suv_ad.png" width="200"><br><sub>Auto Ad</sub></td>
</tr>
</table>

### LoRA vs Base Model Comparison

<p align="center">
<img src="showcase/comparison_grid.png" width="100%">
</p>

<details>
<summary><b>Generation Parameters</b></summary>

- **Resolution:** 768&times;768 (showcase), 100 inference steps
- **True CFG Scale:** 4.0 with negative prompt
- **Seeds:** 42, 2026, 777 (3 seeds &times; 8 prompts = 24 LoRA showcase images)
- **Precision:** Full bf16 with pre-merged DoRA weights
- **Hardware:** Single RTX 3090 (24 GB VRAM) + 64 GB RAM
- **LoRA Scale:** 0.65

</details>

---

## The Engineering Challenge

### Fitting 20.5 Billion Parameters on a $1,500 Consumer GPU

Qwen-Image-2512 has a **20.5 billion parameter** S3-DiT transformer. Loading it in bf16 requires **41 GB of VRAM**. My RTX 3090 has 24 GB.

This isn't a "just reduce batch size" problem. Every architectural decision &mdash; from training strategy to inference pipeline &mdash; was dictated by this hard constraint. Here's how I solved it:

| Challenge | Solution | Impact |
|-----------|----------|--------|
| 41 GB model vs 24 GB VRAM | 4-bit NF4 quantization during training | Fits in ~10 GB VRAM |
| Text encoder takes 6 GB | Pre-cache all 25,834 embeddings to disk, then unload encoder | Freed 6 GB for larger LoRA rank |
| Need bf16 for DoRA inference | Custom shard-by-shard LoRA merge + disk offload pipeline | Full quality on consumer hardware |
| 80 hours of training | Graceful SIGTERM/SIGINT checkpointing + resume system | Survived power events & crashes |
| Windows pagefile limits | Custom Python mmap for safetensors + commit-aware device mapping | Zero-copy model loading |
| Mixed resolution dataset | Two-stage curriculum (512px &rarr; 768px) | 2.25&times; faster than 768px-only training |
| CPU RAM pressure during inference | Commit-aware device mapping via ctypes MEMORYSTATUSEX | Dynamic GPU/CPU/disk layer placement |

### 80 Hours of Continuous GPU Time

This was not a quick weekend project. The training ran for **80 continuous hours** on a single GPU:

```
Stage 1: 512px Resolution &mdash; Learning Composition
  Steps:     10,000 (~3.4 epochs over 12,917 images)
  Speed:     ~22.5 seconds/step
  Duration:  ~63 hours
  Final loss: 0.96

Stage 2: 768px Resolution &mdash; Sharpening Detail
  Steps:     1,500 (fine-tuning from step 10,000)
  Speed:     ~36 seconds/step
  Duration:  ~17 hours
  Final loss: 0.17

Total:       11,500 steps | ~80 hours | Single RTX 3090
```

For context: this same training would take ~20 hours on an A100 or ~12 hours on an H100. I didn't have access to either. **Every optimization mattered.**

### The VRAM Budget

```
Component                         VRAM Usage
─────────────────────────────────────────────
Transformer (4-bit NF4)           ~10.0 GB
LoRA adapters (720 layers)         ~1.5 GB
VAE (bf16)                         ~0.5 GB
Optimizer states (AdamW 8-bit)     ~3.0 GB
Activations (grad checkpoint)      ~4.0 GB
Overhead & buffers                 ~3.0 GB
─────────────────────────────────────────────
TOTAL                             ~22.0 GB / 24 GB available
Text encoder: UNLOADED (embeddings pre-cached to disk, saving 6 GB)
```

---

## 8 Technical Innovations

| # | Innovation | What & Why |
|---|-----------|------------|
| 1 | **DoRA + rsLoRA** | Weight-Decomposed LoRA closes the gap with full fine-tuning. rsLoRA stabilizes high-rank training at r=64 |
| 2 | **12 target modules per block** | Most LoRAs target 4 attention projections. This adapter targets all 12 linear layers (attention + MLP, both image and text streams) for deeper style adaptation |
| 3 | **Two-stage resolution curriculum** | 10K steps at 512px learns composition; 1.5K steps at 768px sharpens detail. 2.25&times; more efficient than full-resolution training |
| 4 | **Min-SNR-gamma=5.0** | Flow matching loss weighting that prevents wasting gradient updates on trivially noisy timesteps |
| 5 | **VLM dual-caption enrichment** | Qwen3-VL-8B generated 25,834 captions: detailed paragraphs + concise tags. Stochastic 70/30 mixing during training for robust text alignment |
| 6 | **Adaptive rank pattern** | r=64 for attention modules, r=32 for MLP modules. MLPs need less capacity &mdash; saving VRAM where it matters least |
| 7 | **Text embedding pre-caching** | Encode all captions once, persist to disk, unload the 6 GB text encoder. Train indefinitely on cached embeddings |
| 8 | **Custom 12,917-image dataset** | Built from scratch via Pexels API &mdash; filtered, deduplicated, VLM-enriched, and published as an [open dataset](https://huggingface.co/datasets/RayyanAhmed9477/pexels-advertisement-photography) |

---

## The DoRA Discovery

After 80 hours of training, I loaded the adapter at 4-bit quantization. The output was garbage. I tried 8-bit &mdash; still garbage. After two days of debugging, I tried full bf16 and the output was perfect.

**DoRA breaks under quantization.** Its magnitude-direction weight decomposition compounds rounding errors from both components, corrupting the learned adaptation. Standard LoRA tolerates quantization fine; DoRA does not.

**This is not documented anywhere** in PEFT, the DoRA paper, or the HuggingFace ecosystem. I'm documenting it here so others don't waste two days.

| Inference Method | Quality | Explanation |
|-----------------|---------|-------------|
| **bf16 + PeftModel** | Professional, sharp | Full DoRA precision preserved |
| 8-bit quantized | Garbled | Magnitude-direction decomposition corrupted |
| 4-bit quantized | Garbled | Same issue, worse |
| merge_and_unload (any quant) | Garbled | PEFT warns "rounding errors" |

---

## Evaluation Results

### Quantitative Metrics (n=20 per model, identical prompts & seeds)

| Model | Aesthetic Score | CLIP Score | Aesthetic Std |
|-------|:--------------:|:----------:|:-------------:|
| Base Qwen-Image-2512 (8-bit + True CFG) | 5.50 &plusmn; 1.56 | **0.1849** &plusmn; 0.061 | 1.56 |
| **+ LoRA Adapter (bf16, this model)** | **5.56 &plusmn; 1.06** | 0.1188 &plusmn; 0.038 | **1.06** |

**Key findings:**
- **Aesthetic score improved** (+1.1%) &mdash; the LoRA generates more visually appealing ad imagery
- **Consistency dramatically improved** &mdash; standard deviation dropped from 1.56 &rarr; 1.06 (**32% reduction**). The adapter produces *reliably good* output rather than high-variance results
- **CLIP score trade-off** &mdash; this is expected and intentional. The LoRA was trained to produce *advertising-style* imagery which prioritizes aesthetic impact and mood over literal text-to-image correspondence. A perfume ad doesn't literally depict "dramatic Rembrandt lighting" &mdash; it *creates* Rembrandt lighting
- **Lower CLIP variance** (0.038 vs 0.061) confirms more consistent style adherence across diverse advertising categories

> **A note on metrics for domain-specific models:** CLIP score measures general text-image similarity using ViT-L-14. For specialized models like advertisement generators, aesthetic quality and output consistency are more meaningful production metrics than raw text alignment. The showcase gallery above demonstrates actual output quality across 8 industries and 3 seeds.

---

## Dataset: Pexels Advertisement Photography

> **Open dataset:** [RayyanAhmed9477/pexels-advertisement-photography](https://huggingface.co/datasets/RayyanAhmed9477/pexels-advertisement-photography) (12,917 images, 25,834 captions)

Built entirely from scratch for this project:

```
Pexels API (40 targeted advertising search queries)
    │
    ├── Automated Scraping Pipeline
    │     12,917 high-resolution images collected
    │     40 categories: product photography, luxury goods, food styling,
    │     automotive, fashion, technology, real estate, cosmetics, sports,
    │     jewelry, furniture, beverages, electronics, watches, perfume...
    │
    ├── Quality Filtering
    │     Resolution ≥ 512px (shortest side)
    │     Aesthetic scoring + perceptual deduplication (imagehash)
    │     Avg resolution: 842×886 px (median: 768×1024)
    │
    └── VLM Caption Enrichment (Qwen3-VL-8B, ~6.5 hours)
          Dual captions per image:
            caption_long:  150-250 word paragraph (layout, style, lighting, composition)
            caption_short: Concise tags ("product photography, smartphone, studio lighting")
          → 25,834 total captions generated
```

**Why I built my own dataset** (I tried an existing one first):

| | ADImgNet (first attempt) | Pexels (this model) |
|---|:---:|:---:|
| **Images** | 3,201 (filtered from 9,003) | **12,917** |
| **Max resolution** | 300px | **1024px+** |
| **Avg resolution** | ~280px | **842&times;886px** |
| **Categories** | Academic labels | **40 real commercial ad categories** |
| **Captions** | Original only | **Dual VLM-enriched (25,834)** |
| **Publicly available** | No | **[Yes &mdash; open dataset](https://huggingface.co/datasets/RayyanAhmed9477/pexels-advertisement-photography)** |

---

## Model Architecture

```
Qwen-Image-2512 (20.5B params, S3-DiT, Flow Matching)
    │
    ├── Text Encoder: Qwen2.5-VL
    │     Processes prompt → hidden states [batch, seq_len, 3584]
    │     Pre-cached during training → unloaded to save 6 GB VRAM
    │
    ├── Transformer: 60 S3-DiT blocks (10.6B params)
    │     Each block → 12 LoRA-adapted linear layers:
    │
    │     Image Stream Attention [rank=64, DoRA]:
    │       to_q, to_k, to_v, to_out.0
    │
    │     Text Stream Attention [rank=64, DoRA]:
    │       add_q_proj, add_k_proj, add_v_proj, to_add_out
    │
    │     Image MLP [rank=32, DoRA]:
    │       img_mlp.net.0.proj, img_mlp.net.2
    │
    │     Text MLP [rank=32, DoRA]:
    │       txt_mlp.net.0.proj, txt_mlp.net.2
    │
    │     Total: 720 adapted layers | 428M trainable params (4.02%)
    │
    └── VAE: AutoencoderKLQwenImage
          Decodes latents → images
```

### LoRA Configuration

```yaml
peft_type: LORA
r: 64                      # Primary rank (attention modules)
lora_alpha: 128             # Scaling factor (effective scale = alpha/sqrt(r) via rsLoRA)
lora_dropout: 0.05
use_dora: true              # Weight-Decomposed LoRA (requires bf16 inference!)
use_rslora: true            # Rank-Stabilized scaling
rank_pattern:               # Adaptive rank per module type
  img_mlp: 32
  txt_mlp: 32
alpha_pattern:
  img_mlp: 64
  txt_mlp: 64
target_modules:             # 12 modules × 60 blocks = 720 adapted layers
  - to_q, to_k, to_v, to_out.0
  - add_q_proj, add_k_proj, add_v_proj, to_add_out
  - img_mlp.net.0.proj, img_mlp.net.2
  - txt_mlp.net.0.proj, txt_mlp.net.2
```

---

## Quick Start

### Installation

```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install diffusers transformers peft accelerate
```

### IMPORTANT: bf16 Precision Required

> **DoRA + Quantization = Degraded Output.** This model uses DoRA (Weight-Decomposed LoRA) which decomposes weight updates into magnitude and direction components. Quantization corrupts these components. **You must run inference at bf16 precision.**

### Inference (CPU Offload &mdash; Any GPU with 48 GB system RAM)

```python
import torch
from diffusers import DiffusionPipeline
from peft import PeftModel

model_id = "Qwen/Qwen-Image-2512"
lora_id = "RayyanAhmed9477/qwen-image-2512-lora-advertisement"

# Load at bf16 (REQUIRED for DoRA)
pipe = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16)
pipe.transformer = PeftModel.from_pretrained(pipe.transformer, lora_id)
pipe.enable_model_cpu_offload()  # Needs ~48 GB RAM, ~8 GB VRAM

# Generate — must use true_cfg_scale (guidance_scale is IGNORED for this model)
image = pipe(
    prompt="Professional product advertisement photograph of a premium smartphone, "
           "hero shot on clean gradient background, dramatic studio lighting, "
           "commercial photography, 8K quality",
    negative_prompt="",          # Required for true CFG to activate
    true_cfg_scale=4.0,
    num_inference_steps=30,
    height=768, width=768,
    generator=torch.Generator(device="cpu").manual_seed(42),
).images[0]

image.save("ad_output.png")
```

### Inference (Full GPU &mdash; A100/H100 with 50+ GB VRAM)

```python
pipe = DiffusionPipeline.from_pretrained("Qwen/Qwen-Image-2512", torch_dtype=torch.bfloat16)
pipe.transformer = PeftModel.from_pretrained(
    pipe.transformer, "RayyanAhmed9477/qwen-image-2512-lora-advertisement"
)
pipe.to("cuda")  # Requires ~50 GB VRAM

image = pipe(
    prompt="High-end luxury perfume advertisement, crystal bottle on black surface, "
           "dramatic lighting, premium commercial photography",
    negative_prompt="",
    true_cfg_scale=4.0,
    num_inference_steps=30,
    height=768, width=768,
    generator=torch.Generator(device="cpu").manual_seed(42),
).images[0]
```

### Prompt Engineering Guide

The model responds best to structured prompts:

```
[Photography type] of [product/subject], [setting/background],
[lighting description], [photography style], [quality keywords]
```

| Industry | Example Prompt |
|----------|---------------|
| **Technology** | "Professional product advertisement of a flagship smartphone, clean gradient background, three-point studio lighting, commercial photography, ultra sharp" |
| **Luxury** | "High-end luxury fragrance advertisement, crystal perfume bottle on glossy black surface, dramatic Rembrandt lighting, premium commercial photography" |
| **Food** | "Premium artisan coffee advertisement, cappuccino with latte art in white ceramic cup, warm golden morning lighting, food photography" |
| **Automotive** | "Luxury automotive advertisement, sleek black SUV on mountain road at golden hour, cinematic lighting, professional automotive photography" |
| **Fashion** | "Premium luxury watch advertisement, timepiece on dark velvet, dramatic side lighting highlighting metallic details, editorial jewelry photography" |
| **Real Estate** | "Luxury penthouse interior advertisement, modern living room with floor-to-ceiling windows, warm ambient lighting, architectural photography" |

---

## Training Configuration

### Stage 1: Composition Learning (512px) &mdash; 63 Hours

```yaml
Resolution:             512x512
Steps:                  10,000 (~3.4 epochs)
Speed:                  ~22.5 seconds/step
Duration:               ~63 hours
Batch Size:             1 (gradient accumulation = 4, effective batch = 4)
Optimizer:              AdamW 8-bit (lr=1e-4, cosine schedule, 300 warmup steps)
Loss:                   Min-SNR-gamma=5.0 weighted flow matching MSE
Noise Offset:           0.05 (improves contrast and dark areas)
Mixed Precision:        bf16 with gradient checkpointing
Final Loss:             0.96
```

### Stage 2: Detail Sharpening (768px) &mdash; 17 Hours

```yaml
Resolution:             768x768 (resume from checkpoint-10000)
Steps:                  1,500 (total: 11,500)
Speed:                  ~36 seconds/step
Duration:               ~17 hours
Learning Rate:          3e-5 (lowered to preserve Stage 1 features)
Warmup:                 50 steps
Final Loss:             0.17
```

**Why curriculum learning works:** Low resolution teaches *what* makes a good advertisement (composition, color harmony, product framing). High resolution then teaches *how* to render it sharply (edges, textures, fine detail). The lower learning rate in Stage 2 prevents catastrophic forgetting of the compositional understanding learned in Stage 1.

### Training Curves

<img src="plots/training_overview.png" width="100%">

<details>
<summary><b>Individual training plots</b></summary>

#### Training Loss
<img src="plots/training_loss.png" width="100%">

Steady descent across both stages. The jump at step 10,000 reflects the resolution change from 512px to 768px &mdash; the model quickly adapts.

#### Learning Rate Schedule
<img src="plots/learning_rate.png" width="100%">

Stage 1: Linear warmup to 1e-4, cosine decay over 10,000 steps. Stage 2: Warm restart to 3e-5, cosine decay over 1,500 steps.

#### Gradient Norm
<img src="plots/grad_norm.png" width="100%">

Gradient clipping at 1.0 maintains stability across both resolution stages.

#### VRAM Usage
<img src="plots/vram_usage.png" width="100%">

Consistent ~22 GB throughout, well within the RTX 3090's 24 GB limit.

</details>

---

## Hardware & Reproducibility

### Training Hardware

| Component | Specification |
|-----------|--------------|
| **GPU** | NVIDIA RTX 3090 &mdash; 24 GB VRAM, consumer-grade ($1,500) |
| **RAM** | 64 GB DDR4 |
| **OS** | Windows 11 Pro |
| **Stage 1 Duration** | ~63 hours (10,000 steps &times; 22.5s/step) |
| **Stage 2 Duration** | ~17 hours (1,500 steps &times; 36s/step) |
| **Total Training** | **~80 hours on a single consumer GPU** |
| **Dataset Preparation** | ~8 hours (API scraping + filtering + VLM captioning) |
| **Evaluation & Showcase** | ~4 hours (40 eval images + 24 showcase images) |
| **Total Project GPU Time** | **~92 hours** |

### Inference Requirements

| Setup | VRAM | RAM | Speed (768px, 30 steps) | Quality |
|-------|:----:|:---:|:-----------------------:|:-------:|
| **bf16 + CPU Offload** | ~8 GB | ~48 GB | ~120s | Full quality |
| **bf16 Full GPU** (A100) | ~50 GB | ~16 GB | ~20s | Full quality |
| 4-bit/8-bit (any) | ~12 GB | ~20 GB | ~35s | **Degraded (DoRA issue)** |

### Reproduce Training

```bash
git clone https://github.com/Rayyan9477/MarketMind
cd MarketMind/image_model_finetuning
pip install -r requirements.txt

# Stage 1: ~63 hours on RTX 3090
python train_lora.py --config configs/training_config.yaml

# Stage 2: ~17 hours on RTX 3090
python train_lora.py --config configs/training_config_hires.yaml \
    --resume-from-checkpoint outputs/checkpoints/checkpoint-10000
```

---

## Lessons Learned

Building this model taught me more about engineering under constraints than any course or tutorial:

1. **Dataset quality > quantity > architecture.** Switching from 300px ADImgNet images to 842px Pexels images was the single highest-impact change. No amount of LoRA rank or training steps can overcome a low-resolution, poorly-curated dataset.

2. **Resolution curriculum is free performance.** Training at 512px (2.25&times; cheaper per step) teaches composition first; fine-tuning at 768px adds sharpness. You get both for less than training at 768px alone.

3. **DoRA is powerful but has hidden deployment costs.** The quality improvement is real. The bf16-only requirement for inference is painful. For production systems where quantized serving matters, standard LoRA may be more practical.

4. **VLM caption enrichment was the highest-ROI per-image investment.** Running Qwen3-VL-8B over every training image gave the model far richer training signal. Stochastic 70/30 mixing of long/short captions improves both text alignment and unconditional generation quality.

5. **Pre-caching embeddings is non-negotiable on consumer GPUs.** The 6 GB savings from unloading the text encoder was the difference between "impossible on RTX 3090" and "fits with headroom to spare."

6. **Build monitoring and graceful resume from day one.** Over 80 hours on Windows, things go wrong. My SIGTERM-based graceful pause + checkpoint resume system saved the project multiple times.

7. **Windows memory management is its own engineering domain.** CUDA allocations consume Windows commit charge (RAM + pagefile). I built commit-aware device mapping that queries available system commit via ctypes before placing model layers across GPU, CPU, and disk. This is rarely documented but critical for large model work on consumer hardware.

---

## File Structure

```
qwen-image-2512-lora-advertisement/
├── adapter_config.json              # LoRA config (DoRA, rsLoRA, 12 target modules)
├── adapter_model.safetensors        # LoRA weights (1.7 GB)
├── training_config.yaml             # Stage 1 config (512px, 10K steps)
├── training_config_hires.yaml       # Stage 2 config (768px, 1.5K steps)
├── evaluation_results_lora.json     # LoRA evaluation metrics (n=20)
├── evaluation_results_base.json     # Base model evaluation metrics (n=20)
├── README.md                        # This file
├── showcase/                        # Generated showcase gallery
│   ├── v2_seed_*_*.png              # LoRA outputs (8 categories × 3 seeds = 24 images)
│   ├── lora_*.png                   # Additional LoRA samples
│   ├── base_*.png                   # Base model outputs for comparison
│   ├── *_ad.png                     # Ad overlay composites
│   └── comparison_grid.png          # Side-by-side comparison
├── plots/                           # Training visualization
│   ├── training_overview.png        # Combined 2×2 overview
│   ├── training_loss.png
│   ├── learning_rate.png
│   ├── grad_norm.png
│   └── vram_usage.png
└── samples/                         # Additional evaluation samples
```

---

## Troubleshooting

<details>
<summary><b>Images look garbled or blurry</b></summary>

You're running at quantized precision. This LoRA uses **DoRA** which **requires bf16**:
- Ensure `torch_dtype=torch.bfloat16` in `from_pretrained()`
- Do NOT use `BitsAndBytesConfig` or `load_in_4bit`/`load_in_8bit`
- Do NOT call `merge_and_unload()` &mdash; use PeftModel wrapping instead
</details>

<details>
<summary><b>Images don't match the prompt</b></summary>

Qwen-Image-2512 **ignores `guidance_scale`**. You must use `true_cfg_scale`:
```python
pipe(prompt="...", negative_prompt="", true_cfg_scale=4.0)
```
</details>

<details>
<summary><b>Out of memory</b></summary>

Use CPU offload (needs ~48 GB system RAM but only ~8 GB VRAM):
```python
pipe.enable_model_cpu_offload()  # Instead of pipe.to("cuda")
```
</details>

<details>
<summary><b>QwenImagePipeline not found</b></summary>

Update diffusers: `pip install --upgrade diffusers>=0.33.0`
</details>

---

## Citation

```bibtex
@misc{ahmed2026qwenimagelora,
  title   = {Qwen-Image-2512 LoRA for Professional Advertisement Generation:
             80 Hours of DoRA + rsLoRA Fine-Tuning on Consumer Hardware},
  author  = {Rayyan Ahmed},
  year    = {2026},
  url     = {https://huggingface.co/RayyanAhmed9477/qwen-image-2512-lora-advertisement},
  note    = {80 hours training on a single NVIDIA RTX 3090. 12,917 Pexels images,
             two-stage resolution curriculum (512px to 768px), 11,500 steps,
             DoRA + rsLoRA with Min-SNR-gamma flow matching loss.
             Part of the MarketMind AI Marketing Platform.}
}
```

## Acknowledgments

- [ModelScope](https://modelscope.ai/) &mdash; Hosting the 2026 Qwen-Image LoRA Training Competition
- [Qwen Team / Tongyi Lab](https://github.com/QwenLM) &mdash; The Qwen-Image-2512 base model
- [Hugging Face](https://huggingface.co/) &mdash; Diffusers, PEFT, Transformers, and Accelerate libraries
- [DoRA (Liu et al., 2024)](https://arxiv.org/abs/2402.09353) &mdash; Weight-Decomposed Low-Rank Adaptation
- [Pexels](https://www.pexels.com/) &mdash; High-quality stock photography under permissive license

---

<div align="center">

**Built entirely on a single NVIDIA RTX 3090** | **80 hours of training** | **12,917 images** | **25,834 VLM captions**

**Stack:** PEFT 0.18.0 | Diffusers 0.37.0 | DoRA + rsLoRA | Flow Matching | Min-SNR-gamma=5.0

*Created by [Rayyan Ahmed](https://github.com/Rayyan9477) as part of the [MarketMind](https://github.com/Rayyan9477/MarketMind) AI Marketing Platform*

</div>
