Developer BitPoet has introduced an innovative inpainting method for the Ideogram 4 model, which allows for image editing using reference frames instead of traditional masks.

image
image
image

What Happened

Developer BitPoet has implemented a generation control method via an "image-link-prompt" combination. To achieve this, the ai-toolkit and ComfyUI tools were modified, a specialized set of nodes (ComfyUI-bitpoet-IG4Inpaint) was created, and LoRA weights were prepared. The training process was conducted in two stages: first at low resolution (512 px) to master instruction semantics, and then on high-resolution images (1920×1072) to achieve final quality.

Context

The traditional inpainting process usually requires the use of geometric masks to specify the areas subject to change. This new approach proposes a shift toward semantic control, where the neural network receives a visual example of exactly what needs to be inserted or changed within the existing context.

Why It Matters for the Industry

The emergence of effective reference-based editing methods expands the possibilities of generation control, allowing for more precise integration of new objects into an existing context without the need for complex masks. This creates new niches for specialized design tools and automated content creation, paving the way for a transition from mask-based to reference-based editing as an industry standard.

Why It Matters for Users

For users and designers, this represents a qualitative leap in convenience: there is no longer a need to manually "paint over" areas on an image. Instead, one can literally show the neural network a supporting reference image so that it understands which object or style needs to be integrated into the current scene.

What Is Not Yet Known / Limitations

The current implementation is a research prototype and requires significant computational resources (VRAM), as well as the use of a specific tool stack (ComfyUI and a modified ai-toolkit), which limits its use in ready-made commercial API solutions at this time.

Sources

Author

Look at AI, Editorial Team