Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Submitted in March 2024. This paper proposes improved noise sampling techniques for training rectified flow models and designs a new Transformer architecture that uses separate weights for image and text and supports bidirectional information flow. Experiments show that the method outperforms existing diffusion models in text-to-image synthesis, and the largest model surpasses current state-of-the-art on multiple metrics. The authors will release experimental data, code, and model weights.
This work combines Rectified Flow with Transformer architecture, significantly improving the quality of high-resolution text-to-image generation through improved noise sampling and bidirectional cross-modal attention. Its core contribution is demonstrating that rectified flow can surpass traditional diffusion models when scaled, and showing predictable performance improvements with scale. This has important implications for technology route selection in image generation.
The paper proposes an improved noise sampling strategy that biases towards perceptually relevant scales for training rectified flow models, addressing shortcomings of original sampling methods in perceptual quality. Additionally, a new Transformer architecture is designed where image and text tokens use separate weights and interact via bidirectional attention, enhancing text understanding. Experiments are conducted at multiple scales, validating the correlation between validation loss and generation quality. The largest model outperforms baselines like Stable Diffusion in human preference evaluations. The method achieves better text alignment and image fidelity while maintaining linear computational complexity.
This technology directly improves the quality and controllability of text-to-image generation, offering significant value to creative industries such as advertising, design, gaming, and film. Open-source models and code will accelerate industry adoption, potentially leading to new generation tools and platforms. Additionally, rectified flow as an alternative to diffusion models may change the ecosystem of model training and deployment.
It is recommended that image generation product teams evaluate this model as a potential next-generation engine and consider integrating it into design tools or content creation platforms. Investment institutions may focus on startups based on rectified flow technology and the application of this architecture in advertising creative automation.
Future attention should be paid to the scalability of this architecture in areas such as video generation and 3D content creation, as well as computational costs during large-scale deployment. The practical application effects of open-source models and community feedback will be important signals. Moreover, whether the bidirectional cross-modal design can be extended to other multimodal tasks (e.g., visual question answering) is worth exploring.