HIGH-FLYER | AI BLOG

New Releases

FlashAttention: A Novel Attention Algorithm with IO Awareness, Fast and Memory-Efficient

2025-11-25

At the heart of the Transformer model is the self-attention mechanism, which has both time and storage complexity at the O(N2)O(N2) level in terms of sequence length. As the scale of large language models (LLMs) continues to grow, equipping LLMs with longer contextual backgrounds poses a significant engineering implementation challenge. A team of researchers from the Department of Computer Science at Stanford University and the State University of New York at Buffalo has published a novel attention algorithm called FlashAttention, which not only has a longer context than PyT

categorization

HAI Platform: An Open-Source AI Training Platform

2025-11-25

With the rapid development of large AI models, many companies have begun building large-scale deep learning training clusters based on GPUs. As computing power demands grow, an AI platform capable of centrally and efficiently managing GPU resources is increasingly becoming a key tool for AI teams to reduce costs and improve efficiency. Building an AI platform capable of managing the resources of a large-scale GPU cluster primarily involves addressing the following challenges: Resource Scheduling: As computing power continues to expand and the computational demands of training tasks vary widely, how can the relationship between tasks and computing power be managed to maximize cluster resource utilization?

Continuous Batching：一种提升 LLM 部署吞吐量的利器

2025-11-25

由于 LLM 巨大的 GPU 内存开销和计算成本，在大多数应用中，机器学习工程师通常通过内部调整（如量化和对 CUDA 核的定制）来优化。然而，由于 LLM 通过迭代生成其输出，并且 LLM 推理通常涉及内存而不是计算，因此在很多实践中，优化系统级批处理可以使性能差异达到10倍甚至更多。 一种最近提出的优化方法是连续批处理（Continuous batching），也称为动态批处理或基于迭代级的批

LLaMA-2 技术详解（一）：数据打标

2025-11-25

LLaMA 是目前备受关注的开源大语言预训练模型。最近 Meta 发布了 LLaMA 2，它是 LLaMA 的下一代版本，具有商业友好的许可证。LLaMA 2 有 3 种不同的尺寸：7B、13B 和 70B，在对话用例下微调的模型版本被称为 LLaMA 2-Chat。Llama-2-Chat 模型在大多数基准测试中都优于现有的开源聊天模型，在对有用性和安全性的人工评估中，LLama-2-Chat

深入浅出 GPT-4 的体系结构

2025-11-25

GPT-4（Generative Pre-trained Transformer 4）是 OpenAI 发布的最新 GPT 系列模型。它是一个大规模多模态模型，相比 GPT-3.5 / ChatGPT，GPT-4 可以接受图像和文本两种形式的输入，产生文本输出。输出依旧是一个自回归的单词预测任务。技术上，GPT-4 采用了专家混合（MoE）技术，进一步增强模型的能力。整体来说，GPT-4 在各种专

CC_Cleaner: A Smooth, Efficient, and Easily Extensible Data Cleaning Workflow

2025-11-25

AGI is the perfect embodiment of data × algorithms × computing power, and the elegant art of research + engineering + organization. In the preliminary stages of training large models, big data cleaning serves as the foundation of data processing. Take the Common Crawl dataset as an example. It is readily available on Amazon and is a free, petabyte-scale web crawler dataset that includes data collected over more than 12 years: raw web page data (WARC), metadata extracts (WAT), and text extracts (WET). How to handle the vast and complex raw data

HAI-LLM: An Efficient and Lightweight Tool for Training Large Language Models

2025-11-25

To better leverage the computational power of GPU clusters and train powerful, trillion-parameter models with remarkable capabilities, an efficient and streamlined large-model training tool is essential. Huanfang-Basic Research has recently developed a deep learning training tool called HAI-LLM, which implements four parallel training methods: data parallelism supported by ZeRO, pipelined parallelism, tensor-slicing model parallelism, and sequential parallelism. This parallel processing capability adapts to the needs of different workloads, supporting ultra-large models on the order of trillions of parameters and scaling to thousands of GPUs. Based on the characteristics of the Firefly cluster, the in-house developed ha