FlashAttention is an open-source CUDA/attention kernel that speeds up Transformer training and inference by computing exact attention with optimized GPU memory usage. It is used by AI researchers and engineers to reduce latency and cost when deploying large language models.