Microsoft announced the general availability of the Azure NDV5 VM series. Each NDV5 VM comes with 8 NVIDIA H100 GPUs, each providing 3,958 teraFLOPS (8-bit FP8), 80GB of GPU memory, and 3.35TB/s of GPU memory bandwidth.
Analog Iterative Machines introduce a new approach to optimization problems, which are often hard to solve using conventional digital computers. The approach is based on analog iterative machines, which are devices that use physical phenomena to perform computations in parallel and continuously.
DeepSpeed ZeRO++ is a new optimization technique that reduces the communication and memory overhead of training large language models (LLMs) and chat models.