PyTorch 2.1 Released: 35x Speedup in Compilation, ExecuTorch Enables AI Deployment on Mobile Devices

Horace He, a leading figure in PyTorch, departs Meta to join a startup founded by OpenAI's former CTO

Netflix Recruits Machine Learning Scientists and Engineers to Drive Content Intelligence

Hackers Upload Malicious AI Models on HuggingFace Using 'Corrupted' Pickle Files

Recently, cybersecurity researchers discovered two malicious machine learning models that were uploaded quietly to the renowned machine learning platform HuggingFace. These models utilized a novel technique, successfully bypassing security detection through 'corrupted' pickle files, raising concerns. Karlo Zanki, a researcher at ReversingLabs, pointed out that the beginning of the pickle files extracted from these PyTorch format archives suggests that

Lightning AI Secures $50 Million in Funding, PyTorch Lightning Hits 160 Million Downloads

New York-based artificial intelligence company Lightning AI has recently completed a new funding round of $50 million. The investment round includes notable firms such as Cisco Investments, JPMorgan Chase, K5 Global, and technology giant NVIDIA. To date, Lightning AI's total funding has reached $103 million. Led by founder and CEO William Falcon, Lightning AI is known for its deep learning framework, PyTorch Lightning.

PyTorch Team Successfully Boosts Llama7B Inference Speed by 10x

The PyTorch team has achieved a 10x increase in Llama7B's inference speed, reaching 244.7 tok/s, through optimization techniques in less than 1000 lines of native PyTorch code. The optimization methods include using the torch.compile function from PyTorch 2.0, GPU quantization, speculative decoding, tensor parallelism, as well as using weight quantization in different precisions like int8 and int4. By combining these techniques, including 'compi

Accelerating Generative AI Models with PyTorch

The PyTorch team released a blog post titled 'Accelerating Generative AI with PyTorch II: GPT, Fast', detailing how to speed up generative AI models using native PyTorch. By utilizing Torch.compile and static KV caching, CPU overhead is reduced, achieving nearly a 10x increase in model speed. Employing INT8 quantization alleviates memory bandwidth bottlenecks, leading to further significant performance improvements. Speculative decoding is used to break serial dependencies, enabling...

PyTorch Team Successfully Optimizes Meta Model, Achieving 8x Speedup While Maintaining Accuracy

The PyTorch team successfully rewrote Meta's SAM model, achieving an 8x speedup while maintaining accuracy. The optimization methods included applications of various PyTorch features like Bfloat16, GPU synchronization optimization, and Torch.compile. The article provides an in-depth analysis of SAM model performance, bottleneck resolution, and optimization techniques using new features such as SDPA technology. The rewrite of the SAM model addressed the matrix multiplication bottleneck through methods like pruning, leading to significant performance improvement.

Huawei Joins PyTorch Foundation as Primer Member: Promoting AI Development

Huawei has been a long-time supporter and contributor to the PyTorch ecosystem. Joining the PyTorch Foundation accelerates community development. Huawei has launched a comprehensive intelligence strategy that aligns with the mission of the PyTorch Foundation.

OpenLM: A Model Training Library Designed for Medium-Sized Language Models

OpenLM is a PyTorch codebase aimed at training medium-sized language models. It has achieved remarkable results by training two language models, OpenLM-1B and OpenLM-7B. OpenLM-1B performs exceptionally well in zero-shot text classification and multiple-choice tasks, while OpenLM-7B continuously improves zero-shot performance through ongoing training. Future work includes supporting multimodal models, expert mixture, and dataset combinations, as well as expanding OpenLM to support training larger models.

Intel Joins PyTorch Foundation as a 'Premier' Member: Driving 'AI Everywhere'

Intel has become a 'Premier' member of the PyTorch Foundation, aiming to advance the development of artificial intelligence. Intel has collaborated with PyTorch for many years, focusing on providing hardware and software solutions to promote the use of artificial intelligence. Intel has fully optimized PyTorch to enhance its efficiency and performance on Intel hardware.