PyTorch Team Successfully Optimizes Meta Model, Achieving 8x Speedup While Maintaining Accuracy

Horace He, a leading figure in PyTorch, departs Meta to join a startup founded by OpenAI's former CTO

Netflix Recruits Machine Learning Scientists and Engineers to Drive Content Intelligence

Hackers Upload Malicious AI Models on HuggingFace Using 'Corrupted' Pickle Files

Recently, cybersecurity researchers discovered two malicious machine learning models that were uploaded quietly to the renowned machine learning platform HuggingFace. These models utilized a novel technique, successfully bypassing security detection through 'corrupted' pickle files, raising concerns. Karlo Zanki, a researcher at ReversingLabs, pointed out that the beginning of the pickle files extracted from these PyTorch format archives suggests that

Lightning AI Secures $50 Million in Funding, PyTorch Lightning Hits 160 Million Downloads

New York-based artificial intelligence company Lightning AI has recently completed a new funding round of $50 million. The investment round includes notable firms such as Cisco Investments, JPMorgan Chase, K5 Global, and technology giant NVIDIA. To date, Lightning AI's total funding has reached $103 million. Led by founder and CEO William Falcon, Lightning AI is known for its deep learning framework, PyTorch Lightning.

Anthropic Launches Prompt Evaluation Tool to Help Developers Optimize Prompt Quality More Quickly and Efficiently Step-by-step justification: 1. Identify the main subject: Anthropic is the company that has introduced a new tool. 2. Identify the action: launches is the verb describing the introduction of the tool. 3. Identify the tool's purpose: prompt evaluation tool is the tool designed to help developers. 4. Identify the benefits of the tool: help developers optimize prompt quality more quickly and efficiently describes how the tool improves the development process. 5. Combine the elements into a coherent and concise sentence: Anthropic Launches Prompt Evaluation Tool to Help Developers Optimize Prompt Quality More Quickly and Efficiently communicates all the necessary information while being easy to understand.

July 10, 2024 —— Anthropic Inc. announced today that its AI development platform has introduced new features aimed at simplifying the development process for AI applications. The new features include the ability to generate, test, and evaluate prompts within the Anthropic console, as well as functions for automatically generating test cases and comparing outputs. Writing excellent prompts is now very simple, just describe the task to Claude. The console provides an built-in prompt generator su

PyTorch Team Successfully Boosts Llama7B Inference Speed by 10x

The PyTorch team has achieved a 10x increase in Llama7B's inference speed, reaching 244.7 tok/s, through optimization techniques in less than 1000 lines of native PyTorch code. The optimization methods include using the torch.compile function from PyTorch 2.0, GPU quantization, speculative decoding, tensor parallelism, as well as using weight quantization in different precisions like int8 and int4. By combining these techniques, including 'compi

ChatGPT Refuses to Write Another Line of Code Unless Users Promise a 'Tip'?

When users promised to give a tip while using ChatGPT, the output seemed to be more enriched. One user conducted an experiment and found that promising a $20 tip increased ChatGPT's output by 6%; promising $200 increased it by 11%. Some users are concerned that this might affect ChatGPT's neutrality, making it more focused on monetary incentives. However, others believe this is merely a training result where polite individuals often receive better responses.

Accelerating Generative AI Models with PyTorch

The PyTorch team released a blog post titled 'Accelerating Generative AI with PyTorch II: GPT, Fast', detailing how to speed up generative AI models using native PyTorch. By utilizing Torch.compile and static KV caching, CPU overhead is reduced, achieving nearly a 10x increase in model speed. Employing INT8 quantization alleviates memory bandwidth bottlenecks, leading to further significant performance improvements. Speculative decoding is used to break serial dependencies, enabling...

Huawei Joins PyTorch Foundation as Primer Member: Promoting AI Development

Huawei has been a long-time supporter and contributor to the PyTorch ecosystem. Joining the PyTorch Foundation accelerates community development. Huawei has launched a comprehensive intelligence strategy that aligns with the mission of the PyTorch Foundation.

PyTorch 2.1 Released: 35x Speedup in Compilation, ExecuTorch Enables AI Deployment on Mobile Devices

PyTorch 2.1 has been released with a 35x speedup in compilation and support for tracing NumPy code. ExecuTorch enables AI deployment on mobile devices, extending to smartphones, VR headsets, and more. The new version provides automatic dynamic shape support, multiple performance improvements, and quantization support. NumPy code can be compiled to C++/CUDA via torch.compile, achieving a 35x acceleration on MacBook. Python execution efficiency has improved, even as Python usage increases, with execution speeds several times faster.