Anthropic has announced the launch of a new 'Prompt Caching' feature in its Claude series of large language models, aimed at significantly reducing AI costs for enterprises and enhancing performance. This feature allows users to store and reuse specific context information without incurring additional costs or delays. In public testing, the Claude 3.5 Sonnet and Claude 3 Haiku models will implement this feature, expected to achieve up to 90% cost reduction and double the response speed in certain scenarios. This feature is particularly suited for use cases that require...

The Anthropic API introduces a new prompt caching feature aimed at optimizing the user experience with the Claude model. This feature allows users to cache frequently used contextual information between API calls, significantly reducing the costs and latency of long prompts by up to 90% and 85%, respectively. Currently, it supports the public beta versions of Claude 3.5 Sonnet and Claude 3 Haiku, with plans to expand to Claude 3 Opus in the future. It is suitable for use cases such as conversational agents, coding assistants, handling large documents, and agent searches.

The Kimi Open Platform under Dark Side of the Moon has announced a 50% reduction in context cache storage costs. Specifically, the cache storage cost has been reduced from 10 yuan/1M tokens/min to 5 yuan/1M tokens/min.

Yesterday, Kimi Open Platform under the Moon's Dark Side announced the public test of Context Caching technology. This technology can reduce the cost of using large text flagship models by up to 90% for developers, without changing the API price, and significantly improve the response speed of the model. Context Caching is an efficient data management technology that allows the system to pre-store large amounts of data or information that may be frequently requested. This way, when you request t

The ModelCache architecture under Ant Group's CodeFuse includes the adapter, embedding, similarity, and data_manager modules. ModelCache can reduce the inference cost of large model applications and enhance user experience. Cache hits can reduce average latency by 10 times, with speed improvements of up to 14.5%. ModelCache will continue to optimize performance and accuracy, improving recall time.