Search AI Products and News

"MiniCPM-V2.6" is an edge-side multimodal artificial intelligence model that, with only 8B parameters, has achieved SOTA (State of the Art) results in single image, multiple images, and video understanding, all under 20B parameters, significantly enhancing edge AI's multimodal capabilities and being fully comparable to GPT-4V.

Shusheng · Puyu Lingbi (InternLM-XComposer) Version 2.5 was developed by the Shanghai Artificial Intelligence Laboratory, focusing on long context input and output capabilities, operating smoothly within a length of 96K, and trained with 24K interleaved image-text data. Key upgrades include: high-resolution image understanding, fine-grained video understanding, and multi-turn multi-image dialogue. In application, it can create web pages and write high-quality text-image articles. Evaluations show it surpasses state-of-the-art open-source models across 16 benchmark tests and performs at par with key tasks compared to GPT-4V and Gem.

Lately, a user named Home Assistant on the Internet has come up with a new twist on GPT-4V. He boldly connected GPT-4Vision directly to his home camera, allowing the AI to observe him around the clock without any blind spots, and the resulting video has been viewed over a million times on X! Imagine walking around your home and suddenly realizing you're being watched 24/7 by an AI that covers every angle. It might feel a bit like a "lab rat," but this foreign netizen takes it in stride. He think

In the world of AI, we have just welcomed a remarkable new member—Cambrian-1, a large multimodal language model (MLLM) developed by industry giants LeCun and Xie Saineng. The emergence of this model represents not only a technological leap but also a profound reflection on the research of multimodal learning.

A joint research team from Beihang University and Nanyang Technological University conducted extensive security tests on the GPT-4o model. Through thousands of API queries, researchers evaluated the security of text, image, and audio modalities of GPT-4o. The study found that while GPT-4o showed improvements in security against text jailbreak attacks, the newly introduced audio modality presented new attack surfaces, and its overall multimodal security was not as strong as the previous generation model GPT-4V.

Microsoft has open-sourced the multimodal model LLaVA-1.5, inheriting the LLaVA architecture and introducing new features. Researchers have tested it in visual question answering, natural language processing, image generation, and other areas, showing that LLaVA-1.5 has reached the highest level among open-source models.

The new gameplay of GPT-4V has garnered over 3000+ stars on GitHub's hot list, making web developers feel threatened. By simply drawing, framing, and clicking execute, users can generate web pages with buttons quickly and easily. Developer Sawyer Hood created this new gameplay using the GPT-4V API in just 5 hours, earning over 2700 likes. Combining the open-source whiteboard tldraw with the GPT-4V API allows for instant web page creation, with some users even creating DesignerGPT.

The newly proposed image reasoning benchmark HallusionBench is used to examine visual language models like GPT-4V, revealing issues with language and visual hallucinations. Models like GPT-4V exhibit a high error rate of up to 90% in generating language hallucinations influenced by parametric memory within HallusionBench. Additionally, models such as GPT-4V are prone to geometric and other visual illusions, indicating that their current visual capabilities are still limited. Simple image manipulations can easily mislead these models, reflecting their fragility.

["Zhejiang University alumni collaborated with Microsoft Research to launch the multimodal model LLaVA, challenging GPT-4V.","LLaVA has demonstrated excellent performance on 11 testing datasets, garnering over 6k stars.","The model exhibits comprehensive capabilities, exceeding 85% of the level of GPT-4V.","The open-source code, model, and training data for LLaVA are now available for use."]

The CogVLM-17B, developed through a collaboration between Tsinghua University and Zhipu AI, is a domestic multimodal model with outstanding performance. CogVLM-17B can not only recognize objects in images but also distinguish between fully visible and partially visible objects. The model employs a unique deep fusion method, achieving deep alignment of image features and text features through four key components. CogVLM-17B outperforms Google's models in various fields and is aptly referred to as the '14-sided warrior', showcasing its multimodal capabilities.