Apple researchers have developed a new method for training large language models (LLMs) that seamlessly integrates both text and visual information.
The company’s findings, detailed in a research paper titled “MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training,” showcase a new approach to creating more intelligent and flexible AI systems. By utilizing a diverse dataset comprising image-caption pairs, interleaved image-text documents, and text-only data, Apple’s claims that the MM1 model sets a new standard in AI’s ability to perform tasks such as image captioning, visual question answering, and natural language inference with a high level of accuracy.
Apple’s research focuses on the combination of different types of training data and model architectures, which enables the AI to understand and generate language based on a mix of visual and linguistic cues. This capability is vital for tasks that require a nuanced comprehension of the world, such as interpreting complex images or answering questions that involve visual elements.
Read more at MacRumors.com