Meta has announced its new free and open-source large language model (LLM), Llama 3.2, during Meta Connect 2024. This new language model is Meta’s first Llama model to feature visual processing capabilities. Llama 3.2 integrates a new model architecture that adds image encoder representations to the language model, allowing AI to analyze text as well as image and visual data. According to Meta CEO Mark Zuckerberg, the company’s decision to offer Llama 3.2 as an open-source model is in line with its broader strategy to integrate artificial intelligence across its platform.

Zuckerberg described Llama 3.2 as “our (Meta’s) first open-source, multimodal model,” meaning it can interpret and respond to visual information, thereby expanding its range of applications. Zuckerberg explained that Llama 3.2’s main feature is its ability to understand both text and images. This “multimodal” capability allows the model to provide feedback based on visual data. For example, if a user uploads a photo of a bird, Llama 3.2 can analyze the image and identify the species. It can also modify images on request, such as adding or removing objects or changing the background. Llama 3.2 is now integrated into Meta’s platforms, such as Instagram and Facebook, allowing users to interact with the AI ​​in creative ways.

Additionally, Llama 3.2 includes voice interaction features, allowing users to interact with the AI ​​using voice commands. Meta has also introduced several celebrity voices including Dame Judi Dench, John Cena, and Awkwafina to make these user-AI interactions more engaging. The feature will initially be available in markets such as the US, Canada, Australia, and New Zealand, and is planned to expand further later.

Zuckerberg explained that Meta has optimized Llama 3.2 for mobile devices, allowing developers to create AI-powered applications that can work on smartphones. These applications will be able to interact with the phone’s camera or assist in performing tasks on the device, offering more direct and practical uses of AI in everyday scenarios.

Meta Llama 3.2 supports Meta AI

Meta is integrating Llama 3.2 across all of its platforms, but a key feature is that Llama 3.2 also supports Meta AI, a chatbot assistant available on WhatsApp, Messenger, and Instagram that is already used by more than 180 million people weekly. With its new visual processing capabilities, Meta AI can now assist users with a variety of tasks, such as making product recommendations based on images or answering questions using both text and visual data.

Meta is offering different versions of Llama 3.2 with varying levels of complexity and parameter size, ranging from 1 billion to 90 billion parameters. According to Zuckerberg, the smaller versions of Llama are designed to run efficiently on mobile devices, while the larger versions are meant for more demanding tasks.

In addition to Llama 3.2, Meta has introduced several updates to its AI tools across all its platforms. One such update is AI-generated captions for Instagram and Facebook Stories, allowing users to receive automated caption suggestions based on the content of their photos. Meta is also introducing AI-generated themes for direct messaging on Instagram and Messenger, adding more personalization options for users.

Meanwhile, for content creators, Meta has also introduced a live translation tool for videos. The feature will allow creators to dub videos into different languages ​​while maintaining lip sync, allowing them to reach a wider audience. Meta’s experimental AI tools also include live language translation between languages ​​like Spanish and English, as well as AI avatars for influencers to interact with their followers.