Open Source Today (2024-07-31): 2.7B Parameter SSM Hybrid Model Zamba2
Discover the latest in AI with open-source projects like Zamba2-2.7B, Cinemo, Megatron Energon, MindSearch, Agent Zero, and Outfit Anyone.
Here are some interesting AI open-source models and frameworks I wanted to share today:
Project: Zamba2
Zamba2-2.7B is a hybrid model combining state space models (SSM) and Transformer blocks.
It is based on the Zamba architecture, featuring a Mamba backbone with alternating shared Transformer blocks.
Zamba2-2.7B improves upon Zamba1 in three major ways: replacing Mamba1 blocks with Mamba2 blocks, using two alternating shared attention blocks, and applying LoRA projectors on each shared MLP block.
The model uses the Mistral v0.1 tokenizer, is pre-trained on 3T of text and code data, and annealed with 100B high-quality data in the second stage.
https://github.com/Zyphra/Zamba2
https://huggingface.co/Zyphra/Zamba2-2.7B
Project: Cinemo
Cinemo is an image animation project based on motion diffusion models, aiming for consistent and controllable image animation.
The project provides pre-trained weights and sampling code, supporting applications like motion transfer and video editing.
Animation generation can be experienced via simple command-line operations or a Gradio interface.
https://arxiv.org/abs/2407.15642
https://github.com/maxin-cn/Cinemo
Project: Megatron Energon
Megatron Energon is NVIDIA's open-source Megatron multimodal data loader, also usable independently.
It excels at loading large-scale training data for multimodal models, merging various datasets, distributing workloads across multiple nodes and processes, ensuring reproducibility and recoverability, and adapting easily to different data samples and processing types.
https://github.com/NVIDIA/Megatron-Energon
Project: MindSearch
MindSearch is an open-source AI search engine framework with performance similar to Perplexity.ai Pro.
Users can deploy their search engines using closed-source LLMs (e.g., ChatGPT, Claude) or open-source LLMs (e.g., InternLM2.5-7b-chat).
MindSearch aims to solve various life problems by browsing hundreds of web pages to provide deep and extensive knowledge answers, showcasing detailed solution paths, and greatly enhancing the credibility and usability of responses.
https://arxiv.org/abs/2407.20183
https://github.com/InternLM/MindSearch
Project: Agent Zero
Agent Zero is a dynamic, learnable AI agent development framework.
Designed as a general-purpose personal assistant, it can remember previous solutions, code, facts, and instructions to complete tasks faster and more reliably.
Agent Zero uses the operating system as a tool to complete tasks, can write its own code, and use the terminal to create and use its own tools. The framework is fully transparent, readable, understandable, customizable, and interactive.
https://github.com/frdel/agent-zero
Project: Outfit Anyone
Outfit Anyone is a high-quality virtual try-on project supporting any clothing and any person for virtual fitting.
The project is a client that calls an API; the model is not open source and is fixed, with no ability to upload or modify the model.
Users can upload their own clothing for virtual try-on.
