large language models (LLMs) that maintain state, and it is built upon LangChain with the intention of being used in conjunction with it.

LangGraph expands the capabilities of the LangChain Expression Language by enabling the coordination of multiple chains or actors across multiple computational steps in a cyclical manner. This design is influenced by Pregel and Apache Beam. The current interface is modeled after NetworkX.The primary function of LangGraph is to introduce cycles into your LLM application. It is essential to note that this is NOT a directed acyclic graph (DAG) framework. If you wish to create a DAG, you should utilize the LangChain Expression Language directly. Cyclical structures are vital for agent-like behaviors, as they allow you to repeatedly call an LLM in a loop and request its next action.

How it works

Concept of stateful, multi-actor applications and how LangGraph enables their creation using LLMs

LangGraph is a library that enables the creation of stateful, multi-actor applications with LLMs (LangModel Models) using LangChain. It extends the LangChain Expression Language, allowing the coordination of multiple chains or actors across multiple steps of computation in a cyclic manner. This is particularly useful for building agent-like behaviors, where an LLM is called in a loop to determine the next action.

The concept of stateful, multi-actor applications is central to LangGraph. It allows the creation of applications where multiple actors (representing different components or entities) maintain their state and interact with each other in a coordinated manner. This is achieved by defining a StatefulGraph, which is parameterized by a state object that is passed around to each node. Each node then returns operations to update that state. These operations can either set specific attributes on the state or add to the existing attributes. The main type of graph in LangGraph is the StatefulGraph, which facilitates the management of state within the application.

Essentially, LangGraph enables the creation of stateful, multi-actor applications by providing a framework for coordinating multiple actors and managing their state using LLMs and LangChain.

LangGraph vs directed acyclic graph (DAG).

The main difference between LangGraph and a directed acyclic graph (DAG) is that LangGraph allows for cycles, while a DAG does not. A DAG is a directed graph with no directed cycles, meaning it is impossible to start at a vertex and follow the edges in such a way that eventually loops back to the same vertex. On the other hand, LangGraph specifically provides the ability to create cyclic behavior, allowing for repeated actions and interactions between actors in the graph

LangGraph Use Cases

Some examples of applications that can benefit from LangGraph include:

1. Agent-like Behaviors: LangGraph is useful for applications that require agent-like behaviors, where an LLM is called in a loop, asking it what action to take next. This can be applied in chatbots, conversational agents, or any system where an agent needs to make sequential decisions based on the state of the conversation or environment. CrewAI is building something similiar to this using LangChain.
2. Coordinating Multiple Chains or Actors: LangGraph extends the LangChain Expression Language with the ability to coordinate multiple chains or actors across multiple steps of computation in a cyclic manner. This feature is beneficial for applications that involve coordinating and managing multiple interconnected processes or actors.
3. Web-Enabled Agents: WebVoyager, built with LangGraph is a new kind of web-browsing agent using multi-model AI.
4. Stateful Applications: Applications that need to maintain and update a state as they progress, such as task-oriented dialogue systems, can benefit from the stateful nature of LangGraph.
5. Custom Tool Integration: LangGraph allows the integration of custom tools, making it suitable for applications that require the use of diverse external tools and services in their decision-making processes.

LangGraph is beneficial for applications that require agent-like behaviors, coordination of multiple actors, cyclic behavior, stateful processing, and integration of custom tools. It is particularly well-suited for building complex, interactive, and stateful language-based applications.

Compare and Contrast LangGraph

Would be interesting to see how it compares to Llamaindex, PyTorch. LLM Frameworks in general seem to get a lot of flak for over-complicating things. DSPy has also been gaining popularity. DSPy, is a framework for algorithmically optimizing LM prompts and weights, especially when LMs are used multiple times within a pipeline. DSPy separates the flow of the program from the parameters of each step, allowing for more systematic and powerful optimization of LM prompts and weights. DSPy also introduces new optimizers that are LM-driven algorithms that can tune the prompts and/or weights of LM calls to maximize a given metric.

Final Thoughts

LangGraph shows promise as a valuable addition to the growing ecosystem of LangChain. With its ability to enable the creation of stateful, multi-actor applications using LLMs and LangChain, LangGraph opens up new possibilities for building complex and interactive language-based systems.

The future of agents looks promising, as they are expected to have massive use cases. While agents in the past may have been ineffective and token-consuming, advancements in technologies like LangGraph can help address these challenges. By allowing for more advanced agent runtimes from academia, such as LLM Compiler and plan-and-solve, LangChain aims to enhance the effectiveness and efficiency of agents.

Stateful tools are also on the horizon for LangChain, which would enable tools to modify the state of applications. This capability would further enhance the flexibility and adaptability of stateful, multi-actor applications, enabling them to better respond to dynamic environments.

Moreover, LangChain is actively exploring the integration of more controlled human-in-the-loop workflows. This would provide opportunities for human involvement and guidance in decision-making processes, augmenting the capabilities of automated systems.

In the future, LangGraph and LangChain are expected to continue evolving and growing, offering more advanced features and expanding their capabilities. This opens up exciting research directions and potential applications that could benefit from advancements in LangGraph and similar technologies.

Overall, LangGraph’s potential to improve agent performance, enable stateful applications, and support multi-agent workflows positions it as a promising tool in the realm of language-based systems. As the LangChain ecosystem continues to thrive and innovate, we can anticipate even more exciting developments in the near future.

Alibaba, the world’s largest e-commerce giant in China, has released Qwen 1.5, a groundbreaking language model that has been making waves in the AI community. Developed in-house by Alibaba’s AI lab, Qwen 1.5 is the latest in line of innovative models. Back in November Alibaba released version 1 of Qwen 72B. This release includes several models, including their largest open source model, the 72B chat, which has surpassed the performance of other state-of-the-art models such as Claude 2.1 and GPT 3.5 on both MT-Bench and Alpaca-Eval v2. With a total of 6 models, Qwen 1.5 is capable of processing a 32K context length, making it a versatile and powerful tool for a wide range of applications.

Benchmarks & Performance

When it comes to benchmarks and Qwen 1.5 truly shines. In particular, the Qwen 1.5-7B model has shown impressive results in tool-use, outperforming the Mistral-7B model. This achievement highlights the robust capabilities of Qwen 1.5 in tasks requiring specialized knowledge and application.

The largest model in the Qwen 1.5 lineup, the 72B chat, delivers performance that is comparable to that of GPT-4, a highly advanced language model. This demonstrates the immense power and potential of Qwen 1.5 in leveraging artificial intelligence for complex language processing tasks.

With overall strong metrics across its different models, Qwen 1.5 offers users a reliable and efficient solution for a wide range of applications. Its impressive performance in various benchmarks showcases Alibaba’s commitment to pushing the boundaries of AI technology and delivering cutting-edge solutions to the e-commerce industry and beyond.

Closing Thoughts

In closing, Qwen 1.5 has demonstrated its remarkable capabilities and performance, particularly with its 72B model. This powerful language model exhibits performance that is comparable to, and even surpasses, Mistral-medium. This comparison serves as an encouragement for Mistral to release their proper mistral-medium model instead of relying on leaked Miqu weights. By doing so, it opens up the opportunity for further fine-tuning and improvement.

It’s worth noting that Qwen 1.5 has already paved the way for the development of a flagship LLM series called Quyen. This highlights the immense potential and impact of Qwen 1.5 in driving innovation and progress in the field of AI and language processing.

As we embrace the advancements brought forth by Qwen 1.5, we can anticipate further breakthroughs and discoveries that will shape the future of AI and its applications in various industries. Alibaba’s commitment to pushing the boundaries of AI technology is evident in the development and release of Qwen 1.5, ultimately driving progress and innovation in the e-commerce industry and beyond.

Nomic embeded-text-v1 is the newest SOTA long-context embedding model. Tired of drowning in unstructured data – text documents, images, audio, you name it – that your current tools just can’t handle? Welcome to the open seas of understanding, where Nomic AI’s Embeddings act as your life raft, transforming this chaos into a treasure trove of insights.

Forget rigid spreadsheets and clunky interfaces. Nomic Atlas, the platform redefining how we interact with information, empowers you to explore, analyze, and structure massive datasets with unprecedented ease. But what truly sets Nomic apart is its commitment to openness and accessibility. That’s where Embeddings, their latest offering, comes in.

Embeddings are the secret sauce, the vector representations that unlock the meaning within your data. Imagine each data point as a ship on a vast, trackless ocean. Embeddings act as lighthouses, guiding you towards similar data, revealing hidden connections, and making sense of the seemingly incoherent.

And the best part? Nomic’s Embeddings are truly open source, meaning they’re free to use, modify, and share. This transparency fosters collaboration and innovation, putting the power of AI-powered analysis directly in your hands.

The Struggle with Unstructured Data

AI loves structured data. Imagine feeding spaghetti to a baby – that’s like throwing unstructured data at AI. Text documents, images, videos – a tangled mess AI struggles to digest. It craves the neat rows and columns of structured data, the spreadsheets and databases where information sits organized and labeled. Nomic open source AI’s Embeddings are transforming that spaghetti into bite-sized insights, ready for AI and unlock the hidden potential within your data.

Understanding Embeddings

Where Embedding Can Help

Embedding models have the potential to assist companies and developers in several key ways:

• Handling Long-Form Content: Many organizations have vast troves of long-form content in research papers, reports, articles, and other documents. Embedding models can help make this content more findable and usable. By embedding these documents, the models can enable more semantic search and retrieval, allowing users to find relevant content even if the exact search keywords don’t appear in a document.
• Auditing Model Behavior: As AI and machine learning models permeate more sensitive and critical applications, explainability and auditability become crucial. Embedding models can assist by providing a meaningful vector space that developers can analyze to better understand model behavior. By examining how certain inputs map to vector spaces, developers can gain insight into how models handle different data points.
• Enhancing NLP Capabilities: Embedding models serve as a foundational layer that enhances many other natural language processing capabilities. By structuring language in vector spaces, embedding enables better performance downstream on tasks like sentiment analysis, topic modeling, text generation, and more. Embedding essentially extracts more understanding from text.

Embedding models empower more semantic search and retrieval, auditable model behaviors, and impactful NLP capabilities. Companies need embedders to help structure and exploit long-form content. And developers need embedding to infuse AI transparency and interpretability into sensitive applications. The vector spaces embedding provides for language are critical for many modern NLP breakthroughs.

Nomic AI’s Training Details

Nomic AI’s Embeddings boast impressive performance, and understanding their training process sheds light on this achievement. Instead of relying on a single training stage, Nomic employs a multi-stage pipeline, meticulously crafted to extract the most meaning from various sources.

Imagine baking a delicious cake. Each ingredient plays a specific role, and their careful combination creates the final masterpiece. Similarly, Nomic’s pipeline uses different “ingredients” in each stage:

Stage 1: Unsupervised Contrastive Learning:

• Think of this as building the cake’s foundation. Nomic starts with a large, pre-trained BERT model. Think of BERT as a skilled baker with a repertoire of techniques.
• Next, they feed BERT a unique dataset of weakly related text pairs. This might include question-answer pairs from forums like StackExchange, reviews with titles and bodies, or news articles with summaries. These pairings help BERT grasp semantic relationships between different types of text.
• Think of this stage as BERT learning the basic grammar and flavor profiles of different ingredients.

Stage 2: Finetuning with High-Quality Labeled Data:

• Now, the cake gets its delicious details! Here, Nomic introduces high-quality labeled datasets, like search queries and corresponding answers. These act like precise instructions for the baker, ensuring the cake isn’t just structurally sound but also flavorful.
• A crucial step in this stage is data curation and hard-example mining. This involves selecting the most informative data points and identifying challenging examples that push BERT’s learning further. Think of this as the baker carefully choosing the freshest ingredients and mastering complex techniques.

This two-stage approach allows Nomic’s Embeddings to benefit from both the broad knowledge base of the pre-trained BERT model and the targeted guidance of high-quality labeled data. The result? Embeddings that capture rich semantic meaning and excel at various tasks, empowering you to unlock the true potential of your unstructured data.

Conclusion

Nomic AI’s Embeddings offer a compelling proposition: powerful performance, unparalleled transparency, and seamless integration. By reportedly surpassing OpenAI’s text-embedding-3-small model and sharing their entire training recipe openly, Nomic empowers anyone to build and understand state-of-the-art embeddings. This democratization of knowledge fosters collaboration and innovation, pushing the boundaries of what’s possible with unstructured data.

There is also seamless integration with popular LLM frameworks like Langchain and Llamaindex makes Nomic Embeddings instantly accessible to developers working on advanced search and summarization tasks. This translates to more efficient data exploration, uncovering hidden connections, and ultimately, deriving deeper insights from your information ocean.

So, whether you’re a seasoned data scientist or just starting your AI journey, Nomic Embeddings are an invitation to dive deeper. With their open-source nature, powerful performance, and seamless integration, they unlock a world of possibilities, empowering you to transform your unstructured data into a gold mine of insights.

In the ever-evolving landscape of natural language processing, the pursuit of more powerful and versatile language models has led to remarkable breakthroughs. Among these, Mixtral 8x7B stands tall as a Sparse Mixture of Experts (SMoE) language model, showcasing a paradigm shift in performance and efficiency. This cutting-edge model, built upon the foundation of Mistral 7B, introduces a novel architecture with eight feedforward blocks (experts) per layer, revolutionizing the way tokens are processed.

With a keen focus on optimizing parameter usage, Mixtral 8x7B provides each token access to an impressive 47 billion parameters, all while utilizing a mere 13 billion active parameters during inference. Its unique approach, where a router network dynamically selects two experts for each token at every layer, allows for unparalleled adaptability and responsiveness.

Under the Hood: Mixtral 8x7B Architecture

Mixtral 8x7B is a Sparse Mixture of Experts (SMoE) language model that has the same architecture as Mistral 7B, with the difference that each layer is composed of 8 feedforward blocks (experts). For every token, at each layer, a router network selects two experts to process the current state and combine their outputs. Even though each token only sees two experts, the selected experts can be different at each timestep. As a result, each token has access to 47B parameters, but only uses 13B active parameters during inference.

Mixtral was trained with a context size of 32k tokens and it outperforms or matches Llama 2 70B and GPT-3.5 across all evaluated benchmarks. In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks. The model architecture parameters are summarized in Table 1, and a comparison of Mixtral with Llama is provided in Table 2. Mixtral outperforms or matches Llama 2 70B performance on almost all popular benchmarks while using 5x fewer active parameters during inference.

Here’s why Mixtral is special:

• It’s very good at different tasks like math, coding, and languages.
• It uses less power than other similar models because it doesn’t have to use all its experts all the time.
• This makes it faster and more efficient.

Think of it like this:

• You need to solve a math problem and a coding problem.
• Mixtral picks the math expert for the math problem and the coding expert for the coding problem.
• They both work on their tasks and give you the answers, but you only talk to them one at a time.
• Even though you don’t see all 8 experts all the time, they’re all ready to help if needed.

Benchmark Performances

The benchmark performances of the Mixtral 8x7B model, a Sparse Mixture of Experts (SMoE) language model, are compared to Llama 2 70B and GPT-3.5 across various tasks. Mixtral outperforms or matches Llama 2 70B and GPT-3.5 on most benchmarks, particularly in mathematics, code generation, and multilingual understanding.

It uses a subset of its parameters for every token, allowing for faster inference speed at low batch-sizes and higher throughput at large batch-sizes. Mixtral’s performance is reported on tasks such as commonsense reasoning, world knowledge, reading comprehension, math, and code generation. It is observed that Mixtral largely outperforms Llama 2 70B on all benchmarks, except on reading comprehension benchmarks, while using 5x fewer active parameters. Detailed results for Mixtral, Mistral 7B, Llama 2 7B/13B/70B, and Llama 1 34B2 are provided, showing that Mixtral outperforms or matches Llama 2 70B performance on almost all popular benchmarks while using 5x fewer active parameters during inference.

Impressive Retrieval

The retrieval accuracy of the Mixtral model is reported to be 100% regardless of the context length or the position of the information in the sequence. The model is able to successfully retrieve information from its context window of 32k tokens, regardless of the sequence length and the location of the information in the sequence.

Licensing and Open Source Community

Mixtral 8x7B is a Sparse Mixture of Experts (SMoE) language model that is licensed under the Apache 2.0 license, making it free for academic and commercial usage, ensuring broad accessibility and potential for diverse applications. The model is released with open weights, allowing the community to run Mixtral with a fully open-source stack. The French startup recently raised $415M in venture funding and has one of the fastest-growing open-source communities.

It’s worth noting that the details regarding the data used for pre-training and the specific loss function employed are conspicuously absent from the available information. This omission leaves a gap in our understanding of the model’s training process. There is no mention of whether any additional loss for load balancing is being utilized, which could provide valuable insights into the model’s optimization strategy and robustness. Despite this gap, the outlined architectural and performance characteristics of Mixtral 8x7B offer a compelling glimpse into its capabilities and potential impact on the field of natural language processing.

Microsoft has announced that Phi-2, its highly regarded Transformer model, will now be completely open source under the MIT License. This is a groundbreaking development that promises to usher in a new era of innovation and exploration within the field.

What is Phi-2?

Phi-2 is a state-of-the-art Transformer model boasting a whopping 2.7 billion parameters. It’s built for handling a variety of NLP tasks and was trained with an extensive dataset comprising 250 billion tokens, sourced from a combination of NLP synthetic data and carefully filtered web data.

Key Features of Phi-2:

• Transformer Model: Phi-2 operates on the transformer architecture, renowned for its effectiveness in processing sequential data and powering major advancements in natural language processing. Despite having only 2.7 billion parameters, Phi-2 has demonstrated strong performance on various benchmarks, often surpassing larger models. This suggests that it might offer a good balance of performance and efficiency.
• Massive Dataset: Phi-2 was trained on a massive dataset of 250 billion tokens, which includes both synthetic and real-world data. This diversity of data helps the model learn a broader range of language patterns and styles.
• **QA, Chat, and Code**: Specifically designed to perform well with QA formats, chat formats, and code generation, Phi-2 is versatile in its application.
• Research-Oriented: The model has not been fine-tuned with reinforcement learning from human feedback, positioning it as an ideal candidate for pure research purposes.

A Leap Towards Open Innovation

The recent shift to an MIT License for Phi-2 signifies a momentous occasion for developers, researchers, and hobbyists alike. Open-source licensing removes barriers to access, allowing for greater collaboration and transparency in research and development efforts.

What the MIT License Means for Phi-2:

• Unrestricted Access: Developers can use, modify, and distribute the model with fewer legal implications, fostering an environment of open innovation.
• Community Contributions: The open-source community can now contribute to Phi-2’s development, potentially accelerating improvements and enhancements.
• Wider Adoption: With fewer restrictions, Phi-2 could see increased utilization across various projects and domains, leading to a better understanding of its capabilities and limitations.

Outperforming the Competitors

In my weeks of exploration, it’s become evident that Phi-2 stands out among its peers. Compared with smaller models like the Gemini Nano 2, Phi-2 has shown superior performance on common benchmarks such as MMLU (MultiModal Language Understanding) and BBH (Beyond the Benchmark Hub).

As the AI community starts to leverage the now open-sourced Phi-2, the potential to bridge the performance gap with larger models on complex tasks and reasoning becomes more tangible. The added MIT License is set to catalyze innovation, paving the way for new breakthroughs in the utility and efficiency of AI models like Phi-2.

Conclusion: A New Chapter for AI Research

The decision by Microsoft to fully open source Phi-2 under the MIT License marks a pivotal point in AI research. By lowering the barriers to entry, Microsoft is not only promoting transparency but also empowering a broad range of researchers and developers to contribute to the advancement of AI.

Stay tuned, as I continue to delve into Phi-2’s capabilities and prepare to release an extensive guide that will complement our series of publications. The future of AI research has never looked brighter, and with tools like Phi-2 readily available, the possibilities are endless. Join us in exploring this remarkable model and become a part of the next wave of AI innovation!

In a move that turned heads and sparked instant debate, open-source model startup Mistral AI released their latest LLM not with a splashy launch event or polished press release, but with a simple tweet containing a single link: a magnet URL for a massive torrent file.

This audacious approach stands in stark contrast to the carefully orchestrated media blitz that accompanied Google’s recent Gemini launch, or the “over-rehearsed professional release video talking about a revolution in AI” that OpenAI’s Andrej Karpathy mocked on social media. While other companies were busy crafting narratives and highlighting their technological prowess, Mistral simply dropped the mic with a torrent link.

The LLM in question, MoE 8x7B, has generated immediate buzz within the AI community. Described by some as a “scaled-down GPT-4,” it’s believed to be a Mixture of Experts model with 8 individual experts, each possessing 7 billion parameters. This architecture mirrors what we know of GPT-4, albeit with significantly fewer parameters.

This bare-bones release, devoid of any formal documentation or promotional materials, is characteristic of Mistral AI. As AI consultant and community leader Uri Eliabayev noted, “Mistral is well-known for this kind of release, without any paper, blog, code or press release.” While some may find this approach unconventional, it has undoubtedly generated a significant amount of attention and speculation. As open source AI advocate Jay Scambler aptly put it, “It’s definitely unusual, but it has generated quite a bit of buzz, which I think is the point.”

Whether this unorthodox approach marks a new era of open-source AI development remains to be seen. However, one thing is certain: Mistral AI has succeeded in capturing the imagination of the AI community, and their enigmatic release has sparked important conversations about transparency, accessibility, and the future of large language models.

Details of the Release

In a tweet Mistral drops a torrent link containing 8x 7B MoE model but, there were no further details.

Torrent Info

Mistral AI provided minimal details about the release of MoE 8x7B, opting for a cryptic tweet containing only a torrent link. However, some insights can be gleaned from the limited information available.

Key Takeaways:

• Model Parameters: The params.json file reveals several key parameters:
  • Hidden dimension: 14336 (3.5x expansion)
  • Dimension: 4096
  • Number of heads: 32 (4x multiquery)
  • Number of KV heads: 8
  • Mixture of Experts (MoE): 8 experts, top 2 used for inference
• Related Code: While no official code for MoE 8x7B is available, the GitHub repository for megablocks-public likely contains relevant code related to the model’s architecture.
• Noticeably Absent: Unlike many other LLM releases, MoE 8x7B was not accompanied by a polished launch video or press release.

These details suggest that MoE 8x7B is a powerful LLM with a unique architecture. The MoE approach allows for efficient inference by utilizing only the top 2 experts for each token, while still maintaining high performance. The 3.5x expansion of the hidden dimension and 4x multiquery further enhance the model’s capabilities.

The timing of the release, just before the NeurIPS conference, suggests that Mistral AI may be aiming to generate interest and discussion within the AI community. The absence of a traditional launch event is likely intentional, as it aligns with Mistral’s more open-source and community-driven approach.

While the lack of detailed information may leave some wanting more, it also fosters a sense of mystery and excitement. This unorthodox approach has undoubtedly captured the attention of the AI community, and we can expect to see further analysis and experimentation with MoE 8x7B in the coming weeks and months.

Potential Parameters

The parameters in the JSON file provide some insights into the architecture of MoE 8x7B. The hidden dimension is 14336, which is 3.5 times larger than the dimension of 4096. This suggests that MoE 8x7B is a very powerful LLM with a high degree of complexity.

The number of heads is 32 and the number of KV heads is 8. This indicates that MoE 8x7B uses a multiquery attention mechanism, which allows it to process multiple input sequences simultaneously.

The MoE architecture is a type of Mixture of Experts model, which means that it consists of multiple experts, each specialized to perform a different task. In the case of MoE 8x7B, there are 8 experts. Only the top 2 experts are used for inference, which allows for efficient computing and high performance.

Basically, the parameters in the JSON file suggest that MoE 8x7B is a cutting-edge LLM with a unique architecture. It is still too early to say how well MoE 8x7B will perform on real-world tasks, but it is certainly a promising development in the field of AI.

What could this mean for the future of AI?

The release of MoE 8x7B demonstrates a few important trends in the field of AI:

• The increasing importance of open source software. MoE 8x7B is an open-source LLM, which means that anyone can download and use it for free. This is a significant development, as it democratizes access to powerful AI technology.
• The rise of new LLM architectures. MoE 8x7B is a Mixture of Experts model, which is a relatively new type of LLM architecture. This suggests that the field of LLM research is still evolving and that there is still significant room for innovation.
• The increasing focus on efficiency and performance. MoE 8x7B is designed to be efficient and performant, even when used on resource-constrained devices. This is important for enabling the use of LLMs in real-world applications.

This release of MoE 8x7B is a positive development for the future of AI. It demonstrates the power of open source software, the rise of new LLM architectures, and the increasing focus on efficiency and performance. It is likely that we will see more and more innovative and powerful LLMs being released in the coming years, and MoE 8x7B is a clear example of this trend.