Models Families

• OpenAI
  • GPT-3.5
  • GPT-3.5 Turbo
  • GPT-4
  • GPT-4 Turbo
  • GPT-4o
  • GPT-4o mini
• Anthropic
  • Claude-3-Opus
  • Claude-3-Haiku
  • Claude-3.5-Sonnet
• Cohere
• PaLM2
• Mozaic
• Dolly
• Mistral
  • Mistral 7B
  • Mixtral 8x7B
  • Mixtral 8x22B
  • Mistral Small
  • Mistral Medium
  • Mistral Large
  • Mathstral
  • Codestral
  • Mistral Nemo 12B (Mistral + Nvidia)
• Google Gemini
• Meta LLama
  • Llama-3 8B
  • Llama-3 70B
• Vicuna
• Gorilla-LLM
  • gorilla-openfunctions-v2
    Gorilla OpenFunctions extends Large Language Model(LLM) Chat Completion feature to formulate executable APIs call given natural language instructions and API context.
    ...

Machine Learning plaforms

Vertex AI (by Google)

• https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/overview

DataBricks

• https://www.databricks.com/

SageMaker (by AWS)

• https://aws.amazon.com/sagemaker/

Azure ML (by Microsoft)

• https://azure.microsoft.com/en-us/products/machine-learning

Orchestration

LangChain

• https://www.langchain.com

LlamaIndex

• https://www.llamaindex.ai

Semantic-Kernel (by Microsoft)

• https://github.com/microsoft/semantic-kernel

Data Parsing tools

Memory DB tools

Data Validation tools

Evaluation tools

Security tools

Serving tools

LLM Metrics

User Feedback

TruBrics

Enables AI teams to collect, analyse and manage user prompts & feedback on models. This allows teams to:

• 🚨 Identify bugs - users are constantly running inference on models, and may be more likely to find bugs than an ML monitoring system.
• 🧑‍💻️ Fine tune - users often hold domain knowledge that can be useful to fine tune models.
• 👥 Align - identifying user preferences will help to align models to users.
• https://trubrics.github.io/trubrics-sdk/

Cache (to reduce costs and speed up inference)

GPTCache

ChatGPT and various large language models (LLMs) boast incredible versatility, enabling the development of a wide range of applications. However, as your application grows in popularity and encounters higher traffic levels, the expenses related to LLM API calls can become substantial. Additionally, LLM services might exhibit slow response times, especially when dealing with a significant number of requests. To tackle this challenge, we have created GPTCache, a project dedicated to building a semantic cache for storing LLM responses.

• https://github.com/zilliztech/GPTCache

Token estimators

Tiktokenizer

• https://tiktokenizer.vercel.app/

Interpretability

OpenAI Explanations

• https://github.com/openai/automated-interpretability

Autotuned prompts / Coded prompts

DSPy

• https://dspy-docs.vercel.app/
• https://github.com/stanfordnlp/dspy/tree/main?tab=readme-ov-file
• https://spectrum.ieee.org/prompt-engineering-is-dead
• https://arxiv.org/abs/2310.03714
• https://arxiv.org/abs/2401.12178
• https://arxiv.org/abs/2312.13382
• https://arxiv.org/abs/2212.14024

NeuroPrompts

• https://arxiv.org/abs/2311.12229

AutoPrompt

• https://arxiv.org/pdf/2402.03099.pdf