_Tools
Models Families
- OpenAI
- Cohere
- Anthropic
- PaLM2
- Gemini
- Mozaic
- Dolly
- Mistral
- Google Gemini
- LLama2
- Vicuna
...
Machine Learning plaforms
Vertex AI (by Google)
DataBricks
SageMaker (by AWS)
Azure ML (by Microsoft)
Orchestration
LangChain
LlamaIndex
Semantic-Kernel (by Microsoft)
Data Parsing tools
Memory DB tools
Data Validation tools
Evaluation tools
Security tools
Serving tools
LLM Metrics
User Feedback
TruBrics
Enables AI teams to collect, analyse and manage user prompts & feedback on models. This allows teams to:
- 🚨 Identify bugs - users are constantly running inference on models, and may be more likely to find bugs than an ML monitoring system.
- 🧑💻️ Fine tune - users often hold domain knowledge that can be useful to fine tune models.
- 👥 Align - identifying user preferences will help to align models to users.
- https://trubrics.github.io/trubrics-sdk/
Cache (to reduce costs and speed up inference)
GPTCache
ChatGPT and various large language models (LLMs) boast incredible versatility, enabling the development of a wide range of applications. However, as your application grows in popularity and encounters higher traffic levels, the expenses related to LLM API calls can become substantial. Additionally, LLM services might exhibit slow response times, especially when dealing with a significant number of requests. To tackle this challenge, we have created GPTCache, a project dedicated to building a semantic cache for storing LLM responses.
Token estimators
Tiktokenizer
Interpretability
OpenAI Explanations
Autotuned prompts / Coded prompts
DSPy
- https://dspy-docs.vercel.app/
- https://github.com/stanfordnlp/dspy/tree/main?tab=readme-ov-file
- https://spectrum.ieee.org/prompt-engineering-is-dead
- https://arxiv.org/abs/2310.03714
- https://arxiv.org/abs/2401.12178
- https://arxiv.org/abs/2312.13382
- https://arxiv.org/abs/2212.14024