Choosing Your LLM Path: Custom-built, Open-Source, or Paid Options?
Custom models, open-source like Llama2, or paid like GPT-3.5/4 or Claude 2: what should you choose?
đź“Ś When to Build Models Internally?
I’d only build custom models if your tasks are simpler machine learning tasks (regression, clustering, classification, etc.), and even then I would use standard libraries for most use cases, or for tasks like object comparison using basic neural networks like CNNs. If it’s an LLM you need, then don’t build it yourself.
⚖️ Paid or Open-Source LLMs?
Often, the best approach is a mix of both. Platforms like Langchain enable efficient model selection based on cost-effectiveness. But be warned! Mismanagement can make open-source options pricier than their paid counterparts.
🔓 Open-Source LLMs: When and Why?
Choose these when:
* Your tasks don’t demand the absolute cutting-edge models.
* You have engineering resources to set up and manage infrastructure. You must understand your data and hosting costs here!
* Legal constraints exist. For instance, many enterprises won’t work with a platform that leverages LLMs like ChatGPT due to unclear legal ownership over generated outputs.
đź’° Paid LLMs: The Best Choice If...
* You have low-volume tasks or the volume is manageable cost-wise.
* Only top-tier models can handle the given task.
🌟 Why I Recommend OpenAI's API for Most Use Cases
From my experience, most applications can get away with just using OpenAI’s user-friendly API, eliminating the need for overheads like data management and model hosting. Don’t use a full dev team when a simple API call will do all you need.
📧 A Personal Example:
I built an email categorization app using GPT-4, which categorized my email for me, separated urgent emails or emails I needed to respond to, and listed actionable items/tasks. The entire setup took just 30 minutes! GPT-4 wrote the majority of the code, and I connected it using OpenAI's and Gmail’s APIs. The cost? Just a few bucks a month. Unless there's a need for specific data fine-tuning or RAG, most projects don't require the hassle of full-fledged model hosting.
