Skip to main content

Are Small Language Models the future of GenAi?

(Source: Midjourney v6.1; prompt: two GenAi bots battling it out with smarts and wit)

I did a little experiment. I asked ChatGPT 4o (in the cloud) and the Llama 3.1 70b model (locally via Ollama), the same question: "What verse changes did Schiller make for Beethoven's 9th symphony?". Both models got the right answer, in fairness GPT 4.o did it much faster and had a much more elaborate answer than Llama3.1 70b that took a good 2 minutes (running on my desktop PC!). But in the end, I liked the answer from Llama 3.1 better because it was concise, to the point, minimalistic (as you would expect from a 70b model) and in a way more helpful. 

If you want to follow the the experiment in real-time, I've recorded it for just that purpose. This is also the reason why this is a blog post and not just an update, because I can upload more than one video file. 


ChatGPT 4o:






Llama 3.1 (70b) via Ollama (jump to 50 seconds, it takes a while to load the model): 





What do you think?

Labels:

Comments

Post a Comment

Popular posts from this blog

Vibe Coding Alert! How I Rebuilt a Wix Site and Fed the “AI Will End SaaS” Panic

My better half is an artist and maintains a Wix.com site. For the second time in two years, Wix decided to raise the hosting fees. That’s when I suggested to my spouse that I could rebuild the website and host it on Firebase (where I host most of my projects). I assumed this wouldn’t be a big deal (I was wrong) and started researching ways to use a lightweight CMS with Firebase support. Such a system exists — it’s called FireCMS — and it’s excellent. Before I dive deeper, here’s her original site (no longer a paid Wix site):  Miyuki's WIX site Her instructions were clear: replicate it as closely as possible. So I went to work. I created a product development document with use cases, scope, screenshots from the original site, the required features, and of course FireCMS integration. I used ChatGPT to draft the document, then set up a new Firebase instance, and finally launched the Vibe Coding agent (Claude Code). The process wasn’t too different from my other projects, but what sur...
2 comments
Read more

How I Ended Up Creating an AI Playground to Illustrate and Educate

TL;DR AI Playground User Guide
1 comment
Read more

MCP + Context: engineering for the context – hard lessons learned

  Intro I have built my own orchestration framework because most of what I’ve seen was too complex or tried to lock you into creating workflows a certain way. I wanted something very simple and yet maximally flexible. I’m not going into details here on the framework — that’s another blog post — but I will in some cases explain why I could do what I did thanks to the flexibility of the framework, which is a dynamic DAG, can do call-backs, and uses functions and MCP servers. I will also not explain in detail what I’m doing with my current workflow, other than to say I was looking for a way to bypass large language models and instead run it on my own system at home. I succeeded with that — but that’s another blog post. Instead, what I will try to explain in this post is the most important thing after prompt engineering: context engineering, and why it’s so crucial to manage that aspect (especially when you run this at home). Stage setting A couple of weeks ago, Anthropic posted this: ...
1 comment
Read more