It feels like just yesterday when we were all surprised by the announcement of Jev - a "System one" model that does only one thing but does that one thing really fast: decide - it was in fact September 15th. All but three weeks later, I've utilized Jev in three different projects with mixed results. And of course, after Jev was released, all of a sudden there is an avalanche of decision models. As a quick recap, Jev can only:
Answer yes or no (binary or, as they call it, a Noul). For example, a typical question could be: "Does this sentence contain the word Bla? - the sentence: Bla Bla Bla" - the answer would be returned as either true or false - in this case as true.
Score or rank different options. For example: "What is the direct object in this sentence: Peter saw that his golden retriever was destroying his flower bed?" - with choices given as "An animal, A man-made object, A plant" - this gives me a score of 70% for "An animal", 9% for "A man-made object" and 21% for "A plant".
Select a particular option. With the exact same question and choices as in 2 - it will pick the animal.
So here are a couple of lessons I've taken away from Jev:
- Jev is not good with non-generic and somewhat niche questions and choices - the more generic, the better.
- Jev is very cheap and because of that, you're tempted to use it for many things and as often as possible. In one of my projects, Jev is now more expensive than my calls to the LLM, but to be fair, I do get a bit more quality and speed (running in parallel) compared to just using the LLM to resolve it.
- Jev is not good at all with large texts and narrow choices - the smaller the text, the better.
But this is probably just the beginning - I could envision a system that is much more complex and built on top of Jev, and that can react very quickly - think, for example, of the stock market - how do you interpret the latest headlines? I guess what I'm trying to say is that I've taken my project and looked for opportunities to have some of the decision-making outsourced to Jev - I would expect whole new systems to appear that are decision-first and then LLM 2nd.
The main learning is, of course: context engineering is still king. It doesn't matter if your model is super-fast if your context is engineered wrongly or is not applicable - but when you have a set of decisions to make and you have data points, it's absolutely preferred over an LLM call.
First lessons learned. What did you learn? Comments please!
#Jev TypeSafe AI
Comments
Post a Comment