Decision models are getting a lot of attention, with models such as Jev helping to put an appealing idea on the map: AI that makes the small decisions inside a workflow fast enough for software to respond almost as soon as a message arrives. Imagine a customer reporting a problem in Dutch. Before anyone has opened the message, it could already be recognised as urgent, routed to the right team and queued for follow-up. These models are also relatively lightweight and therefore easy to self-host, and there are even ready-made scripts for fine-tuning them yourself. Fast decisions, optimised for your own workflows, without sending each message to an external model API: that combination had us itching to try it.
During Mozaik's four-day trip to Nizza Monferrato in Piedmont, Ruben, Aziz, Dimitris and Eelco put that idea to the test. We fine-tuned a small model on English and Dutch Slack conversations, with Dutch workplace language an explicit part of the experiment. With limited time and resources, we were able to incorporate the model in our existing work and show how surprisingly successful and straightforward the training can be.
Turning incoming messages into work
At Mozaik, we are building CID, short for Consider-It-Done. It connects AI to a customer's data and tools so it can pick up requests, follow the team's procedures and carry out work with human approval. For example, it can gather information for a requested report, prepare it and ask for approval before sending it.
The first step in this workflow is deceptively simple: a message comes in, and our application has to decide whether this message needs action. "Kun je hier nog even naar kijken?" is a familiar Dutch request to take a look at something. "Is inmiddels geregeld" says it has already been handled. Recognising the language is only part of the job; the model also needs to understand what those messages mean for the work.
CID currently uses a large language model, or LLM, to interpret these messages. We wanted to see whether a small decision model could handle those initial judgements, freeing up the larger model for planning and writing.
Why we wanted to try a smaller model
A decision model chooses from answers you define, rather than generating free text: yes or no for whether action is needed, for example, or low, medium and high for urgency. It also returns probabilities for those answers. The software can use them directly to choose its next step.
The speed is what makes this exciting. If the first decision takes very little time, as short as a few milliseconds, useful work can start while the message is still fresh. For a busy inbox, the prospect of doing that cheaply, over and over, is compelling.
We chose Kev, Jared Palmer's open family of decision models, starting with its smallest version, Kev-0.8B. A model this small can run predictions on most modern laptops or a relatively small virtual machine, with speed depending on the hardware and workload. That makes hosting it ourselves practical: the messages it classifies stay on our own hardware or in our private cloud. We control where decisions happen and how the model connects to our systems.
With Kev being open, we also have the opportunity to fine-tune it to our needs: give it extra training on our task to improve its predictions. The training data of these models is primarily English, while our customers generally communicate in Dutch. Fine-tuning would let us test whether we can close that gap, and at the same time tune the model to our specific task and jargon.
Adapting the model to Dutch workplace language
The first step of fine-tuning is gathering training data. To optimise performance for the task at hand, the training material had to resemble our actual use cases, so it needed to look like real workplace conversation. That meant half-sentences, references to earlier messages, sarcasm and requests that someone had already dealt with. A collection of perfectly polite instructions would not be much of a test.
We generated nearly 6,000 fictional conversations, roughly half in Dutch and half in English, across 32 workplace settings. The Dutch conversations were written directly in Dutch, including informal phrasing and the occasional typo. After a few iterations of providing examples and guiding the model, we put two DeepSeek V4.1 Flash agents, accessed through OpenRouter, on the job: one in charge of writing the conversations, the other labelling what the predicted results should be on 11 different decisions, including "Does this message need action?", "How urgent is this message?" and "Who should handle this message?". These conversations and labels became the examples used to teach the small decision model our task in the next step.
Kev's training recipe handled much of the machinery. We fine-tuned the model on our self-hosted NVIDIA DGX Spark, with a training run of about five hours. Generating and labelling the examples through OpenRouter cost $10.45; the training itself ran on our own hardware.
We were impressed by how straightforward that part was. Fine-tuning can sound like a substantial research project. Here, we had a model adapted to our task after five hours of training, using tools we could follow and reuse. Defining good examples and checking the answers still took care, but the training itself was surprisingly accessible.
The results and a name
After some teasing about what to call it, our fine-tuned model became "Joost". The Dutch expression "Joost mag het weten" roughly means "Who knows?". An optimistic choice for something we were training to give answers. Several rounds of logo and mascot iteration followed.
Training on our English and Dutch examples made a substantial difference. Across our eleven questions, agreement with the AI-generated labels rose from about 50% to 81%, with equal gains across Dutch and English. On the central question, whether a message should become a task, it went from 62% to 91%. Seeing that jump from a single training run was the highlight of the experiment.
During the hackathon demo, we were able to show what it means to actually incorporate this model in a workflow we know: posting a message in Slack and seeing a follow-up task appear within a second feels like magic. The decisions it made also matched our own expectations on inspection, and in several cases where our fine-tuned model and the LLM diverged, we actually agreed with the former. Now that the hackathon is over, the next step is to validate the impact on our live applications, and we are confident that these results will hold.
Where decision models can help
Think about the messages or documents your team sorts through every day. Does someone repeatedly read them just to decide where they should go, whether they need attention, or what should happen next? That is a good place to look. A support inbox could route messages to billing or technical support as they arrive. An operations team could have possible blockers brought to its attention among routine updates. A document workflow could flag incomplete submissions before someone spends time processing them.
For builders keen to try it, start with one decision and real examples in the languages your team uses. Kev's quick start and training recipe are useful starting points. Try the existing model, inspect its mistakes, and fine-tune where your Dutch phrasing, terminology or rules need extra attention. Check performance on separate examples in each language. If keeping messages on your own infrastructure is part of the goal, test that setup too. Self-hosting is a benefit of choosing an open model such as Kev; you still need suitable hardware and someone to run it. Measure the complete workflow, including any later steps that call external services.
We are actively considering decision models for customer projects. We left Italy with much more enthusiasm for trying them: with only three days of hackathon time and a conservative budget of around $10, we were able to prove the potential of these models on a flow that is at the basis of many use cases. There is also a lot of room to improve: we can curate new fine-tuning datasets to raise performance, maybe even beyond the best LLMs, test larger model sizes, and investigate other stages in our workflows where decision models can help.
If you recognise one of your own problems here, we would like to hear about it. Bring us a few examples in the language your team uses, along with your requirements for where the data should stay. We can help you test whether a decision model fits and put it to work in your environment.
