Long story short: Unsloth is awesome, local and private - but it's a lot harder to use right now. We export the JSONL format they consume, but you still need to install it, open a notebook, tweak some code, export GGUFs, import GGUFs to Ollama, then add the model in Kiln. The usability is no where near the level we have for fine-tuning on OpenAI or Fireworks (a few clicks and you're running a serverless fine-tune).

Proposal:

Can we write a bridge so that running something like uvx run kiln-unsloth-bridge starts up a local API with higher level unsloth interface. Super high level idea:

• start_finetune API
  • takes a base model ID, a path to JSONL, optional val set JSONL, hyper params, deployment config
  • Streams progress (SSE)
  • Writes out progress and fine-tune to a local directory when done.
  • Depending on deployment options, deploys it. Ollama GGUFs to start.
  • Error handling

Questions:

Can this be a simple python library? It probably can for some use cases, but Kiln is packaged as a pyinstaller app - I'm betting some of the CUDA/MLX driver bits won't work if packaged in our app format. I should mess around with this and try it. I'm okay with a uvx run xyz if not, but I wouldn't want anything more tedious than that or we won't see a ton of adoption.

Are the unsloth folks planning a higher level api like this? Do they want to own it? Is this better in unsloth, independent, part of kiln, or a collab? There are probably about 50 important details to get right here: the default params for tuning each model, warnings/errors based on available memory/hardware, validation sets, resuming jobs, etc, etc.