System

Small language model experiments

A hands-on model stack used to understand tokens, training, inference, and serving — not a chatbot product.

This system entry exists so the model work can be found beside QuoteSnap rather than hidden inside an essay. I experimented with the components of a language model in order to understand them. I am not shipping this as a public assistant, and I am not claiming frontier-scale training.

Purpose

I built and studied a small language-model stack because I wanted the model itself to stop being a black box in my product work. The objective was understanding, not competing with hosted frontier models.

Scope

The work covered data preparation, tokenization, embeddings, transformer behaviour, attention, training, inference, context limitations, quantization, and serving constraints. Each layer changed how I thought about the software around a model.

Main result

The experiment made me less interested in treating model capability as the entire product. It made representation, context quality, deployment cost, latency, structured outputs, and deterministic fallbacks much more important in my architecture decisions.

Inference is a product decision

A model can be accurate and still be the wrong model for a product envelope of latency, cost, and hardware. A half-second interaction encourages a different interface from a twenty-second interaction. Several model calls, tool calls, retrieval, reranking, image processing, and validation can accumulate delay even when each step looks acceptable. Quantization makes the same point in another form: maximum model capability is not always the objective.

This is one reason I became more interested in smaller models, selective context, caching, specialist systems, and deterministic fallbacks. The objective is not to remove intelligence. It is to use expensive intelligence where it produces enough value.

What this system is not

This is not presented as a frontier-scale model, a replacement for commercial assistants, or evidence of novel model architecture. It is a hands-on research system used to understand the model stack deeply enough to make better decisions when building larger AI products.

Publication status

The code, datasets, checkpoints, and training logs are not yet public. The current citable artefact is EXP-007 and the engineering conclusions that came from the work.

Abubakar, Imam. "Small language model experiments." Imam Abubakar AI Research, 2026. https://imamabubakar.com/research/small-language-model

This archive contains independent research, experiments, architecture studies, and technical observations. Some findings are exploratory and should not be interpreted as peer-reviewed scientific conclusions unless explicitly stated. Architecture studies of third-party products are based on publicly observable behaviour, published technical information, and engineering analysis. They are not claims of access to private source code or infrastructure.