utensil
PublicREADME.md
# utensil
[](https://github.com/zserge/utensil/actions/workflows/test.yml)
[](LICENSE)
A single-header C99 tensor library for training and running small neural networks, with an optional Apple Metal GPU backend. Built for tinyML and embedded machine learning work, where the dependency footprint and code size of mainstream frameworks are hard to justify.
- **No dependencies beyond libc and, optionally, the platform's own GPU framework.** No BLAS, no autograd engine, no build system beyond a compiler.
- **Single header.** Drop `utensil.h` into a project and `#include` it.
- **Manual backpropagation.** Every layer has an explicit, hand-written forward and backward function. There is no computation graph or automatic differentiation; gradients flow through function calls you write yourself, the same way you would derive them on paper.
- **Explicit device placement.** Tensors carry a device (`UT_CPU` or `UT_METAL`) and move between them lazily and explicitly.
- **Small enough to read in one sitting.** The library is a few thousand lines of straightforward C. Understanding a layer means reading its forward and backward functions, not tracing through several layers of abstraction.
This makes utensil suited to running and training small models directly on target hardware, to prototyping new layers or quantization strategies, and to teaching or auditing exactly how a network's numbers are produced — the kind of control tinyML deployments need and general-purpose frameworks are not built to give.
## Status
utensil is under active development. The API may still change between releases. It currently supports CPU execution on any C99 platform and GPU acceleration on Apple platforms via Metal and Metal Performance Shaders.
## Getting started
utensil is a single header. Copy `utensil.h` into your project:
```c
#include "utensil.h"
int main(void) {
ut_tensor* a = ut_randn(2, (int[]){2, 3}, 0.f, 1.f, UT_CPU);
ut_tensor* b = ut_randn(2, (int[]){3, 2}, 0.f, 1.f, UT_CPU);
ut_tensor* c = ut_matmul(a, b);
ut_free_all(a, b, c);
return 0;
}
```
On macOS, linking the Metal frameworks enables GPU execution automatically; tensors allocated with `UT_METAL` run on the GPU, tensors allocated with `UT_CPU` run on the CPU, and both can be freely mixed in the same program.
```sh
cc -std=c99 -O2 -DACCELERATE_NEW_LAPACK -framework Metal -framework MetalPerformanceShaders \
-framework Foundation -framework Accelerate main.c -o main
```
On any other platform, compiling without those frameworks silently falls back to the CPU backend; no source changes are required.
### Building and running the tests
```sh
make test
```
### Building the examples
```sh
make examples
```
`examples/` contains complete, runnable models: an MLP and a CNN trained on MNIST, and a ResNet-8 and a scaled-down MobileNetV1 trained on CIFAR-10. Each example includes its own data loader and training loop, and is a reasonable starting point for a new model.
## API overview
### Tensors and devices
```c
typedef enum { UT_CPU, UT_METAL } ut_dev;
typedef struct ut_tensor {
ut_shape shape;
float* data;
void* gpu_buf;
int rc;
struct ut_tensor* owner;
ut_dev dev;
bool dirty_cpu;
bool dirty_gpu;
} ut_tensor;
ut_tensor* ut_alloc(int ndim, const int* dim, ut_dev dev);
ut_tensor* ut_randn(int ndim, int* dim, float mean, float stddev, ut_dev dev);
ut_tensor* ut_from_data(int ndim, const int* dim, const float* data, ut_dev dev);
ut_tensor* ut_clone(ut_tensor* t);
ut_tensor* ut_retain(ut_tensor* t);
void ut_free(ut_tensor* t);
void ut_free_all(/* variadic ut_tensor* list */);
```
Tensors are reference-counted and can be views onto another tensor's storage (`ut_view`, `ut_reshape`, `ut_transpose`). A tensor's CPU and GPU copies are synchronized lazily, only when a call needs the other side's data:
```c
void ut_sync_cpu(ut_tensor* t); // pull GPU data to the CPU mirror
void ut_sync_gpu(ut_tensor* t); // push CPU data to the GPU buffer
void ut_to_device(ut_tensor* t, ut_dev dev);
```
### Core operations
```c
ut_tensor* ut_neg(ut_tensor* a);
ut_tensor* ut_exp(ut_tensor* a);
ut_tensor* ut_sigmoid(ut_tensor* a);
ut_tensor* ut_tanh(ut_tensor* a);
ut_tensor* ut_relu(ut_tensor* a);
ut_tensor* ut_relu6(ut_tensor* a);
ut_tensor* ut_hardsigmoid(ut_tensor* a);
ut_tensor* ut_hardswish(ut_tensor* a);
ut_tensor* ut_add(ut_tensor* a, ut_tensor* b);
ut_tensor* ut_sub(ut_tensor* a, ut_tensor* b);
ut_tensor* ut_mul(ut_tensor* a, ut_tensor* b);
ut_tensor* ut_div(ut_tensor* a, ut_tensor* b);
ut_tensor* ut_scale(ut_tensor* a, float s);
ut_tensor* ut_matmul(ut_tensor* a, ut_tensor* b);
ut_tensor* ut_softmax(ut_tensor* t, int dim);
```
Backward functions for the activations above (e.g. `ut_relu_backward`, `ut_relu6_backward`) take the upstream gradient and the layer's forward-pass input and return the gradient with respect to that input.
### Layers
Each layer follows the same shape: an `_alloc` function to create it, a `_forward` function that optionally fills a cache for backpropagation, a `_backward` function that consumes that cache and accumulates parameter gradients into caller-provided tensors, and a `_free` function.
```c
// Fully connected
ut_linear ut_linear_alloc(int in, int out, bool bias, ut_dev dev);
ut_tensor* ut_linear_forward(ut_linear* l, ut_tensor* x);
ut_tensor* ut_linear_backward(ut_linear* l, ut_tensor* x, ut_tensor* grad_out,
ut_tensor* dW, ut_tensor* db);
// Layer normalization
ut_layernorm ut_layernorm_alloc(int d, ut_dev dev);
ut_tensor* ut_layernorm_forward(ut_layernorm* l, ut_tensor* x, ut_layernorm_cache* cache);
ut_tensor* ut_layernorm_backward(ut_layernorm* l, ut_layernorm_cache* cache,
ut_tensor* grad_out, ut_tensor* dW, ut_tensor* db);
// 1D and 2D convolution
ut_conv1d ut_conv1d_alloc(int in_c, int out_c, int kw, int stride, int pad, bool bias, ut_dev dev);
ut_conv2d ut_conv2d_alloc(int in_c, int out_c, int kh, int kw, int stride, int pad, bool bias, ut_dev dev);
// Depthwise 2D convolution (MobileNet-style depthwise-separable blocks)
ut_dwconv2d ut_dwconv2d_alloc(int c, int kh, int kw, int stride, int pad, bool bias, ut_dev dev);
// Batch normalization (2D, train/eval modes with running statistics)
ut_batchnorm2d ut_batchnorm2d_alloc(int c, ut_dev dev);
ut_tensor* ut_batchnorm2d_forward(ut_batchnorm2d* l, ut_tensor* x, bool training,
ut_batchnorm2d_cache* cache);
// Pooling
ut_tensor* ut_global_avgpool2d(ut_tensor* x);
ut_tensor* ut_maxpool2d(ut_tensor* x, int kh, int kw, int stride, int pad, ut_maxpool2d_cache* cache);
ut_tensor* ut_avgpool2d(ut_tensor* x, int kh, int kw, int stride, int pad);
```
Each `_forward` has a matching `_backward` and each cache type has a matching `_cache_free`; see `utensil.h` for the complete, per-layer signatures.
### Loss functions
```c
float ut_cross_entropy(ut_tensor* logits, const int* labels, ut_tensor* grad_in);
float ut_mse(ut_tensor* pred, ut_tensor* target, ut_tensor* grad_in);
```
Both compute the scalar loss and write the gradient with respect to the input into `grad_in` in the same call.
### Optimizers
```c
ut_sgd ut_sgd_alloc(ut_tensor** params, int n, float lr, float momentum);
void ut_sgd_step(ut_sgd* o, float clip);
ut_adam ut_adam_alloc(ut_tensor** params, int n, float lr, float beta1, float beta2,
float eps, float weight_decay);
void ut_adam_step(ut_adam* o, float clip);
```
Both optimizers own a `grads[]` array parallel to `params[]`; layers accumulate gradients directly into the corresponding entry, and `_step` applies the update and zeroes the accumulator, with optional gradient-norm clipping.
## Design notes
**Lazy CPU/GPU sync.** A tensor can be dirty on the CPU side, the GPU side, or neither; each operation reads whichever side it needs and marks the other dirty, so data crosses the CPU/GPU boundary only when a specific operation actually requires it, not on every call.
**No autograd.** Backpropagation is written by hand for every layer. This is more code than a graph-based autograd engine, but it keeps every layer's math visible in one function and avoids the runtime and memory overhead of building and walking a computation graph — a deliberate tradeoff for constrained targets.
**Command batching on Metal.** GPU dispatches are queued onto a single command buffer and only committed when the CPU actually needs a result, letting independent GPU work pipeline instead of round-tripping to the CPU after every operation.
## License
Apache License 2.0. See [LICENSE](LICENSE).