Tutorial: design a custom OLS ABI

This advanced tutorial designs a hand-written C ABI for examples/ols/, installs its generated Python package, and customizes the façade. Start with Your first wrapper if you want JuliaLibWrapping to derive the boundary from ordinary Julia signatures with @api.

Hand-written entrypoints are appropriate here because the example deliberately uses caller-allocated output buffers, an application-specific result struct, and a custom scikit-learn-style Python interface. These are policy choices an ABI generator should not guess.

Hand-written ABI entrypoints

A hand-written entrypoint exposes its declared carrier types directly. The author is responsible for conversions, ownership, error statuses, and the foreign-language interface, while JuliaLibWrapping supplies the mechanical C and Python bindings. Read JLWInterop carriers and Manual error status handling before designing such a boundary.

The subject is ordinary least squares (OLS) regression, which exercises several JLWInterop types:

CMatrix and CVector are aliases for CArray specializations, and their leading :borrowed parameter says the library never owns these buffers. The example uses X \ y from LinearAlgebra.

1. The Julia source

examples/ols/src/ols.jl defines three Base.@ccallable entrypoints. Note that all three of these take arguments with types defined in JLWInterop:

module ols

using JLWInterop
using LinearAlgebra

struct FitResult
    status::JLWStatus
    coeffs::CVector{:borrowed, Float64}
    r_squared::Float64
end

Base.@ccallable function fit(X::CMatrix{:borrowed, Float64},
                              y::CVector{:borrowed, Float64},
                              coeffs_buf::CVector{:borrowed, Float64})::FitResult
    # … shape checks, then `coeffs = X \\ y`; copy into `coeffs_buf`,
    # compute R^2, and return FitResult with JLWStatus(0,…) on success
    # or JLWStatus(code, msg) on a recognized failure.
end

Base.@ccallable function predict(coeffs::CVector{:borrowed, Float64},
                                  X::CMatrix{:borrowed, Float64},
                                  out::CVector{:borrowed, Float64})::JLWStatus

Base.@ccallable function summary_report(result::FitResult,
                                         buf::CString{:borrowed})::JLWStatus

See examples/ols/src/ols.jl for the complete implementation.

coeffs_buf and out are caller-allocated buffers: the library writes into them but does not own them, which is what the :borrowed parameter records. summary_report's CString{:borrowed} buffer works the same way.

Errors travel back as a JLWStatus, either returned directly (predict, summary_report) or embedded in a return struct (fit's FitResult). The Python emitter recognizes both forms and translates a non-zero code into a JLWError exception — see Manual error status handling.

2. The entry Project.toml

A minimal Project.toml for the library:

name = "ols"
uuid = "7e81292c-b63a-42d7-9477-255b6fedc2ed"
version = "0.1.0"

[deps]
JLWInterop = "65e54657-ed21-41a3-96db-71ab7fa6d94b"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[compat]
JLWInterop = "0.2"
julia = "1.13"

Requirements:

• ABI export requires Julia 1.13 or later.

• The [deps] here describe what must be baked into ols.so. Keep it minimal, without build tooling or test dependencies. [sources] entries may use relative paths, such as path = ".." when this project is a subdirectory of the package it wraps. build_library handles juliac's project relocation without modifying the original project.

3. Build the library and Python package

The build uses two environments:

• The entry project (examples/ols/Project.toml, activated by julia --project=.) declares the runtime deps just described.
• A build-env (examples/ols/build-env/Project.toml) declares the build tooling — JuliaLibWrapping and JuliaC. build.jl pushes this directory onto LOAD_PATH so that using JuliaLibWrapping resolves there. Keeping the build tooling out of the entry project's [deps] keeps the library's runtime dependencies minimal.

The build-env's Project.toml is just:

[deps]
JuliaC = "acedd4c2-ced6-4a15-accc-2607eb759ba2"
JuliaLibWrapping = "d61f35a8-f6af-436f-bc10-cee6b101f7bd"

[compat]
JuliaC = "0.3"
JuliaLibWrapping = "0.2"
julia = "1.13"

Run the remaining commands from examples/ols.

Instantiate each environment from your shell:

julia --project=build-env -e 'using Pkg; Pkg.instantiate()'
julia --project -e 'using Pkg; Pkg.instantiate()'

The build.jl script temporarily adds build-env to LOAD_PATH:

push!(LOAD_PATH, joinpath(@__DIR__, "build-env"))
using JuliaLibWrapping, JuliaC

standard_build(@__DIR__; libname = "ols", verbose = true)
pop!(LOAD_PATH)

using JuliaC is what activates JuliaLibWrapping's weak dependency on JuliaC.jl; without it build_library errors with a hint pointing at this line.

standard_build is a convenience wrapper around build_library for the conventional layout — src/<libname>.jl as the entry, out/ as the artifact directory, both a C header and a bundled Python ctypes package named <libname>_py. For layouts outside that convention, or to drop one of the targets, call build_library directly; standard_build's docstring shows the equivalent expansion.

Then run the build from your shell:

julia --project=. build.jl

After a successful build, out/ contains:

out/
├── ols.so              # compiled shared library
├── ols.abi.json        # ABI metadata emitted by juliac
├── ols.h               # C header (CTarget output)
├── pyproject.toml      # Python package metadata
└── ols_py/
    ├── __init__.py
    ├── _lowlevel.py    # generated ctypes bindings (regenerated every build)
    ├── _facade.py      # public API (created once; user-editable)
    └── bundle/         # juliac --bundle tree: libjulia, stdlibs, BLAS, …

bundle = true is essential for a pip install user who has no Julia on their machine. See the bundling section of the distribution guide for what is in the bundle tree and how the loader finds libjulia from inside the wheel.

4. Install the Python package

Create a clean virtualenv with no system Julia, and install. Put the virtualenv outside the entry-project directory (here, /tmp/ols-venv): juliac copies the whole project tree on every build, and a venv nested inside it has symlinks that break that copy.

python -m venv /tmp/ols-venv
source /tmp/ols-venv/bin/activate
pip install numpy
pip install -e ./out

The ./ selects a local path. -e makes later _facade.py edits available without reinstalling.

The bundled libjulia and stdlibs live inside the wheel; the loader in _lowlevel.py searches bundle/lib/ first so the baked-in RUNPATH resolves them at import time, with no LD_LIBRARY_PATH required.

5. Call it from Python

Verify the generated wrapper

First call predict, which is generated automatically because the emitter recognizes all of its argument and return types. The second call uses invalid input to demonstrate error handling:

import numpy as np
from ols_py import predict, JLWError

coeffs = np.array([0.06, 1.98])
X = np.asfortranarray(np.column_stack([np.ones(5), np.arange(1.0, 6.0)]))
out = np.zeros(5)
predict(coeffs, X, out)
# out is now X @ coeffs

try:
    bad = np.asfortranarray(np.zeros((5, 3)))   # wrong number of columns
    predict(coeffs, bad, out)
except JLWError as e:
    print(e.code, e.message)   # 1, "coeffs length must match X cols"

The expected out is array([2.04, 4.02, 6., 7.98, 9.96]), followed by the error message.

np.asfortranarray is required for any CMatrix{:borrowed,T} argument: JLWInterop's CArray is column-major, and the automatically created façade rejects a row-major view rather than silently transposing. You can edit the wrapper to accept a different interface.

Edit the façade

In contrast with predict, fit is not automatically wrapped: it returns a FitResult, and JuliaLibWrapping declines to make choices about what that should look like from the Python perspective. The starter façade re-exports it from _lowlevel with a TODO: hand-wrap comment naming the obstacle. You edit _facade.py to provide the wrapper you want. The generated low-level layer still raises JLWError on a non-zero status, so a scikit-learn-style wrapper is:

# in ols_py/_facade.py, replacing the auto-generated TODO line
def fit(X, y):
    X = np.asfortranarray(X)
    y = np.ascontiguousarray(y, dtype=np.float64)
    coeffs = np.zeros(X.shape[1])
    result = _lowlevel.fit(
        _lowlevel.CMatrix_borrowed_Float64.from_numpy(X),
        _lowlevel.CVector_borrowed_Float64.from_numpy(y),
        _lowlevel.CVector_borrowed_Float64.from_numpy(coeffs),
    )
    return coeffs, result

Note that fit accepts a plain (row-major) X and converts it with np.asfortranarray internally, so callers do not need the conversion required by predict. Alongside the coefficients, this wrapper returns the raw result struct, which carries r_squared and is what summary_report (next) consumes. A production wrapper might instead bundle these into a small result object exposing coeffs, r_squared, and a .summary() method; this returns the pieces directly to keep the example short.

summary_report is also a hand-wrap case (its FitResult argument is an unrecognized struct). The caller allocates a writable CString{:borrowed} buffer, passes it in, and decodes the bytes after the call:

def summary_report(result_struct, capacity=256):
    import ctypes
    buf_bytes = (ctypes.c_uint8 * capacity)()
    buf = _lowlevel.CString_borrowed(
        length=capacity,
        data=ctypes.cast(buf_bytes, ctypes.POINTER(ctypes.c_uint8)),
    )
    _lowlevel.summary_report(result_struct, buf)
    return bytes(buf_bytes).rstrip(b"\x00").decode("utf-8")

Trying out fit and summary_report

With both wrappers in place, the two compose directly. fit is the inverse direction of predict: hand it a design matrix and observed responses, get back the coefficients. To make the result easy to check, we feed it the predictions from the predict call above as the observations — a perfectly linear y, so fit should recover the same [0.06, 1.98] with an R² of 1.0. Restart Python first so the _facade.py edits are picked up (the editable install means no reinstall is needed):

import numpy as np
from ols_py import fit, summary_report

X = np.column_stack([np.ones(5), np.arange(1.0, 6.0)])   # row-major is fine here
y = np.array([2.04, 4.02, 6.0, 7.98, 9.96])              # the predictions from §5, now treated as data

coeffs, result = fit(X, y)
print(coeffs)                   # ≈ [0.06, 1.98] — the coefficients predict() used
print(result.r_squared)         # 1.0 (the points lie exactly on a line)
print(summary_report(result))   # "OLS fit: 2 coefficients, R^2 = 1.0"

The result returned by fit is the FitResult struct expected by summary_report. Try changing one entry of y and re-running: the coefficients shift slightly and result.r_squared drops below 1.0, which the summary string reflects.

6. Adding an entrypoint later

When you add a new Base.@ccallable to ols.jl:

• _lowlevel.py is regenerated on every write_wrapper / build_library call — your new entrypoint shows up automatically.
• _facade.py is written once and then never touched. To pick up new entrypoints in the starter façade, delete the file and rebuild; JuliaLibWrapping will regenerate it (auto-wrapping where it can, leaving # TODO: hand-wrap markers where it cannot).
• __init__.py is regenerated to re-export from _facade.

You trigger all of this by re-running the same build:

julia --project=. build.jl

_lowlevel.py, pyproject.toml, and __init__.py are rewritten in place. Because the package was installed with pip install -e, restart Python to pick up those changes without reinstalling.

Keep _facade.py under version control alongside the build script. To generate wrappers for new functions, delete it on a branch, rebuild, and merge the relevant generated functions into the existing file.