Linear Regression in C

A self-contained linear regression implementation in pure C, with training, prediction, file persistence, and custom matrix/vector utilities — no external libraries required.

Overview

This project fits an ordinary least squares linear model using the closed-form normal equation. Models support an optional intercept, can be saved to and loaded from disk, and are built entirely on hand-written vector and matrix routines. It is intended as a compact, readable reference implementation.

Project structure

.
├── LinearRegression.h / .c   # Model interface: train, predict, save, load
├── matrix.h / matrix.c       # Matrix operations, including transpose and inversion
├── vector.h / vector.c       # Vector operations used throughout the model
├── tester.c                  # Example program that trains, predicts, saves, and reloads
├── tester.py                 # Equivalent scikit-learn script for comparison
└── benchmark.png             # Timing comparison figure

Features

• Train a linear regression model on one or more features.
• Optional intercept term.
• Predict outputs for new inputs from the trained weights.
• Serialize a trained model to disk and load it back.
• Custom vector and matrix algebra, with no third-party dependencies.

API reference

LinearRegression (LinearRegression.h)

Function	Description
LinearRegression *train_model(Feature *feats, Output *output, long feat_count, bool has_intercept)	Fit a model from features and the output variable.
long double run_model(LinearRegression *model, data_row input)	Predict the output for a new input row.
void save_model(LinearRegression *model, char *path)	Write the model weights and metadata to a file.
LinearRegression *load_model(char *path)	Read a serialized model back from a file.
void free_model(LinearRegression *model)	Release all memory held by the model.

A Feature pairs a name with a Vector of data points; an Output is a Feature representing the target variable.

Vector (vector.h)

Function	Description
Vector init_vec(long double *data, long size)	Initialize a vector from existing data.
Vector empty_vec(long size)	Create a zero-initialized vector of the given size.
void free_vec(Vector *v)	Free memory held by a vector.
long double dot(Vector *v1, Vector *v2)	Compute the dot product of two vectors.

Matrix (matrix.h)

Function	Description
Matrix init_matr(Vector *rows, long row_num)	Initialize a matrix from an array of row vectors.
Matrix empty_matr(long row_num, long col_num)	Create a zero-initialized matrix of the given dimensions.
void free_matr(Matrix *m)	Free memory held by a matrix.
long double *get_m_pos(Matrix *m, long row, long col)	Get a pointer to an element for in-place modification.
long double get_m_val(Matrix *m, long row, long col)	Read the value at a position.
Vector *get_row(Matrix *m, long row_num)	Get a pointer to a row vector.
Vector *get_col(Matrix *m, long col_num)	Get a column as a vector.
Matrix mul_matr(Matrix *m1, Matrix *m2)	Multiply two matrices (m1 × m2).
Matrix inv_matr(Matrix *m)	Invert a square matrix.
Matrix t_matrix(Matrix *m)	Transpose a matrix.

Compilation

Compile the tester with all implementation files included:

gcc tester.c LinearRegression.c matrix.c vector.c -o tester
./tester

Keep the headers and source files in the same directory, or adjust the include paths accordingly.

How it works

train_model()

Builds the design matrix X from the features (prepending a column of ones when an intercept is requested) and solves for the weights with the normal equation:

w = (XᵀX)⁻¹ Xᵀy

run_model()

Computes a prediction as the dot product of the weights and the input row (adding the intercept term when present):

ŷ = wᵀ · x

save_model() and load_model()

Persist a trained model by writing the weight count, intercept flag, and each weight together with its name to a binary file, and read them back in the same format.

Memory management

• Call free_model() when you are done with a model.
• Free any vectors you create with empty_vec() or init_vec().
• Free any matrices you allocate if you extend the project.

Example usage

From tester.c:

int n = 5; // number of data points

// Zero-initialized vectors for the features and the target.
Vector x = empty_vec(n), z = empty_vec(n), y = empty_vec(n);

int primes[5] = {2, 3, 5, 7, 11};

for (int i = 0; i < n; i++) {
    x.data[i] = i + 1;                                // x: 1, 2, 3, 4, 5
    z.data[i] = primes[i];                            // z: 2, 3, 5, 7, 11
    y.data[i] = 1 + 2 * x.data[i] + 0.5 * z.data[i];  // linear target
}

// Feature array built from vectors x and z.
Feature feats[2] = { { "x", x }, { "z", z } };
Output output = { "y", y };

// Train with an intercept.
LinearRegression *model = train_model(feats, &output, 2, true);

// New input: x = 6, z = 13 (expected 1 + 2*6 + 0.5*13 = 19.5).
long double input[] = { 6.0, 13.0 };

save_model(model, "Test");
printf("Prediction before save: %Lf\n", run_model(model, input));
free_model(model);

// Reload and predict again.
model = load_model("Test");
printf("Prediction after load: %Lf\n", run_model(model, input));
free_model(model);

free_vec(&x);
free_vec(&z);
free_vec(&y);

Comparison with scikit-learn

tester.py reproduces the same fit-predict-save-load flow with scikit-learn, which is useful for cross-checking the C results. benchmark.png shows a timing comparison between the two implementations on this small example.

Future improvements

• Regularization (for example, ridge regression).
• Gradient descent as an alternative to the normal equation.
• Input/output transformations to model non-linear relationships.

License

Released under the MIT License.