Movie Recommendation Engine using Collaborative Filtering (MovieLens 100K Dataset)

Every streaming platform lives or dies by its recommendations, yet most student "movie recommender" projects are popularity lists dressed up as AI — sort by average rating and call it a day. A real recommendation engine is a rating-prediction problem on a sparse user–item matrix, and it deserves to be built and evaluated like one. This project implements genuine user-based collaborative filtering on the MovieLens 100K benchmark: cosine similarity over mean-centered ratings, neighbour-weighted prediction, and a head-to-head comparison with SVD matrix factorization, all scored with RMSE and MAE on a held-out test split. A live web demo makes the algorithm tangible — rate a handful of movies and watch the engine find users like you and predict your unseen ratings, with the exact formula shown alongside. Because the dataset, the similarity computation and the evaluation protocol are all standard and documented, the student can defend every design choice in the viva instead of waving at a black box.

1. The MovieLens 100K files are loaded: 100,000 ratings (1–5) from 943 users on 1,682 movies, plus movie titles, release dates and 19 genre flags. Ratings are split 80/20 into train and test in the classic u1.base/u1.test style.
2. Training (collaborative filtering): each user's mean rating is computed, ratings are mean-centered, and user–user cosine similarity is calculated over co-rated movies only. Training (SVD): the Surprise library factorizes the train matrix into latent user/item factors.
3. Evaluation: both models predict the held-out test ratings. RMSE and MAE are computed on the standard 5-fold split; the design targets are RMSE ≈ 0.95 and MAE ≈ 0.75 (development targets, not measured claims).
4. Inference for a target user: the 25 most similar neighbours are found, their mean-centered ratings of each unseen movie are similarity-weighted, the user's own mean is added back, and the result is clipped to 1–5 and ranked into a top-N list.
5. In the web demo, the visitor's star ratings become a new row in the matrix; the identical user-based CF algorithm runs in JavaScript over a 12,030-rating sample, showing predicted scores, how many similar users voted, and the three nearest neighbours with their similarity values.
6. The model does not learn in real time: similarities are fixed at training time. New ratings are incorporated on the next training run — this is stated explicitly in the demo and the report.

Tech stack:

• Python 3
• Surprise (scikit-surprise) for SVD
• pandas · NumPy
• Jupyter Notebook (training & evaluation)
• JavaScript (in-browser CF engine)
• HTML5 · CSS (CineMatch demo UI)
• MovieLens 100K dataset
• Git

Dataset & model details

• Dataset: MovieLens 100K — GroupLens Research, University of Minnesota (grouplens.org/datasets/movielens/100k/). 100,000 ratings on a 1–5 scale from 943 users on 1,682 movies; every user rated at least 20 movies; collected 19 Sep 1997 – 22 Apr 1998. The movie file carries title, release date and 19 genre flags (Action, Adventure, Animation, Children's, Comedy, Crime, Documentary, Drama, Fantasy, Film-Noir, Horror, Musical, Mystery, Romance, Sci-Fi, Thriller, War, Western); the user file carries age, gender, occupation and zip code (not used as model features).
• Task: Rating prediction — input = sparse user–item rating matrix (6.3% dense); output = predicted 1–5 rating for each unseen movie, ranked into top-N recommendation lists.
• Model: User-based collaborative filtering (mean-centered cosine similarity on co-rated items, 25 nearest neighbours, 0.1 similarity floor, neighbour-weighted prediction) compared against SVD matrix factorization via the Surprise library.
• Metrics: RMSE and MAE on a held-out test split (standard 5-fold protocol). Design targets: RMSE ≈ 0.95, MAE ≈ 0.75. These are development targets for the built-to-order training run — no accuracy is claimed as measured until the training run is executed for the order.

• [User-based collaborative filtering core] Mean-centered cosine similarity over co-rated movies, 25 nearest neighbours, similarity floor of 0.1, weighted prediction clipped to the 1–5 scale — the real algorithm, not a popularity sort.
• [SVD matrix-factorization baseline] A second model trained with the Surprise library decomposes the rating matrix into latent factors; RMSE/MAE are compared head-to-head so the viva can discuss both approaches.
• [Live rate-and-recommend web demo] Rate 5+ movies in the browser and receive a top-10 list with predicted scores, voter counts and your three most similar users — the identical CF algorithm runs in JavaScript on a 12,030-rating sample.
• [Full training & evaluation notebook] Dataset loading, 80/20 train/test split in the classic u1.base/u1.test style, per-model RMSE and MAE, error analysis and top-N list generation in one reproducible Jupyter notebook.
• [Evaluation dashboard] RMSE/MAE cards, the rating-prediction formula with worked explanation, genre-distribution chart drawn from the real dataset flags, and sparsity analysis.
• [Genre-aware catalog browser] Search and filter the 150 most-rated movies by title or genre inside the demo, each with its real genre flags from the dataset.
• [Honest limitation documentation] Cold start, 1990s catalog, sparsity and offline-training behavior are stated plainly in the demo and the report — strong viva material.
• [Top-N serving function] A clean predict-and-rank function that takes any user profile and returns ranked recommendations with predicted ratings.

• Complete training & evaluation Jupyter notebook (CF + SVD, RMSE/MAE, top-N generation)
• Trained model artifacts (similarity data + Surprise SVD dump) with the serving function
• CineMatch live web demo (single HTML file, works offline)
• Evaluation plots and genre-distribution analysis for the report
• Project report PDF (background, CF theory, methodology, evaluation, limitations)
• PPT presentation for final review
• Viva Q&A preparation document (collaborative filtering, similarity metrics, SVD, RMSE/MAE, cold start, sparsity)