Market Basket Analysis using Apriori Algorithm (Online Retail Dataset)

Market basket analysis is the classic unsupervised-learning problem behind "customers who bought this also bought" — and Apriori is the algorithm every data-mining course teaches but few students ever run on real data. Textbook examples use ten toy transactions; this project uses the real thing: the UCI Online Retail data set from a UK-based gift retailer, cleaned and sampled to 4,000 genuine transactions across 3,637 products. The centerpiece is a single-file web demo that implements Apriori faithfully — candidate generation, subset pruning, tid-list support counting — and mines the embedded data live in the browser in about half a second. Students can move the minimum-support and minimum-confidence sliders and watch the rule set change, which turns abstract threshold theory into something tangible. A basket-builder view then puts the rules to work as a recommender, and a data-insights view profiles the sample with top-product and basket-size charts. Everything is built with standard tools (Python for data preparation, plain JavaScript for the mining engine), so the student can explain every stage in the viva.

1. The UCI Online Retail rows are cleaned in Python: cancelled invoices dropped, zero/negative quantities removed, blank descriptions removed, and invoice numbers grouped into baskets of 2+ items.
2. A reproducible 4,000-transaction sample (seed 42) is drawn from the 18,338 clean baskets and embedded in the demo as compact integer-ID arrays.
3. In the browser, tid-lists (sorted transaction-ID lists) are built per product; L1 keeps the 914 products meeting the 1% minimum support.
4. Candidates are generated level by level: frequent (k−1)-itemsets sharing a prefix are joined, and any candidate with an infrequent (k−1)-subset is pruned without counting.
5. Support is counted by intersecting tid-lists with early exit; surviving itemsets form the next level until no candidates remain (2,972 frequent itemsets in the reference run).
6. Rules are generated from every frequent itemset of size 2+: each antecedent/consequent split is tested against the 50% confidence threshold and lift is computed (1,940 rules in the reference run).
7. The explorer shows the top 100 rules sortable by lift, confidence or support; the basket builder scores every candidate product against the basket using live pairwise confidence and lift.

Tech stack:

• Python 3, pandas, openpyxl (data cleaning & sampling script)
• Plain JavaScript (Apriori engine — no ML libraries, the algorithm is implemented by hand)
• Single-file HTML/CSS web demo (rule explorer, basket builder, data insights)
• UCI Online Retail data set (UK gift retailer, Dec 2009 – Dec 2011)

Dataset & model details

• Dataset: UCI Machine Learning Repository "Online Retail" — transactions of a UK-based online gift retailer, December 2009 to December 2011. Cleaned to 18,338 multi-item baskets; the demo embeds a reproducible 4,000-transaction sample (seed 42) covering 3,637 unique products, average 28.8 items per basket.
• Task: Association rule mining (unsupervised); input = transaction itemsets, output = frequent itemsets plus association rules of the form {antecedent} → {consequent} with support, confidence and lift.
• Model: Apriori — level-wise candidate generation with the Apriori pruning property (all subsets of a frequent itemset are frequent), support counted via tid-list intersection; rules generated from all frequent itemsets of size ≥ 2.
• Metrics (reference run, computed — not claimed): At minimum support 1% and minimum confidence 50%, the run finds 2,972 frequent itemsets and 1,940 association rules in ~0.4 s in the browser. Strongest rule: {HERB MARKER BASIL + HERB MARKER ROSEMARY} → {HERB MARKER THYME}, support 1.10%, confidence 97.8%, lift 78.2×. Association mining has no accuracy metric; interestingness is measured by support, confidence and lift.

• [Real Apriori engine in JavaScript] Faithful implementation — level-wise candidate generation, Apriori subset pruning, tid-list intersection with early exit — not a mockup; it mines 4,000 real transactions in about 0.4 seconds in the browser.
• [Rule explorer with live thresholds] Sliders for minimum support (0.5–5%) and minimum confidence (30–90%); re-running re-mines the data instantly and every number in the table is computed live.
• [Support, confidence and lift on every rule] The three canonical interestingness measures, shown per rule and explained in plain language in the demo and the report.
• [Basket builder with live recommender] Search all 3,637 real products, build a basket, and get ranked "frequently bought together" suggestions with live pairwise support, confidence and lift.
• [Data-insights dashboard] Top-10 products bar chart, basket-size distribution histogram, and key sample statistics — all drawn from the embedded real data at page load.
• [Real UCI Online Retail data] A 4,000-transaction cleaned sample embedded in the demo; the full cleaning and sampling Python script ships with the project for reproducibility.
• [Documented methodology] The report walks through the join and prune steps, defines the three measures with worked examples from the actual rules found, and justifies the threshold choices.

• Data cleaning & sampling Python script (UCI xlsx → cleaned baskets → embedded sample)
• Single-file web demo with the hand-implemented JavaScript Apriori engine
• Rule explorer (tunable thresholds), basket builder with recommender, data-insights dashboard
• Project report PDF (background, Apriori theory, methodology, reference-run results with worked rule examples)
• Abstract PDF (10-section)
• PPT presentation for final review
• Viva Q&A preparation document (Apriori property, support vs confidence vs lift, threshold effects, pruning)

• The demo mines a 4,000-transaction sample of a 500k+-row data set — the rules describe the sample, and re-sampling shifts the long tail of rules.
• Rule sets are threshold-dependent: raising minimum support shrinks the rule set and dropping it explodes it; there is no single "correct" set of rules.
• Lift can look dramatic for rare co-occurrences; the minimum-support floor exists precisely to keep those in check.
• Products are gift-retailer SKUs (decor, stationery, kitchenware) — the discovered patterns do not transfer to grocery or other domains.
• Association is co-occurrence, not causation: a rule says items are bought together, not that one causes the other.
• The browser demo suits thousands of transactions; mining millions of rows belongs in the Python script, not the page.
• No accuracy-style metric exists for association mining — the honest evaluation is support, confidence and lift, all reported from the actual run.