Feature Engineering Basics: Turning Raw Data into Better Model Performance

Most student machine learning projects hit the same wall: the model trains, the code runs, but accuracy stalls somewhere disappointing — 72%, maybe 78% — and adding more epochs does nothing. Students respond by swapping algorithms: Random Forest becomes XGBoost, XGBoost becomes a neural network. Usually, that's the wrong lever. The raw data you're feeding the model is rarely in the shape a model learns from best, and reshaping it — feature engineering — is where the real gains live.

Feature engineering is the work of turning raw columns into better columns: extracting the hour from a timestamp, splitting a name into title and surname, scaling numbers so one column doesn't drown the others. Practitioners consistently rank it above algorithm choice for tabular problems, and it's one of the most genuinely useful skills you can demonstrate in a project report and a viva: it shows you understood the data, not just the library call.

This guide covers the feature families every student project uses, the exact techniques with small worked examples, the mistakes that silently leak your test data into training, and a workflow you can apply to your own dataset this week.

Why features matter more than the algorithm

A machine learning model doesn't see your data the way you do. It sees a matrix of numbers. Two columns can carry identical information while being wildly unequal in usefulness: a date_of_birth column forces the model to rediscover the calendar, while an age column hands it the answer. A city column with 200 unique values overwhelms a simple encoding; a derived city_population column might capture the real signal.

Concretely: for tabular student datasets — CSVs of measurements, survey rows, sensor logs — switching from logistic regression to a gradient booster typically buys single-digit percentage points once features are fixed, but a careful feature pass can buy far more, because it changes what the model is allowed to learn. That's the argument to make in your report: engineering the representation is engineering the learning problem itself.

Feature type 1: numeric transforms

Numbers are the easiest columns to improve and the easiest to get subtly wrong.

Scaling. Many models (SVMs, k-NN, neural networks, regularised linear models) are distance- or gradient-based, so a salary column in lakhs and an age column in decades lets one column dominate. Standardisation (subtract mean, divide by standard deviation) and min-max scaling (to 0–1) fix this. Tree-based models don't need scaling — handy fact for your viva.

Skew. Columns like income or sensor readings are often right-skewed: most values small, a long tail of large ones. A log transform (log1p, which handles zeros) compresses the tail and often improves linear models. Check skewness with pandas .skew(); above ~1 is worth transforming.

Binning. Age as a continuous number can be less useful than age-group buckets (18–25, 26–40, …), because the relationship with the target may jump at life-stage boundaries. Binning trades resolution for learnability — useful when you have limited data and the exact number adds noise rather than signal.

Ratios and differences. Derived columns frequently beat their parents: revenue_per_employee instead of revenue and headcount; bedrooms_per_room instead of both counts; max − min as a spread feature for sensor windows. Ask: "what comparison would a domain expert compute in their head?" Then compute it as a column.

Feature type 2: categorical handling

One-hot encoding turns each category into a binary column. Simple and safe for low-cardinality columns (gender, region with 5 options). Dangerous for high-cardinality columns: a city column with 200 values becomes 200 sparse columns and a recipe for overfitting.

Target encoding replaces each category with the average target value for that category (with smoothing). Powerful for high-cardinality columns, but it must be computed inside each cross-validation fold — computed on the full dataset before splitting, it leaks target information and your validation score becomes fiction. This is one of the most common leakage sources in student projects.

Ordinal vs nominal. Tree models can consume integer codes directly, but integer codes imply an order — red=1, blue=2, green=3 tells the model blue is "between" red and green, which is nonsense for nominal categories. Keep ordinal codes (small/medium/large → 1/2/3) only where the order is real.

Feature type 3: dates, times, and text

Datetime decomposition. Never feed a raw timestamp to a model. Extract the parts that carry signal: hour of day, day of week, month, quarter, is-weekend, days-since-start. In an energy-consumption project, hour alone can dominate — consumption follows a daily rhythm the model would otherwise have to reconstruct.

Cyclical encoding. Hour 23 and hour 0 are adjacent in reality but far apart as numbers. Encode cyclical features as sine/cosine pairs: sin(2π·hour/24), cos(2π·hour/24). This is a classic viva-answer feature: it fixes a real geometric mismatch.

Text basics. A free-text column (reviews, descriptions) becomes features via length, word count, counts of specific keywords, or TF-IDF vectors. Full text pipelines (embeddings) are a separate topic; for a student project, length + keyword flags + TF-IDF on the top few hundred terms is a solid, explainable baseline.

A safe, repeatable workflow

Feature engineering done wrong silently invalidates your results. Follow this order:

1. Split first. Hold out your test set before touching anything. Every statistic (means, encodings, scalers) is computed on training data only, then applied to test data.
2. Explore distributions. Plot histograms and value counts. You can't engineer what you haven't looked at — most useful features come from noticing something odd in an EDA plot.
3. Build features in one function. Write a make_features(df) function that takes a raw dataframe and returns the engineered one. Apply it identically to train and test. This single habit eliminates the "train/test mismatch" bug that ruins many student evaluations.
4. Use a pipeline. sklearn's Pipeline + ColumnTransformer applies scaling/encoding per column type and guarantees the same transforms run on new data. It also makes cross-validation honest: put the transforms inside the pipeline so each fold re-fits them, or encoding leakage returns.
5. Measure with ablation. Add features in groups and record validation score at each step. If a feature group doesn't move the score, drop it — extra columns cost you nothing at inference time but add overfitting risk and confusion in your report.
6. Select, don't hoard. With hundreds of columns, use feature selection: drop near-zero-variance columns, drop one of each highly correlated pair (|r| > 0.9), and consider model-based importance (permutation importance is more honest than impurity-based). Fewer, stronger features beat more, weaker ones.

Common leakage traps (and how to spot them)

• Fitting a scaler on the full dataset before splitting. Fix: split first, fit on train, transform both.
• Target encoding computed globally. Fix: compute per fold, or use out-of-fold encoding.
• Imputation with the global mean before splitting. Fix: same rule — statistics from train only.
• Features that use future information. A "total revenue" column in a churn-prediction dataset often includes behaviour from after the churn decision point. If a feature seems too good, it probably saw the future.
• Duplicate rows across train/test. Common with time-series resampling. Deduplicate and prefer time-based splits.

Leakage is the number one reason a student project's "95% accuracy" collapses during viva questioning. If you can explain which leakage traps you checked, your evaluation section becomes your strongest section.

FAQs

How is feature engineering different from data cleaning? Cleaning fixes broken data (missing values, duplicates, wrong types). Feature engineering creates new signal from clean data. You do cleaning first, then engineering — engineered features built on dirty data inherit the dirt.

Do deep learning models need feature engineering? Less for images and raw text, where the network learns representations itself. For tabular data — the bulk of student projects — gradient-boosted trees with engineered features still routinely beat deep learning, and feature engineering matters just as much as ever.

How many features should my project have? There is no target number. Start with everything reasonable, then let validation score and selection decide. A common healthy endpoint is 10–50 strong features. If removing a feature doesn't change the score, it's decoration — cut it and say so in your report.

What's the single highest-value technique for a beginner? Datetime decomposition with cyclical encoding, if your data has timestamps. It takes ten lines of code and often moves accuracy more than any model swap.

Can I automate feature engineering? Tools like Featuretools automate "deep feature synthesis" (aggregations across tables). They're worth trying for tabular relational data, but treat them as a baseline generator: automated features plus a few hand-built domain features is the strongest combination, and the hand-built ones are what you defend in the viva.

Limitations

Feature engineering cannot create information that isn't in the data — it can only surface what's already there. It won't fix a fundamentally wrong problem framing, a mislabeled dataset, or a test set drawn from a different distribution than training (a common failure when the "test" CSV comes from a different source). Treat reported gains honestly: report the validation setup (fold count, split strategy, leakage checks) alongside the score, because an engineered-feature gain measured on a leaky split is worse than no gain at all — it's a false claim your examiner can dismantle in one question.