How Smart Home Hubs Recognise Faces and Give Every Person Their Own Screen

Apple's HomeView smart-home hub went on sale on 13 November 2026 with a 7-inch touchscreen and face recognition for up to six household members, each getting their own Home Screen. This guide explains the engineering underneath: the four-step face-recognition pipeline, how per-user screens switch and fall back, how the hub controls lights and locks over MQTT and relays, why the student version uses offline voice commands, and the privacy limits that shape the design.

Written by Projectech18 min readPublished
For Electronics, IoT and embedded-systems students who want to understand how a smart-home hub recognises faces, switches per-user screens and controls devices. Topics: Face recognition, esp-who, ESP32-S3, OV2640 camera, capacitive touchscreen, MQTT, relay control, offline voice commands, PIR motion sensing, per-user profiles
Technical illustration of a wall-mounted smart home hub with a touchscreen showing room tiles, a camera on top, and wireless links to a ceiling light, fan and door lock, with a family standing in front of it.
Illustration generated for this guide.
In this guide

How Smart Home Hubs Recognise Faces and Give Every Person Their Own Screen

Apple put a new device on sale today, 13 November 2026: HomeView, a smart-home hub built around a 7-inch touchscreen, priced at $299, launching in India, the US, the UK, Australia, Canada, New Zealand, Singapore and South Africa. It is a speaker with a screen that the whole household shares — and that is exactly what makes it interesting engineering. A phone knows who is holding it because one person carries it. A hub sits on a wall and faces everyone, so it has to work out who is standing in front of it before it can show them anything personal. HomeView does that with face recognition: it recognises up to six people in the household and shows each of them their own Home Screen, with their calendar, their music and their devices. This guide explains the engineering underneath: how a hub recognises a face, how per-user screens actually work, how the hub talks to the lights and locks around the house, and how a student builds an honest lab-scale version of the same idea. The student-scale version of this idea is the Apple HomeView-Inspired Smart Home Project with Face Recognition Hub and Per-User Touchscreen Dashboard.

Why a shared screen needs to know who is looking at it

The first generation of smart-home control was voice: a speaker on a shelf that anyone could talk to. It worked, but it had a ceiling. A speaker cannot show a calendar, a camera feed or a room full of device tiles, and it cannot tell who is asking — so it cannot show personal information safely. The answer the industry has converged on is a shared screen: a touchscreen that lives in the room, shows the state of the house at a glance, and lets anyone tap a tile to switch a light or check a camera.

A shared screen creates a new problem that a phone never has. On a phone, identity is settled by the lock screen: one person unlocks it, and everything behind it belongs to them. On a hub, the screen is always on and always visible, and the people walking past it change every few minutes. If the hub showed one person's calendar and messages to whoever walked past, it would leak personal information to the whole household. If it showed nothing personal at all, it would be no more useful than a light switch panel. The design answer is identity-aware display: the hub identifies the person in front of it, then shows that person's screen — and falls back to a shared household screen when it does not recognise anyone.

That is the engineering problem this guide is about. It has four parts: recognising a face reliably enough to switch a screen, keeping a separate screen per person, talking to the devices in the house, and doing all of it without turning the living room into a surveillance camera.

How hub face recognition actually works

Face recognition on a hub is not one step; it is a pipeline of four, and each step has its own failure mode. Understanding the pipeline is what separates a student who can build it from one who just calls a library.

Detection. The camera captures a frame and the system finds the faces in it: where each face is, as a rectangle. On a microcontroller this is usually a small neural network trained to spot face-shaped patterns, and it runs on every frame or every few frames. Detection answers "is there a face, and where?" — nothing more. It fails in poor light, at steep angles, and when the face is partly covered. A hub mounted at eye level in a lit room is an easy case for detection; a hub in a dark hallway is not.

Alignment. The detected face is cropped and warped into a standard position — eyes level, face centred, same size — using landmarks such as the eye and nose positions. This step exists because the next step is sensitive to pose: the same person looks different to a matcher when their head is tilted or turned, and alignment removes most of that variation before matching starts. Alignment is also why enrolment photos matter: a system enrolled with front-facing photos recognises front-facing people best.

Embedding. The aligned face is passed through a neural network that outputs a short list of numbers — typically a few hundred — called an embedding or feature vector. The embedding is the system's compact description of that face: two photos of the same person produce embeddings that are close together, and photos of different people produce embeddings that are far apart. The hub never needs to keep the photos after this step; the embedding is what it compares. This is the step where the choice of model matters most, because a better model produces embeddings that separate people more cleanly.

Matching. The live embedding is compared against the stored embeddings of enrolled household members, usually by a distance measure such as cosine distance. If the closest stored embedding is near enough — below a threshold the designer picks — the person is recognised; otherwise the face is treated as unknown. The threshold is a trade-off, and it is worth stating plainly: a loose threshold recognises family members quickly but may confuse two similar-looking people, while a strict threshold avoids false matches but may fail to recognise a real member in bad light and leave them staring at the shared screen.

On a student build this pipeline runs on the device, not in the cloud. Espressif's esp-who library provides face detection and recognition for the ESP32 family, and it is the practical route for a lab build: the camera module captures frames, detection finds the face, the recognition stage compares against templates enrolled on the device, and up to a handful of people can be stored. The honest limits are part of the design: recognition works at close range in good light, it needs a short enrolment step per person, and it switches a screen — it is not a security lock and must never be presented as one. The full build is the Apple HomeView-Inspired Smart Home Project with Face Recognition Hub and Per-User Touchscreen Dashboard.

How per-user screens work

Once the hub knows who is standing in front of it, the screen has to change — and the change is a small state machine, not a trick. The hub keeps one profile per enrolled person, plus one shared household profile. Each profile is a list of tiles: the person's rooms and devices, their calendar, their music shortcuts, the weather for their commute. When recognition returns a match, the hub loads that profile and redraws the screen. When the face leaves the frame for a few seconds, or when recognition returns "unknown", the hub falls back to the shared profile: the household's common devices, the time, the weather, nothing personal.

Three design details decide whether this feels good or broken. First, the switch has to be fast — under about a second from face to redrawn screen — because a person standing in front of a hub expects it to react the way a phone reacts to a fingerprint. Second, the switch has to be stable: the screen must not flicker between two profiles when two people stand side by side, so the hub should hold the first recognised profile until that person clearly leaves, rather than re-deciding on every frame. Third, the fallback must be graceful: an unrecognised visitor should see a useful household screen, not an error message or a locked device.

There is also a privacy decision hiding inside the profile design. A per-user screen shows personal information — a calendar with the day's appointments, a music service with the person's playlists — so the hub should only show it while that person is actually present. The moment the camera loses the face, the personal tiles should go away. That is the behaviour HomeView's design implies with its per-person screens, and it is the behaviour a student build should copy: recognition is a presence signal, not a login that stays open.

How the hub talks to the lights, fans and locks

A screen that cannot change anything in the room is a picture frame. The hub's other half is device control, and the student-scale version of it is deliberately simple: the hub switches mains loads through a relay module, and it talks to other devices over Wi-Fi with MQTT, the lightweight publish-subscribe protocol that most DIY home automation uses.

The chain is short. The touchscreen or the voice command produces an intent — "turn on the living-room light". The hub publishes that intent as an MQTT message, for example to a topic such as home/living-room/light/set. A relay node subscribed to that topic closes its relay, the bulb gets mains power, and the node publishes its new state back so the hub's tile updates to "on". Temperature and humidity from a sensor such as a DHT22 arrive the same way, as periodic MQTT messages the hub turns into tiles. The pattern is the same for every device: intent out, state back, tile reflects state.

Two things make this robust enough to demonstrate. First, the hub must show the real state, not the commanded state: if a relay node is offline, the tile should say so instead of pretending the light turned on. Second, mains switching needs physical separation — a proper relay module with isolation, a fused supply, and no exposed mains wiring on a breadboard. A student build that switches a 12 V demo lamp instead of a mains bulb is making a sensible safety choice, and the report should say so rather than hiding it.

The wider industry is moving the same control problem onto standard protocols — Matter for device identity and control, Thread for low-power mesh radio — so that hubs from different makers can control the same devices. A student build does not need that stack: Wi-Fi plus MQTT plus relays demonstrates the same ideas (intent, state, feedback) with parts a college lab can actually buy. The guide on MQTT protocol explained with examples covers the protocol itself in detail.

Voice on a hub: local commands versus a cloud assistant

HomeView is built around Siri AI, Apple's conversational assistant, which runs with cloud support and is in beta in the hub's software. A student build cannot copy that, and it should not pretend to. The honest student version is offline voice commands: a microphone module such as the INMP441 listens for a wake word, then matches the next few seconds of speech against a fixed set of commands — "turn on the light", "fan speed two", "show my screen" — using an on-device keyword spotter.

The difference between the two approaches is worth understanding, because it explains why the student version is designed the way it is. A cloud assistant understands free-form language, but it needs an internet connection, sends audio off the device, and depends on a service the student does not control. A local keyword spotter understands only its fixed command list, but it works without internet, keeps audio on the device, and responds in a fraction of a second. For a hub that switches lights in a room, the fixed list covers the real use: the commands are few, they are the same every day, and reliability matters more than conversation.

The build decision that follows is to keep the command list short and visible — ten or so commands, printed on the demo card — and to make the hub say what it heard. A hub that silently ignores an unrecognised phrase looks broken; a hub that shows "I heard 'turn on the light' — did you mean the living-room light?" is demonstrating honestly. The companion guide on how offline multilingual voice assistants work explains the keyword-spotting approach in depth.

The touchscreen itself: what makes a hub feel like a product

A hub lives or dies on its screen, and the screen has three engineering parts students often underestimate. The display is a capacitive touchscreen — a 7-inch, 800 by 480 panel on the student build, driven by an ESP32-S3 board with a touch controller such as the GT911. Capacitive touch is the same technology as a phone screen: it senses the capacitance of a fingertip, so it works through a glass cover and supports multi-touch, but it does not work with gloves and it needs a clean power supply or the touch readings get noisy.

The second part is the user interface code. On a microcontroller the usual approach is a lightweight graphics library that draws tiles, buttons and sliders directly to the display, with one screen per profile and a shared home screen as the fallback. The demo dashboard the student build screenshots is the same idea implemented in a browser: a single HTML page that mimics the hub screen, with per-user tiles and working toggles, so the review can see the interface even before the hardware is in hand.

The third part is the one students forget: the screen should sleep. A hub that glows all night wastes power and annoys the household. A PIR motion sensor wakes the display when someone approaches and lets it dim after a minute of stillness — a small addition that makes the difference between a prototype and something that could live on a wall. It also saves the face-recognition pipeline work, since the camera only needs to run while someone is actually standing there.

Privacy, honestly

A hub with a camera in a shared room raises a real question, and a good student project answers it in the design rather than in a disclaimer. Three decisions cover most of it. First, recognition runs on the device: face templates are stored on the hub, the camera frames are processed locally and discarded, and no video leaves the room. Second, the hub stores embeddings, not photos: the template is a list of numbers that is useless without the matching model, which is a much smaller privacy exposure than a folder of face photos. Third, the camera's field of view is the room in front of the hub, so the household should know it is there — a small on-screen indicator while the camera is active is honest design, and it costs nothing to add.

The limits belong in the same conversation. On-device recognition is less accurate than a cloud service with a large model, it struggles in poor light and at an angle, and two similar-looking family members can confuse it — which is why the threshold and the fallback screen matter. A student report that states these limits, and shows the fallback working when recognition fails, is stronger than one that claims the system never makes a mistake.

Building the student version: what to get right

The student build follows the same pipeline as the real product, with student-scale parts at each stage: an ESP32-S3 board with a 7-inch capacitive touchscreen, an OV2640 camera module for face recognition, a relay module for device switching, a DHT22 for room climate, a PIR sensor to wake the screen, and an INMP441 microphone for offline voice commands. The firmware runs the face pipeline on-device, switches profiles on recognition, and drives the relays over MQTT. The web dashboard mirrors the hub screen so the interface can be shown and tested from a phone.

Three build decisions matter more than the rest. First, enrol faces carefully: several photos per person, in the lighting the hub will actually see, because recognition quality is decided at enrolment time. Second, make the fallback visible in the demo: show the shared screen appearing for an unknown face, because a system that fails gracefully is more convincing than one that only works for its builder. Third, design the demo around the three things the hub actually does — recognise a person and switch their screen, switch a real load through the relay, and take a voice command — and show all three working in one continuous run. The parts list, firmware behaviour, setup guide and review kit are in the Apple HomeView-Inspired Smart Home Project with Face Recognition Hub and Per-User Touchscreen Dashboard.

Frequently asked questions

What is Apple HomeView?
HomeView is Apple's smart-home hub, put on sale on 13 November 2026 at $299. It is a 7-inch touchscreen device, available in tabletop and wall-mounted versions, that controls smart-home devices, shows calendars and weather, makes video calls, and recognises up to six household members by face so each of them gets a personalised Home Screen. It launched in India, the US, the UK, Australia, Canada, New Zealand, Singapore and South Africa, but not in the European Union, where its Siri AI assistant is not yet available.

How does a smart home hub recognise a face?
In four steps: detection finds the face in the camera frame, alignment crops and straightens it, a neural network turns it into a short list of numbers called an embedding, and matching compares that embedding against the stored embeddings of enrolled household members. If the closest match is near enough, the person is recognised and their screen loads; otherwise the hub shows the shared household screen. On a student build the same pipeline runs on-device with Espressif's esp-who library.

How do per-user screens work on one shared device?
The hub keeps one profile per enrolled person plus one shared household profile. Each profile is a set of tiles — the person's rooms, devices, calendar and music. Recognising a face loads that profile and redraws the screen; losing the face or seeing an unknown face falls back to the shared profile. The switch must be fast, stable when two people stand together, and it must hide personal tiles the moment the person leaves.

Can a student build a HomeView-style hub?
Yes, at lab scale. The student version uses an ESP32-S3 board with a 7-inch capacitive touchscreen, an OV2640 camera module with on-device face recognition, a relay module for switching demo loads, and MQTT over Wi-Fi for device control. It recognises up to six enrolled faces and switches per-user screens, takes offline voice commands, and wakes on motion. It is not the Apple product: it has no cloud assistant, its recognition works at close range in good light, and it switches screens rather than locking anything.

How does the hub control lights and fans?
Through relays and MQTT. A tap on a tile or a voice command publishes an intent message, a relay node subscribed to that topic closes its relay to power the load, and the node publishes its new state back so the tile reflects reality. Temperature and humidity arrive the same way, as periodic sensor messages. The hub must show the real state — including when a device is offline — not just the last command it sent.

Is a camera in the living room a privacy problem?
It can be, and the design has to answer it. In the student build, recognition runs entirely on the device, camera frames are processed and discarded, and the stored templates are embeddings — lists of numbers — rather than photos. The hub should also show an indicator while the camera is active, and the camera's job is to switch a screen, not to record. The limits are real too: poor light and steep angles reduce accuracy, and similar-looking family members can be confused, so the fallback shared screen matters.

Limitations

  • Every HomeView figure in this guide — $299 price, 7-inch Liquid Retina display, face recognition for up to six people, tabletop and wall-mounted versions, launch markets, and the Siri AI beta — comes from Apple's announcement and the launch coverage of 12–13 November 2026, not from independent measurement, and Apple has not published a full specification sheet for the device.
  • The guide describes the general engineering of face recognition, per-user screens, MQTT device control and offline voice commands. It does not claim to describe Apple's internal implementation, which Apple has not disclosed.
  • On-device face recognition at student scale works at close range in good light with enrolled faces. It is not a security lock, it can confuse similar-looking people, and it must never be presented as equivalent to a phone's face-unlock security.
  • The student build switches demo loads through relays and controls devices over MQTT on Wi-Fi. It is a lab prototype, not a certified mains device, and mains wiring must be handled with proper isolation or replaced by low-voltage demo loads.
  • Face templates are biometric data. Keep enrolment data on the device, delete training captures after enrolment, run demos only with the consent of everyone in the room, and never enrol someone without their knowledge.
  • The student build's recognition accuracy, screen-switch latency and device count are its own measured values, not Apple's figures. Do not quote HomeView's specifications as the student build's results.

More project guides

More in IoT & Embedded