Running intelligence close to the user changes the feel of a product. Responses can arrive without a network round trip, private context can stay on the device, and essential features can continue to work when connectivity is poor.
The constraint is obvious: phones, glasses, cars, and small computers have tighter limits on memory, energy, and heat. That pressure is pushing better compression, specialized chips, and models designed for a narrow job instead of every job.
This is unlikely to become a winner-take-all contest between local and cloud AI. The more useful pattern is routing: do the fast and sensitive work locally, then call larger systems when the task needs broader knowledge or more compute.
For product teams, the design question becomes architectural. Which context truly needs to leave the device, and which moments are important enough to justify the latency and cost of the cloud?