A Few Dollars’ Worth of Brain: ESP32 and Edge AI
我第一次见到 ESP32 是在深圳华强北的一个柜台上,一块比火柴盒还小的板子,标价十几块钱。老板随口说,现在做智能插座、做灯带,十个里有八个用它。那时候没人会把它和”人工智能”放在一句话里。几年过去,情况变了:同样是这颗芯片的后代,已经能听懂唤醒词、认出人脸、判断电机有没有异响,甚至充当大模型的”耳朵和嘴巴”。这篇文章想聊的,就是这件看上去不太合理的事情是怎么发生的。
I first saw an ESP32 at a counter in Shenzhen’s Huaqiangbei market: a board smaller than a matchbox, priced at a dozen yuan or so. The shopkeeper said, almost in passing, that eight out of ten smart plugs and LED strips used it. Nobody would have put it in the same sentence as “artificial intelligence” back then. A few years on, descendants of that same chip can understand wake words, recognize faces, tell whether a motor sounds wrong, and even serve as the ears and mouth of a large language model. This article is about how that unlikely thing came to happen.
先说清楚一个前提:ESP32 不是为 AI 设计的。上海乐鑫科技最初做的是 Wi-Fi 和蓝牙芯片,目标是让各种设备便宜、稳定地连上网。早期的 ESP32 是双核 Xtensa LX6,主频 240MHz,片内 SRAM 不到 520KB,没有任何专门的神经网络硬件。拿它跑模型,等于让一辆送货三轮车去拉力赛。能做的事很有限,比如一个几十 KB 的关键词识别模型,或者一个很小的异常检测程序,而且得处处精打细算。
Let’s be clear about one thing first: the ESP32 was not designed for AI. Espressif, the Shanghai company behind it, started out making Wi-Fi and Bluetooth chips, with the goal of getting all kinds of devices online cheaply and reliably. The early ESP32 was a dual-core Xtensa LX6 running at 240 MHz, with under 520 KB of on-chip SRAM and no dedicated neural-network hardware. Running a model on it was like entering a delivery tricycle in a rally race. What it could do was limited: a keyword-spotting model of a few dozen kilobytes, or a tiny anomaly detector, and every byte had to be counted.
转折出现在 ESP32-S3。这一代换成了 Xtensa LX7,仍然是双核 240MHz,但加入了面向向量运算的扩展指令。神经网络推理的本质是大量的乘加运算,一条指令一次处理多个 int8 数据,速度就和逐个计算完全不是一个量级。乐鑫官方给出的 ESP-NN 库正是围绕这些指令做了优化,卷积、全连接这些常见算子在 S3 上的速度比旧款快出数倍。同时它支持外挂 PSRAM,容量可以到 8MB 甚至更多,图像类模型终于有地方放中间结果了。
The turning point was the ESP32-S3. This generation moved to the Xtensa LX7, still dual-core at 240 MHz, but added extended instructions for vector math. Neural-network inference is at heart a mountain of multiply-accumulate operations, and an instruction that handles several int8 values at once is in a different league from one that handles them one by one. Espressif’s ESP-NN library is tuned around exactly these instructions, and common operators such as convolution and fully connected layers run several times faster on the S3 than on older parts. It also supports external PSRAM, 8 MB or more, so image models finally have somewhere to keep their intermediate results.
软件这一侧同样重要。TensorFlow Lite Micro 可以直接在 ESP-IDF 上运行;乐鑫自己的 ESP-DL 提供了量化、部署一条龙的工具;ESP-SR 内置了唤醒词引擎 WakeNet 和离线命令词识别 MultiNet,对着设备喊一声”你好,小智”,不用联网就能被唤醒;ESP-WHO 则是一套面向人脸检测和识别的参考方案。对开发者来说,最大的变化是门槛:过去在单片机上做机器学习,要自己啃指令集、手写算子;现在更多时候是在电脑上训练、量化,然后把模型文件烧进去。Edge Impulse 这类平台把流程做得更顺,采集数据、训练、导出、部署,一个下午能走完一遍。
The software side matters just as much. TensorFlow Lite Micro runs directly on ESP-IDF. Espressif’s own ESP-DL offers tooling for quantization and deployment end to end. ESP-SR ships a wake-word engine, WakeNet, and an offline command recognizer, MultiNet, so a device can be woken by a spoken phrase with no network at all. ESP-WHO is a reference solution for face detection and recognition. For developers the biggest change is the barrier to entry. Machine learning on a microcontroller used to mean wrestling with the instruction set and hand-writing operators; now the usual path is to train and quantize on a PC and flash the model file onto the board. Platforms like Edge Impulse smooth the flow further: collect data, train, export, deploy, all in an afternoon.
那么,为什么非要在这么小的芯片上做推理,而不是把数据全部传到云端?我见过的答案大致有三个。第一是延迟。门口的摄像头判断”有没有人”,如果每次都要上传、等待、返回,体验会很糟,本地几十毫秒出结果就是另一回事。第二是隐私。客厅里的声音和画面,用户并不乐意都送到别人的服务器上,唤醒词识别在本地完成,只有真正需要时才联网,这是个很实在的卖点。第三是电量和流量。一个靠电池工作的传感器,每天发射几百次无线信号,电池撑不过一个月;如果在本地先筛掉 99% 的无效数据,只上报异常,续航可以拉长到以年计算。
So why run inference on such a small chip rather than ship all the data to the cloud? I have heard three answers worth taking seriously. The first is latency. If a doorway camera had to upload, wait and download every time it decided whether someone was there, the experience would be dreadful; a local result in a few dozen milliseconds is another matter. The second is privacy. People are not keen on sending the sounds and images of their living room to someone else’s server. Recognizing the wake word locally and going online only when needed is a real selling point. The third is power and bandwidth. A battery-powered sensor that transmits hundreds of times a day will not last a month; if it filters out 99% of useless data locally and reports only anomalies, battery life stretches to years.
工业现场是最典型的例子。一台水泵的振动传感器,每秒产生成千上万个采样点,全传上去没有意义。做法是在 ESP32 上跑一个小型的频谱分析加分类模型,平时只看”正常还是不正常”,一旦轴承开始磨损、频谱里多出一段不该有的峰,才通知后台。农业上也有类似的做法:大棚里的节点根据温湿度和光照的历史变化,在本地预测未来一小时的趋势,提前开关风机。这些模型都不大,往往只有几十到几百 KB,用不着多聪明,要的是便宜、能铺得开。一个厂区要装几百个节点,每个节点贵一百块和贵十块,是完全不同的预算。
Industrial sites are the classic example. A vibration sensor on a water pump produces thousands of samples a second, and uploading them all is pointless. The usual approach is to run a small spectrum-analysis and classification model on the ESP32 that only asks “normal or not”. When a bearing starts to wear and an unwanted peak appears in the spectrum, only then does it alert the back end. Agriculture has similar cases: a node in a greenhouse learns from the history of temperature, humidity and light, predicts the next hour locally, and switches the fans in advance. These models are small, often tens to hundreds of kilobytes, and they don’t need to be clever; they need to be cheap and widely deployable. A factory may need hundreds of nodes, and a hundred yuan versus ten yuan per node is a completely different budget.
这两年最热闹的方向,是把 ESP32 当作大模型的终端。国内一个叫”小智”的开源项目是个典型:一块 ESP32-S3,加麦克风、扬声器和一块小屏幕,就成了一个能聊天的桌面小玩具。它的分工很有意思——本地负责唤醒词检测和音频采集,真正的”思考”由云端大模型完成,结果再以语音流传回设备播放。芯片本身一点也不”聪明”,聪明的是它把人和模型之间那道门开得足够便宜。成本压到几十块,学生和爱好者才玩得起,于是社区里冒出了各种改装:给它换声音、接上机械舵机、做成宠物形状。很多真正有趣的东西,都是这样从廉价硬件里长出来的。
The liveliest direction of the past couple of years is using the ESP32 as a terminal for large language models. An open-source project called Xiaozhi is a good example: an ESP32-S3 plus a microphone, a speaker and a small screen becomes a desktop gadget you can chat with. The division of labor is interesting. The chip handles wake-word detection and audio capture locally, the real “thinking” happens on a cloud model, and the reply streams back as speech for the device to play. The chip itself isn’t smart at all; what’s clever is that it opens the door between people and models cheaply enough. Once the cost is down to a few tens of yuan, students and hobbyists can afford to play, and the community has produced all sorts of mods: swapped voices, servo-driven limbs, pet-shaped shells. Many of the most interesting things grow out of cheap hardware this way.
当然,别把它想得太神。ESP32 的限制是实打实的。内存是最大的瓶颈:即便外挂了 PSRAM,带宽也远不及手机芯片,稍大一点的视觉模型,帧率会掉到个位数。精度同样要付出代价,模型必须量化到 int8 甚至更低,部分任务的准确率会明显下降。还有功耗:全速跑推理时电流以几十到上百毫安计,只有靠”平时深度睡眠、需要时才唤醒”的策略,才能省电,深度睡眠的电流只有几微安。所以真正的工程经验,往往是反过来想:不是”这个模型能不能塞进去”,而是”这个任务最少需要多大的模型”。
Of course, don’t mythologize it. The ESP32’s limits are very real. Memory is the biggest bottleneck: even with PSRAM, bandwidth is far below that of a phone chip, and a somewhat larger vision model will drop to single-digit frame rates. Accuracy pays a price too; models must be quantized to int8 or lower, and some tasks lose noticeable precision. Then there is power: running inference at full speed draws tens to over a hundred milliamps, so saving energy depends on staying in deep sleep, which draws only a few microamps, and waking up when needed. Real engineering experience therefore tends to run the other way around: the question is not “can this model fit?” but “what is the smallest model this task actually needs?”
乐鑫自己显然也看到了这一点。后续的 ESP32-P4 换成了双核 RISC-V,主频提升到 400MHz,并且加入了面向 AI 的指令扩展和更大的片内存储,还带有摄像头接口和图形加速,定位就是更重的边缘视觉与人机交互场景。它本身不带无线,需要搭配其他芯片联网,这个取舍很能说明思路:把”连接”和”算力”拆开,各自按需堆料。另一边,ESP32-C 系列用的是 RISC-V 内核,价格压得更低,适合只需要极轻量智能的场景。一条产品线,既往上够得着,也往下压得住。
Espressif has clearly seen all this. The later ESP32-P4 moves to a dual-core RISC-V design at 400 MHz, adds AI-oriented instruction extensions and more on-chip memory, and includes a camera interface and graphics acceleration, aimed at heavier edge vision and human-machine interaction. It has no built-in wireless and must be paired with another chip to get online, a trade-off that reveals the thinking: separate connectivity from compute and load each as needed. On the other side, the ESP32-C series uses RISC-V cores at even lower prices, suited to scenarios needing only the lightest intelligence. One product family can reach upward and still hold the line at the bottom.
如果要我给想入门的人一点建议,只有三条。第一,先别急着追最新的模型,从一个具体的小问题开始,比如识别三个手势,或者听出水龙头有没有漏水。第二,先量数据再选芯片,很多时候传感器的质量比模型结构更重要。第三,学会看内存和耗时,每次部署都记录峰值内存、推理时间和电流,这些数字比任何宣传都诚实。板子几十块,试错的成本低得不能再低,唯一贵的是时间。
If I had to give newcomers some advice, there would be three points. First, don’t chase the newest model; start with one small, concrete problem, such as recognizing three hand gestures or hearing whether a tap is leaking. Second, measure your data before choosing a chip; sensor quality often matters more than model architecture. Third, learn to read memory and timing: record peak memory, inference time and current draw with every deployment, because those numbers are more honest than any marketing. The board costs a few tens of yuan, so the cost of trial and error is as low as it gets. The only expensive thing is your time.
回头看,ESP32 的故事并不是”小芯片逆袭”那种热血情节,更像是一次缓慢的积累:连接做到极致便宜,再顺手把算力补上,等社区的想象力赶到,AI 就自然地漫进了这些角落。未来的智能,可能不只是数据中心里的庞然大物,也会是墙角里那块不起眼的小板子,几微安的电流,安静地等一句话,或者一个异常的声音。
Looking back, the ESP32’s story isn’t a stirring tale of a small chip’s triumph. It’s more a slow accumulation: make connectivity extremely cheap, add compute along the way, and when the community’s imagination catches up, AI seeps into these corners almost naturally. Future intelligence may not only be the giants in data centers. It may also be that unremarkable little board in the corner of a room, drawing a few microamps, quietly waiting for a word or an odd sound.