A new version of ESP-KVM 0.65.0 has been released, an open firmware that turns the board on the ESP32-P4 microcontroller into an IP-KVM with access to the screen, keyboard and mouse of a remote computer via a browser. The device is connected in front of the HDMI of the target machine and pretends to be a USB keyboard and mouse for it, and sends the image and control to the browser. There is no need to install anything on the target machine; it even works in the BIOS. The code is written in C language and distributed under the Apache 2.0 license.
At the same time, published a separate experimental project – ESP-KVM terminal, which develops firmware for the M5Stack Tab5 tablet, which uses the same ESP32-P4 SoC as KVM. The terminal connects to the ESP-KVM over the network like a regular viewer, only instead of a browser tab it has a five-inch 1280×720 screen.
MJPEG frames are decompressed by a hardware JPEG decoder, and the PPA block scales them and rotates them directly into the display buffer. Up to 20 frames per second comes out if the KVM manages to send them over WiFi. The ESP32-P4 does not have a hardware H.264 decoder, so MJPEG must be enabled on KVM. The terminal finds KVM on the network itself. The pointer is controlled by the touch screen, and text can be entered using the Tab5 keyboard or a regular USB keyboard.
The ESP-KVM 0.65.0 release is almost entirely focused on optimizing performance over slow links. Over WiFi, the device sent frames faster than the Tab5 could receive them. The queue grew and the screen lagged behind for a couple of seconds. Now the display device acknowledges each frame, and the sending device keeps no more than two frames in transit. Without such synchronization, Tab5 on the same network showed about 3 frames per second, and with confirmations – 8-9. ESP-KVM first reports that it understands confirmations, and only then the client sends them.
Through Tailscale, the video was even slower: 1-3 frames per second between two ESP32-P4s on the same WiFi network. The entire tunnel goes through one UDP socket, and its receive queue accommodated 6 packets, when the video packet was about 45. A send that was interrupted by a timeout left half a WebSocket frame in the connection, and the viewer lost the stream markings. Nagle’s algorithm delayed each short message for 200-450 ms, which is why the ping between the terminal and KVM after TCP_NODELAY dropped from 440 to 15 ms. After one side restarted, the nodes could not agree again because a new handshake attempt would overwrite the previous one before it received a response. With each transition between the direct path and the DERP relay, the handshake started anew, approximately once every 15 seconds, although the session keys do not care about the path. After the edits, it turned out to be 11-14 fps at 720p, almost like on a local network.
Briefly about the changes in ESP-KVM after the last news about the release of ESP-KVM 0.51.0: