vilimblog

HDMI to USB Capture Card Roundup

2026-04-22T00:00:00Z

I'm looking for a cheap USB3 HDMI capture card.

These aren't capture "cards" in the traditional sense, but that's the way I still think about them.

My last captures were done using a Brooktree / Conexant PCI bus card on an ancient Linux desktop, so the silicon miniaturization and cost reduction available for the price of a burger at Five Guys is still a bit mind blowing.

But that card at least blasted full uncompressed data into my system, doing a nice job when I wanted to capture as much lossless detail off of an analog medium as I could.

That said, the USB Video Class capture devices are a really mixed bag, and it's really hard to know which ones are worth anything at all, even if they are cheap as chips.

So in the spirit of my last big blog post on dodgy Chinese hardware (see: USB serial adapter shootout), here's a post about the various USB capture cards I could find on eBay.

I'll try to give these devices nicknames, and as I figure out what chips they actually run, I'll post that here too.

To help people cross-reference these devices with eBay, I'll copy in all of the marketing pictures I can find there to this blog post.

It's really not easy to find brand names on these devices, nobody from Shenzhen will stand behind them, and I am too cheap to get a Magewell or Startech device.

Dongle 0: The One I Already Have

This thing is a pile of garbage and has really poor capture quality.

Here's what is captured with ffmpeg:

$ ffmpeg -f v4l2 -input_format mjpeg -video_size 1920x1080 -i /dev/video5 -c:v copy -an -t 8 /tmp/output.mkv
ffmpeg version n8.1 Copyright (c) 2000-2026 the FFmpeg developers
  built with gcc 15.2.1 (GCC) 20260209
  libavutil      60. 26.100 / 60. 26.100
  libavcodec     62. 28.100 / 62. 28.100
  libavformat    62. 12.100 / 62. 12.100
  libavdevice    62.  3.100 / 62.  3.100
  libavfilter    11. 14.100 / 11. 14.100
  libswscale      9.  5.100 /  9.  5.100
  libswresample   6.  3.100 /  6.  3.100
Input #0, video4linux2,v4l2, from '/dev/video5':
  Duration: N/A, start: 63809.788172, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 1920x1080, 30 fps, 30 tbr, 1000k tbn, start 63809.788172
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Output #0, matroska, to '/tmp/output.mkv':
  Metadata:
    encoder         : Lavf62.12.100
  Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 1920x1080, q=2-31, 30 fps, 30 tbr, 1k tbn
Press [q] to stop, [?] for help
[out#0/matroska @ 0x55ca84505740] video:26080KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.041526%
frame=  239 fps= 33 q=-1.0 Lsize=   26091KiB time=00:00:08.04 bitrate=26579.0kbits/s speed=1.12x elapsed=0:00:07.15

$ ffprobe /tmp/output.mkv
ffprobe version n8.1 Copyright (c) 2007-2026 the FFmpeg developers
  built with gcc 15.2.1 (GCC) 20260209
  libavutil      60. 26.100 / 60. 26.100
  libavcodec     62. 28.100 / 62. 28.100
  libavformat    62. 12.100 / 62. 12.100
  libavdevice    62.  3.100 / 62.  3.100
  libavfilter    11. 14.100 / 11. 14.100
  libswscale      9.  5.100 /  9.  5.100
  libswresample   6.  3.100 /  6.  3.100
Input #0, matroska,webm, from '/tmp/output.mkv':
  Metadata:
    ENCODER         : Lavf62.12.100
  Duration: 00:00:08.01, start: 0.000000, bitrate: 26687 kb/s
  Stream #0:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 1920x1080, 30 fps, 30 tbr, 1k tbn
    Metadata:
      DURATION        : 00:00:08.009000000

Here's the capture, remuxed into a standard .mp4 file with mjpeg inside:

20260422-dongle-1-video-capture.mp4

The beginning of the capture is all black frames, so we skip them.

# First, see where the black frames end
ffmpeg -i /tmp/output.mkv -vf "blackdetect=d=0.01:pix_th=0.10" -an -f null - 2>&1 | grep blackdetect
[Parsed_blackdetect_0 @ 0x7f03cc003d80] black_start:0 black_end:2.276 black_duration:2.276

Then extract the first usable frame.

$ ffmpeg -ss 2.276 -i /tmp/output.mkv -frames:v 1 -c:v png /tmp/frame.png
...
Output #0, image2, to '/tmp/frame.png':
  Metadata:
    encoder         : Lavf62.12.100
  Stream #0:0: Video: png, rgb24(pc, gbr/unknown/unknown, progressive), 1920x1080, q=2-31, 200 kb/s, 30 fps, 30 tbn
    Metadata:
      encoder         : Lavc62.28.100 png
      DURATION        : 00:00:08.009000000
[out#0/image2 @ 0x562fd688a100] video:366KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.03 bitrate=N/A speed=0.515x elapsed=0:00:00.06

The first usable frame of the video looks like this, converted to PNG, so all the JPEG artifacts are visible.

This is not great quality.

Clearly the coding chip in the dongle captures at a lower resolution or a high compression and everything fuzzes out.

I'd like to find a USB capture card that works better than this, ideally 1080p60.

Dongle 0: Technical Readout

Here's what dmesg says about it:

[37833.219860] usb 1-1: new high-speed USB device number 10 using xhci_hcd
[37833.349465] usb 1-1: New USB device found, idVendor=534d, idProduct=2109, bcdDevice=21.00
[37833.349475] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[37833.349479] usb 1-1: Product: USB3.0 HD VIDEO
[37833.349483] usb 1-1: Manufacturer: MACROSILICON
[37833.385869] uvcvideo 1-1:1.0: Found UVC 1.00 device USB3.0 HD VIDEO (534d:2109)
[37833.392342] hid-generic 0003:534D:2109.000B: hiddev96,hidraw1: USB HID v1.10 Device [MACROSILICON USB3.0 HD VIDEO] on usb-0000:64:00.3-1/input4
[37834.127009] usb 1-1: USB disconnect, device number 10

Here's what lsusb says about it, it enumerates as a crappy USB2 device.

$ lsusb
Bus 001 Device 026: ID 534d:2109 MacroSilicon USB3.0 HD VIDEO

$ lsusb -tv
/:  Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/5p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    |__ Port 001: Dev 015, If 0, Class=Video, Driver=uvcvideo, 480M
        ID 534d:2109 MacroSilicon
    |__ Port 001: Dev 015, If 1, Class=Video, Driver=uvcvideo, 480M
        ID 534d:2109 MacroSilicon
    |__ Port 001: Dev 015, If 2, Class=Audio, Driver=snd-usb-audio, 480M
        ID 534d:2109 MacroSilicon
    |__ Port 001: Dev 015, If 3, Class=Audio, Driver=snd-usb-audio, 480M
        ID 534d:2109 MacroSilicon
    |__ Port 001: Dev 015, If 4, Class=Human Interface Device, Driver=usbhid, 480M
        ID 534d:2109 MacroSilicon

$ sudo lsusb -s 001:026 -v
Bus 001 Device 026: ID 534d:2109 MacroSilicon USB3.0 HD VIDEO
Negotiated speed: High Speed (480Mbps)
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 [unknown]
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x534d MacroSilicon
  idProduct          0x2109 USB3.0 HD VIDEO
  bcdDevice           21.00
  iManufacturer           1 MACROSILICON
  iProduct                2 USB3.0 HD VIDEO
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x04bb
    bNumInterfaces          5
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         2
      bFunctionClass         14 Video
      bFunctionSubClass       3 Video Interface Collection
      bFunctionProtocol       0
      iFunction               2 USB3.0 HD VIDEO
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass        14 Video
      bInterfaceSubClass      1 Video Control
      bInterfaceProtocol      0
      iInterface              2 USB3.0 HD VIDEO
      VideoControl Interface Descriptor:
        bLength                13
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdUVC               1.00
        wTotalLength       0x0033
        dwClockFrequency       48.000000MHz
        bInCollection           1
        baInterfaceNr( 0)       1
      VideoControl Interface Descriptor:
        bLength                18
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Camera Sensor
        bAssocTerminal          0
        iTerminal               0
        wObjectiveFocalLengthMin      0
        wObjectiveFocalLengthMax      0
        wOcularFocalLength            0
        bControlSize                  3
        bmControls           0x00000000
      VideoControl Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      5 (PROCESSING_UNIT)
      Warning: Descriptor too short
        bUnitID                 2
        bSourceID               1
        wMaxMultiplier          0
        bControlSize            2
        bmControls     0x0000000f
          Brightness
          Contrast
          Hue
          Saturation
        iProcessing             0
        bmVideoStandards     0x09
          None
          SECAM - 625/50
      VideoControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               2
        iTerminal               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      VideoStreaming Interface Descriptor:
        bLength                            15
        bDescriptorType                    36
        bDescriptorSubtype                  1 (INPUT_HEADER)
        bNumFormats                         2
        wTotalLength                   0x03b1
        bEndpointAddress                 0x83  EP 3 IN
        bmInfo                              0
        bTerminalLink                       3
        bStillCaptureMethod                 0
        bTriggerSupport                     0
        bTriggerUsage                       0
        bControlSize                        1
        bmaControls( 0)                     0
        bmaControls( 1)                     0
      VideoStreaming Interface Descriptor:
        bLength                            11
        bDescriptorType                    36
        bDescriptorSubtype                  6 (FORMAT_MJPEG)
        bFormatIndex                        1
        bNumFrameDescriptors               11
        bFlags                              1
          Fixed-size samples: Yes
        bDefaultFrameIndex                  1
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 1 fields
          Field 1 first: No
          Field pattern: Field 1 only
        bCopyProtect                        0
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         1
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1920
        wHeight                          1080
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     4147200
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            400000
        dwFrameInterval( 2)            500000
        dwFrameInterval( 3)           1000000
        dwFrameInterval( 4)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         2
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1600
        wHeight                          1200
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     3840000
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            400000
        dwFrameInterval( 2)            500000
        dwFrameInterval( 3)           1000000
        dwFrameInterval( 4)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         3
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1360
        wHeight                           768
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     2088960
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            400000
        dwFrameInterval( 2)            500000
        dwFrameInterval( 3)           1000000
        dwFrameInterval( 4)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         4
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1280
        wHeight                          1024
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     2621440
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            400000
        dwFrameInterval( 2)            500000
        dwFrameInterval( 3)           1000000
        dwFrameInterval( 4)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         5
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1280
        wHeight                           960
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     2457600
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            200000
        dwFrameInterval( 1)            333333
        dwFrameInterval( 2)            500000
        dwFrameInterval( 3)           1000000
        dwFrameInterval( 4)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         6
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1280
        wHeight                           720
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     1843200
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            166666
        dwFrameInterval( 1)            200000
        dwFrameInterval( 2)            333333
        dwFrameInterval( 3)            500000
        dwFrameInterval( 4)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         7
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1024
        wHeight                           768
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     1572864
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            166666
        dwFrameInterval( 1)            200000
        dwFrameInterval( 2)            333333
        dwFrameInterval( 3)            500000
        dwFrameInterval( 4)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         8
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            800
        wHeight                           600
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      960000
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            166666
        dwFrameInterval( 1)            200000
        dwFrameInterval( 2)            333333
        dwFrameInterval( 3)            500000
        dwFrameInterval( 4)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                         9
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            720
        wHeight                           576
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      829440
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            166666
        dwFrameInterval( 1)            200000
        dwFrameInterval( 2)            333333
        dwFrameInterval( 3)            500000
        dwFrameInterval( 4)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                        10
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            720
        wHeight                           480
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      691200
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            166666
        dwFrameInterval( 1)            200000
        dwFrameInterval( 2)            333333
        dwFrameInterval( 3)            500000
        dwFrameInterval( 4)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            46
        bDescriptorType                    36
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        bFrameIndex                        11
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            640
        wHeight                           480
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      614400
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  5
        dwFrameInterval( 0)            166666
        dwFrameInterval( 1)            200000
        dwFrameInterval( 2)            333333
        dwFrameInterval( 3)            500000
        dwFrameInterval( 4)           1000000
      VideoStreaming Interface Descriptor:
        bLength                             6
        bDescriptorType                    36
        bDescriptorSubtype                 13 (COLORFORMAT)
        bColorPrimaries                     1 (BT.709,sRGB)
        bTransferCharacteristics            1 (BT.709)
        bMatrixCoefficients                 4 (SMPTE 170M (BT.601))
      VideoStreaming Interface Descriptor:
        bLength                            27
        bDescriptorType                    36
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bFormatIndex                        2
        bNumFrameDescriptors               11
        guidFormat                            {32595559-0000-0010-8000-00aa00389b71}
        bBitsPerPixel                      16
        bDefaultFrameIndex                  6
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 2 fields
          Field 1 first: No
          Field pattern: Field 1 only
        bCopyProtect                        0
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         1
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1920
        wHeight                          1080
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     4147200
        dwDefaultFrameInterval        2000000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         2
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1600
        wHeight                          1200
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     3840000
        dwDefaultFrameInterval        2000000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         3
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1360
        wHeight                           768
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     2088960
        dwDefaultFrameInterval        1250000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           1250000
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         4
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1280
        wHeight                          1024
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     2621440
        dwDefaultFrameInterval        1250000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           1250000
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         5
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1280
        wHeight                           960
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     2457600
        dwDefaultFrameInterval        1250000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           1250000
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         6
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1280
        wHeight                           720
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     1843200
        dwDefaultFrameInterval        1000000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         7
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                           1024
        wHeight                           768
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize     1572864
        dwDefaultFrameInterval        1000000
        bFrameIntervalType                  1
        dwFrameInterval( 0)           1000000
      VideoStreaming Interface Descriptor:
        bLength                            38
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         8
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            800
        wHeight                           600
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      960000
        dwDefaultFrameInterval         500000
        bFrameIntervalType                  3
        dwFrameInterval( 0)            500000
        dwFrameInterval( 1)           1000000
        dwFrameInterval( 2)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            42
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         9
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            720
        wHeight                           576
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      829440
        dwDefaultFrameInterval         400000
        bFrameIntervalType                  4
        dwFrameInterval( 0)            400000
        dwFrameInterval( 1)            500000
        dwFrameInterval( 2)           1000000
        dwFrameInterval( 3)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            42
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                        10
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            720
        wHeight                           480
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      691200
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  4
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            500000
        dwFrameInterval( 2)           1000000
        dwFrameInterval( 3)           2000000
      VideoStreaming Interface Descriptor:
        bLength                            42
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                        11
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            640
        wHeight                           480
        dwMinBitRate                   768000
        dwMaxBitRate                196608000
        dwMaxVideoFrameBufferSize      614400
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  4
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            500000
        dwFrameInterval( 2)           1000000
        dwFrameInterval( 3)           2000000
      VideoStreaming Interface Descriptor:
        bLength                             6
        bDescriptorType                    36
        bDescriptorSubtype                 13 (COLORFORMAT)
        bColorPrimaries                     1 (BT.709,sRGB)
        bTransferCharacteristics            1 (BT.709)
        bMatrixCoefficients                 4 (SMPTE 170M (BT.601))
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x0320  1x 800 bytes
        bInterval               1
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       2
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x0c00  2x 1024 bytes
        bInterval               1
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       3
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x1400  3x 1024 bytes
        bInterval               1
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         2
      bInterfaceCount         2
      bFunctionClass          1 Audio
      bFunctionSubClass       1 Control Device
      bFunctionProtocol       0
      iFunction               4 USB3.0 AUDIO
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol      0
      iInterface              4 USB3.0 AUDIO
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               1.00
        wTotalLength       0x0026
        bInCollection           1
        baInterfaceNr(0)        3
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0602 Digital Audio Interface
        bAssocTerminal          0
        bNrChannels             2
        wChannelConfig     0x0003
          Left Front (L)
          Right Front (R)
        iChannelNames           0
        iTerminal               0
      AudioControl Interface Descriptor:
        bLength                 8
        bDescriptorType        36
        bDescriptorSubtype      6 (FEATURE_UNIT)
        bUnitID                 2
        bSourceID               1
        bControlSize            1
        bmaControls(0)       0x01
          Mute Control
        iFeature                0
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               2
        iTerminal               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol      0
      iInterface              4 USB3.0 AUDIO
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol      0
      iInterface              4 USB3.0 AUDIO
      AudioStreaming Interface Descriptor:
        bLength                 7
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink           3
        bDelay                  0 frames
        wFormatTag         0x0001 PCM
      AudioStreaming Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bNrChannels             1
        bSubframeSize           2
        bBitResolution         16
        bSamFreqType            1 Discrete
        tSamFreq[ 0]        96000
      Endpoint Descriptor:
        bLength                 9
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x0100  1x 256 bytes
        bInterval               4
        bRefresh                0
        bSynchAddress           0
        AudioStreaming Endpoint Descriptor:
          bLength                 7
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bLockDelayUnits         0 Undefined
          wLockDelay         0x0000
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        4
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 [unknown]
      bInterfaceProtocol      0
      iInterface              0
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.10
          bCountryCode           33 US
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      23
          Report Descriptors:
            ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x84  EP 4 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              16
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 [unknown]
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)

Funny that the product string is USB3.0 HD VIDEO but it only runs at 480Mbps.

Here's what v4l2-ctl says about the device:

$ v4l2-ctl --list-devices
USB3.0 HD VIDEO: USB3.0 HD VIDE (usb-0000:64:00.3-2):
    /dev/video4
    /dev/video5
    /dev/media2

$ v4l2-ctl -d /dev/video4 --list-formats
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'MJPG' (Motion-JPEG, compressed)
    [1]: 'YUYV' (YUYV 4:2:2)

$ v4l2-ctl -d /dev/video5 --list-formats
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

$ v4l2-ctl -d /dev/video4 --all
Driver Info:
	Driver name      : uvcvideo
	Card type        : USB3.0 HD VIDEO: USB3.0 HD VIDE
	Bus info         : usb-0000:64:00.3-2
	Driver version   : 6.19.12
	Capabilities     : 0x84a00001
		Video Capture
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : uvcvideo
	Model            : USB3.0 HD VIDEO: USB3.0 HD VIDE
	Serial           :
	Bus info         : usb-0000:64:00.3-2
	Media version    : 6.19.12
	Hardware revision: 0x00002100 (8448)
	Driver version   : 6.19.12
Interface Info:
	ID               : 0x03000002
	Type             : V4L Video
Entity Info:
	ID               : 0x00000001 (1)
	Name             : USB3.0 HD VIDEO: USB3.0 HD VIDE
	Function         : V4L2 I/O
	Flags            : default
	Pad 0x01000007   : 0: Sink
	  Link 0x0200000d: from remote pad 0x100000a of entity 'Processing 2' (Video Pixel Formatter): Data, Enabled, Immutable
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'MJPG' (Motion-JPEG)
	Field             : None
	Bytes per Line    : 0
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Rec. 709
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Default (maps to Full Range)
	Flags             :
Crop Capability Video Capture:
	Bounds      : Left 0, Top 0, Width 1920, Height 1080
	Default     : Left 0, Top 0, Width 1920, Height 1080
	Pixel Aspect: 1/1
Selection Video Capture: crop_default, Left 0, Top 0, Width 1920, Height 1080, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 1920, Height 1080, Flags:
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 30.000 (30/1)
	Read buffers     : 0

User Controls

                     brightness 0x00980900 (int)    : min=-128 max=127 step=1 default=-11 value=-11 flags=has-min-max
                       contrast 0x00980901 (int)    : min=0 max=255 step=1 default=148 value=148 flags=has-min-max
                     saturation 0x00980902 (int)    : min=0 max=255 step=1 default=180 value=180 flags=has-min-max
                            hue 0x00980903 (int)    : min=-128 max=127 step=1 default=0 value=0 flags=has-min-max

$ v4l2-ctl -d /dev/video5 --all
Driver Info:
	Driver name      : uvcvideo
	Card type        : USB3.0 HD VIDEO: USB3.0 HD VIDE
	Bus info         : usb-0000:64:00.3-2
	Driver version   : 6.19.12
	Capabilities     : 0x84a00001
		Video Capture
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04a00000
		Metadata Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : uvcvideo
	Model            : USB3.0 HD VIDEO: USB3.0 HD VIDE
	Serial           :
	Bus info         : usb-0000:64:00.3-2
	Media version    : 6.19.12
	Hardware revision: 0x00002100 (8448)
	Driver version   : 6.19.12
Interface Info:
	ID               : 0x03000005
	Type             : V4L Video
Entity Info:
	ID               : 0x00000004 (4)
	Name             : USB3.0 HD VIDEO: USB3.0 HD VIDE
	Function         : V4L2 I/O
Priority: 2
Format Metadata Capture:
	Sample Format   : 'UVCH' (UVC Payload Header Metadata)
	Buffer Size     : 10240

Dongle 0: 2026 Edition

This might be the updated version of the dongle I have, with USB3 and 1080p60, but I'm not spending money to find out.

Dongle 1: Rounded Metal, No Tail, USB A Plug

"4K Video Capture Card HDMI USB 1080P Live Video Streaming Game Recorder USB-C"

Dongle 0, the one I have similar to this, is also a USB 2.0 device, with 1080p30 resolution in MJPEG mode, and it looks really bad.

It's not even worth the 10 EUR they want for it.

Features:

* Unterstützte Eingangsauflösung: max.3840×2160@30Hz;
* Unterstützte Aufnahmeauflösung: max.1920×1080@30Hz im MJPEG-Format;
* Unterstützt 8/10/12 Bit tiefe Farbe;
* Unterstütztes Audioformat: L-PCM
* Unterstützt AWG26 HDMI-kompatibles Standardkabel: Eingang bis zu 15 Meter (1080P und weniger Auflösung);
* Unterstützt die meisten Erfassungssoftware wie VLC,OBS,Vmix, Potplay, Amcap usw.;
* Unterstützt Windows und MacOS;
* Entspricht dem USB-Video- und UVC-Standard;
* Entspricht dem USB-Audio-UAC-Standard;
* Ohne externe Stromversorgung, kompakt und tragbar.
* Abmessungen (L x B x H): ca.64x28x13 (mm)

Hinweis:

Anforderungen an die Konfiguration von Computerhardware
CPU: PC i5-3400 oder höher; NB i7-3537U 2,0 GHz oder höher
Grafikkarte: PC NVIDIA GT630 oder höher; NC NVIDIA GT735M oder höher
Speicher ausführen: 4G RAM

Somehow I don't think this will ever be qualified for telemedicine. Maybe in China?

In another listing, the pictures are similar but a little different.

For example, the screws are different.

And they've added a little USB-C to USB-A adapter cable, and rotated some of the images.

I must be tired of life, so I order one.

I don't know who's doing their media packs, but they're doing a great job! 😀

I like that it's sitting on a faux mirrored surface, but the top and bottom images are not inverted?

Decluttering CachyOS

2026-03-20T00:00:00Z

Following on from the Useful Oneliners for Ubuntu, I also wanted to clean up my CachyOS system, which I made the mistake of installing a bunch of things natively onto.

I'm messing around with Claude again, and it's generating me a useful script.

Prompts

Here goes:

Create a script that downloads https://github.com/CachyOS/cachyos-calamares/blob/cachyos-dev/src/modules/netinstall/netinstall.yaml then reads the file and prints the top level "name" properties
Cache the file and use the cached file in future calls unless told to download again
For the names ("CachyOS required (hidden)", "CachyOS Packages", "CachyeOS shell configuration", "Base-devel + Common packages", and "CPU specific Microcode update packages") add all of the listed package names to a list in memory
Now add command line options based on the names of the remaining groups, if the command line option is passed to the script, add the packages in those subgroups to the base packages list
Add command line help
Change the help text slightly, all this does is print a list of packages for different CachyOS desktop environments and optional extras
Word wrap each line of the command line help to 120 characters
Add an option to alphabetically sort everything combined

The Optional Package Groups

What It Can Do

Here's the command-line help of the final script:

$ ./get_names.py --help
Usage: get_names.py [OPTIONS] [--<group-flag> ...]

Prints a list of packages for different CachyOS desktop environments and optional extras.

The following base groups are always included:
  - Base-devel + Common packages
  - CPU specific Microcode update packages
  - CachyOS Packages
  - CachyOS required (hidden)
  - CachyOS shell configuration

Options:
  -h, --help     Show this help message and exit
  --refresh      Re-download the package list from GitHub (ignores cache)
  --sort         Sort the output package list alphabetically

Optional package groups:
  --kde-desktop                              KDE-Plasma Desktop - Simple by default, powerful when needed.
  --gnome-desktop                            GNOME desktop environment - designed to put you in control and get things
                                             done.
  --xfce4                                    Xfce is a lightweight desktop environment for UNIX-like operating systems.
                                             It aims to be fast and low on system resources, while still being visually
                                             appealing and user friendly.
  --bspwm                                    bspwm is a tiling window manager that represents windows as the leaves of a
                                             full binary tree. bspwm supports multiple monitors and is configured and
                                             controlled through messages.
  --budgie-desktop                           Budgie - an independent, familiar, and modern desktop.
  --cinnamon                                 Linux desktop which provides advanced innovative features and a traditional
                                             user experience.
  --cosmic                                   Linux desktop, which provides an environment that features advanced
                                             functionality and a responsive design.
  --i3-window-manager                        i3 tiling window manager, primarily targeted at developers and advanced
                                             users.
  --hyprland                                 Hyprland is a highly customizable dynamic tiling Wayland compositor that
                                             doesn't sacrifice on its looks.
  --niri                                     A scrollable-tiling Wayland compositor.
  --lxde-desktop                             The Lightweight Desktop Environment.
  --lxqt-desktop                             The Lightweight Qt Desktop Environment.
  --mate-desktop                             MATE Desktop - the continuation of GNOME 2
  --openbox                                  Openbox is a highly configurable, floating window manager with extensive
                                             standards support.
  --qtile                                    Qtile is a X11 window manager that is configured with the Python
                                             programming language.
  --sway                                     Sway is a tiling Wayland compositor and a drop-in replacement for the i3
                                             window manager for X11. It works with your existing i3 configuration and
                                             supports most of i3's features, plus a few extras.
  --ukui                                     It is a lightweight desktop environment, which consumes few resources and
                                             works with older computers. It has been developed with GTK and Qt
                                             technologies. Its visual appearance is similar to Windows 7, making it
                                             easier for new users of Linux.
  --wayfire                                  Wayfire is a wayland compositor based on wlroots. It aims to create a
                                             customizable, extendable and lightweight environment without sacrificing
                                             its appearance.
  --firefox-and-language-package             Add firefox and language pack if possible
  --printing-support                         Support for printing (Cups)
  --support-for-hp-printer-scanner           Extra Packages for HP Printer/Scanner
  --accessibility-tools                      Screen reader and mouse tweaks (impaired vision)

Output

Here's what it outputs for the CachyOS options I originally chose:

$ ./get_names.py --gnome-desktop --firefox-and-language-package --printing-support --support-for-hp-printer-scanner --sort
Using cached file: /home/max/Downloads/idea-cleanup-cachyos-host/netinstall.yaml
accountsservice
alacritty
alsa-firmware
alsa-plugins
alsa-utils
amd-ucode
awesome-terminal-fonts
bash-completion
bluez
bluez-hid2hci
bluez-libs
bluez-obex
bluez-utils
btop
cachyos-fish-config
cachyos-hello
cachyos-hooks
cachyos-kernel-manager
cachyos-keyring
cachyos-micro-settings
cachyos-mirrorlist
cachyos-packageinstaller
cachyos-rate-mirrors
cachyos-settings
cachyos-v3-mirrorlist
cachyos-v4-mirrorlist
cachyos-wallpapers
cachyos-zsh-config
cantarell-fonts
chwd
cpupower
cups
cups
cups-filters
cups-filters
cups-pdf
cups-pdf
dmidecode
dmraid
dnsmasq
dnsutils
duf
efitools
ethtool
fastfetch
ffmpegthumbnailer
file-roller
firefox
firefox-i18n-$LOCALE
foomatic-db
foomatic-db-engine
foomatic-db-gutenprint-ppds
foomatic-db-nonfree
foomatic-db-nonfree-ppds
foomatic-db-ppds
fsarchiver
fwupd
gdm
gedit
ghostscript
git
glances
gnome-backgrounds
gnome-calculator
gnome-console
gnome-control-center
gnome-disk-utility
gnome-keyring
gnome-nettool
gnome-power-manager
gnome-shell
gnome-tweaks
gnome-usage
gsfonts
gst-libav
gst-plugin-pipewire
gst-plugins-bad
gst-plugins-ugly
gutenprint
gvfs-afc
gvfs-gphoto2
gvfs-mtp
gvfs-nfs
gvfs-smb
hdparm
hplip
hwdetect
hwinfo
intel-ucode
iwd
libdvdcss
libgsf
libnma
libopenraw
linux-cachyos
linux-cachyos-headers
linux-cachyos-lts
linux-cachyos-lts-headers
linux-firmware
loupe
lsscsi
malcontent
meld
mesa-utils
micro
modemmanager
mtools
nano
nano-syntax-highlighting
networkmanager
networkmanager-openvpn
nfs-utils
nilfs-utils
noto-fonts
noto-fonts-cjk
noto-fonts-emoji
nss-mdns
octopi
openssh
pacman-contrib
papers
paru
pavucontrol
pipewire-alsa
pipewire-pulse
pkgfile
plocate
poppler-glib
power-profiles-daemon
pv
python-defusedxml
python-packaging
python-pyqt5
python-reportlab
rebuild-detector
reflector
ripgrep
rsync
sg3_utils
showtime
simple-scan
smartmontools
sof-firmware
splix
sushi
system-config-printer
ttf-bitstream-vera
ttf-dejavu
ttf-liberation
ttf-meslo-nerd
ttf-opensans
ufw
unrar
unzip
upower
usb_modeswitch
vim
vlc-plugins-all
wget
wireless-regdb
wireplumber
wpa_supplicant
xdg-user-dirs
xdg-utils
xl2tpd

Now What?

Now I can use this to compare the list of original installed packages to the list of currently-installed packages.

This is similar to what I did previously with Ubuntu's install manifests.

The idea is to clean up my CachyOS host system, to remove all the cruft, and put as many of those userspace commands into Docker container calls, and so on.

Useful Oneliners: Comparing CachyOS Installed Packages and Original Install Manifest Lists

diff -y <(pacman -Q | awk '{print $1}') <(./get_names.py --gnome-desktop --firefox-and-language-package --printing-support --support-for-hp-printer-scanner --sort)

Ok this one's a bit of a mess compared to Ubuntu, because there's a lot of fanout from the packages in the install manifests to what is then installed on the system.

These seem to be meta packages and a lot of noise.

This isn't particularly useful. What I need instead is a directed graph showing which packages cause the most other packages to be installed, and can be cleaned up.

Using cached file: /home/max/Downloads/idea-cleanup-cachyos-host/netinstall.yaml
7zip                                                      <
a52dec                                                    <
aalib                                                     <
abseil-cpp                                                <
accountsservice                                           accountsservice
acl                                                       | alacritty
adobe-source-han-sans-cn-fonts                            <
adobe-source-han-sans-jp-fonts                            <
adobe-source-han-sans-kr-fonts                            <
adwaita-cursors                                           <
adwaita-fonts                                             <
adwaita-icon-theme                                        <
adwaita-icon-theme-legacy                                 <
alsa-card-profiles                                        <
alsa-firmware                                             alsa-firmware
alsa-lib                                                  <
alsa-plugins                                              alsa-plugins
alsa-topology-conf                                        <
alsa-ucm-conf                                             <
alsa-utils                                                alsa-utils
amd-ucode                                                 amd-ucode
ananicy-cpp                                               <
android-file-transfer                                     <
android-tools                                             <
android-udev                                              <
aom                                                       <
apache                                                    <
appstream                                                 <
apr                                                       <
apr-util                                                  <
archlinux-keyring                                         <
argon2                                                    <
aribb24                                                   <
aribb25                                                   <
at-spi2-core                                              <
atkmm                                                     <
atomicparsley                                             <
attr                                                      <
audit                                                     <
autoconf                                                  <
automake                                                  <
avahi                                                     <
awesome-terminal-fonts                                    awesome-terminal-fonts
baobab                                                    <

Claude's Neat Automatic Tricks

The original command-line help wasn't word wrapped:

The word wrap was cleverer than I even asked it to be, and I was testing what it would do. It wrapped and aligned the description text automatically. Nice going!

The Script Itself

#!/usr/bin/env python3
import re
import sys
import urllib.request
import yaml
from pathlib import Path

url = "https://raw.githubusercontent.com/CachyOS/cachyos-calamares/cachyos-dev/src/modules/netinstall/netinstall.yaml"
cache_path = Path(__file__).parent / "netinstall.yaml"

if "--help" in sys.argv or "-h" in sys.argv:
    import io
    temp_data = None
    if cache_path.exists():
        temp_data = yaml.safe_load(cache_path.read_text())
    else:
        print("(Downloading package list to show available groups...)", file=sys.stderr)
        with urllib.request.urlopen(url) as response:
            content = response.read()
        cache_path.write_bytes(content)
        temp_data = yaml.safe_load(cache_path.read_text())

    BASE_GROUPS_HELP = {
        "CachyOS required (hidden)",
        "CachyOS Packages",
        "CachyOS shell configuration",
        "Base-devel + Common packages",
        "CPU specific Microcode update packages",
    }
    optional = [
        item for item in temp_data
        if isinstance(item, dict) and "name" in item and item["name"] not in BASE_GROUPS_HELP
    ]

    print("Usage: get_names.py [OPTIONS] [--<group-flag> ...]")
    print()
    print("Prints a list of packages for different CachyOS desktop environments and optional extras.")
    print()
    print("The following base groups are always included:")
    for name in sorted(BASE_GROUPS_HELP):
        print(f"  - {name}")
    print()
    print("Options:")
    print("  -h, --help     Show this help message and exit")
    print("  --refresh      Re-download the package list from GitHub (ignores cache)")
    print("  --sort         Sort the output package list alphabetically")
    print()
    import textwrap
    print("Optional package groups:")
    for item in optional:
        flag = re.sub(r"[^a-z0-9]+", "-", item["name"].lower()).strip("-")
        desc = item.get("description", "")
        prefix = f"  --{flag:<40} "
        indent = " " * len(prefix)
        wrapped = textwrap.fill(desc, width=120, initial_indent=prefix, subsequent_indent=indent)
        print(wrapped)
    sys.exit(0)

force_download = "--refresh" in sys.argv

if force_download or not cache_path.exists():
    with urllib.request.urlopen(url) as response:
        content = response.read()
    cache_path.write_bytes(content)
    print(f"Downloaded and cached to {cache_path}", file=sys.stderr)
else:
    print(f"Using cached file: {cache_path}", file=sys.stderr)

data = yaml.safe_load(cache_path.read_text())

BASE_GROUPS = {
    "CachyOS required (hidden)",
    "CachyOS Packages",
    "CachyOS shell configuration",
    "Base-devel + Common packages",
    "CPU specific Microcode update packages",
}

def collect_packages(item):
    pkgs = list(item.get("packages") or [])
    for sub in item.get("subgroups") or []:
        pkgs.extend(collect_packages(sub))
    return pkgs

def name_to_flag(name):
    slug = name.lower()
    slug = re.sub(r"[^a-z0-9]+", "-", slug)
    return slug.strip("-")

optional_groups = {
    name_to_flag(item["name"]): item
    for item in data
    if isinstance(item, dict) and "name" in item and item["name"] not in BASE_GROUPS
}

base_packages = []
for item in data:
    if isinstance(item, dict) and item.get("name") in BASE_GROUPS:
        base_packages.extend(collect_packages(item))

for flag, item in optional_groups.items():
    if f"--{flag}" in sys.argv:
        base_packages.extend(collect_packages(item))

packages = sorted(base_packages) if "--sort" in sys.argv else base_packages
for pkg in packages:
    print(pkg)

Reviving My Blog - Checking Broken Links

2026-03-19T00:00:00Z

After doing the conversion to Markdown in the previous post, I wanted to check and to see how many links were broken.

I had Claude write me a Python script to do this.

Turns out, quite a few broken links:

I went through and fixed the ones I could fix manually, if I knew where the contents had migrated and they were under my control.

For the rest of the broken links, I had Claude rewrite them to indicate that they were broken.

Applying fixes to 201 files...
  fixed: 2010/09/23/hello-gdbserver-a-debuggable-jni-example-for-android/index.md
  fixed: 2010/09/26/geocoding-with-mapstraction-v2-and-google-maps-v3/index.md
  fixed: 2011/01/19/testing-rest-api-with-learnboosts-tobi-vows-js/index.md
  fixed: 2012/01/02/sopaprotect-ip-blackout-simulator/index.md
  fixed: 2012/09/04/racing-towards-the-bottom-or-how-not-to-handle-customers/index.md
  fixed: 2012/12/04/spatial_ref_sys-and-geometry_columns/index.md
  fixed: 2012/12/17/django-unit-testing-on-shared-hoster/index.md
  fixed: 2012/12/22/django-integrityerror-on-postgresql/index.md
  fixed: 2013/01/04/old-machine-resurrection/index.md
  fixed: 2013/01/26/os-x-user-account-control-popup-finish_installation-wants-to-make-changes/index.md
  fixed: 2013/01/29/ebay-seller-bait-and-switch-maneuvers/index.md
  fixed: 2013/02/23/digitizing-cassette-tapes/index.md
  fixed: 2013/03/08/guffaw/index.md
  fixed: 2013/03/22/the-underside-of-the-tablet/index.md
  fixed: 2013/04/01/localization-translation-google-spreadsheets/index.md
  fixed: 2013/04/11/lazy-sunday-project-fixing-my-bike-light/index.md
  fixed: 2013/05/02/django-needs-to-die/index.md
  fixed: 2013/05/07/new-emacs/index.md
  fixed: 2013/05/11/small-time-criminals/index.md
  fixed: 2013/05/14/branding/index.md
  fixed: 2013/05/31/twitter-houses/index.md
  fixed: 2013/06/08/bless-you-less-css/index.md
  fixed: 2013/06/09/using-google-protocol-buffers-with-qmake/index.md
  fixed: 2013/06/25/locale-fun/index.md
  fixed: 2013/07/04/django-testing-creating-and-removing-test-users/index.md
  fixed: 2013/07/08/terrible-connectors/index.md
  fixed: 2013/07/10/python-code-coverage-and-cron/index.md
  fixed: 2013/07/12/python-parametrized-unit-tests/index.md
  fixed: 2013/07/19/libevent-gcov-lcov-and-os-x/index.md
  fixed: 2013/07/21/stateful-vs-stateless-browsing-improving-your-privacy-online/index.md
  fixed: 2013/09/04/using-selenium-webdriver-with-phantomjs-on-a-shared-host/index.md
  fixed: 2013/09/13/setting-up-docker-and-buildbot/index.md
  fixed: 2013/10/04/mucking-around-with-libc-on-android/index.md
  fixed: 2013/10/06/ubuntu-13-10-chroot-on-android/index.md
  fixed: 2013/10/08/ndk-configure-sh/index.md
  fixed: 2013/10/23/apple-vs-microsoft-strategy/index.md
  fixed: 2014/01/15/provisioning-os-x-and-disabling-unnecessary-services/index.md
  fixed: 2014/02/09/better-time-synchronization-with-virtualbox-guest-additions/index.md
  fixed: 2014/02/21/qmake-subdirs-project-automatic-dependencies/index.md
  fixed: 2014/03/16/power-hungry-desktops/index.md
  fixed: 2014/03/25/grails-groovy-scala-and-akka/index.md
  fixed: 2014/09/18/windows-7-hibernation/index.md
  fixed: 2015/01/28/the-mathematics-of-brew-pots/index.md
  fixed: 2015/06/24/hackathon-1-intel-edison/index.md
  fixed: 2015/07/15/casualties/index.md
  fixed: 2016/03/03/using-netbeans-and-the-nordic-nrf5-nrf52-sdk/index.md
  fixed: 2016/05/03/esptool-ck-esp8266-and-ftdi-bug-hunting/index.md
  fixed: 2016/05/22/riotboard-gpio-pinout/index.md
  fixed: 2017/02/04/silabs-cp210x-usb-adapters-for-the-win/index.md
  fixed: 2017/04/24/power-profiling-on-mbed-nordic-nrf5-series/index.md
  fixed: 2018/03/30/a-visit-to-huaqiangbei-market-by-an-embedded-systems-engineer/index.md
  fixed: 2019/06/10/new-website-design/index.md
  fixed: 2019/11/07/why-are-my-backups-so-slow/index.md
  fixed: 2020/04/20/noisefloor-firmware/index.md
Done.

Which then looks roughly like:

Doing this with an agent saves me many hours of manually scripting and then processing these files.

Useful Oneliners - Decluttering Ubuntu Server

2026-03-18T00:00:00Z

Here's a useful oneliner I cooked up recently to help me clean up a long-running Ubuntu Server:

diff -y <(dpkg-query -f '${binary:Package}\n' -W) <(curl https://releases.ubuntu.com/24.04/ubuntu-24.04.3-live-server-amd64.manifest | cut -f 1 - )

It shows what packages are installed locally that weren't in the original install manifest.

Some of them may be needed or added by upgrades, but quite a number may have been added surreptitiously, and simply stuck around.

Especially useful for server systems, where userspace should be as clean as possible and many userspace things should run in containers.

The side-by-side diff (installed vs. manifest) will look something like:

adduser                            adduser
amd64-microcode                    amd64-microcode
apparmor                           apparmor
apport                             apport
apport-core-dump-handler           apport-core-dump-handler
apport-symptoms                    apport-symptoms
appstream                          appstream
apt                                apt
apt-utils                          apt-utils
attr                               <
base-files                         base-files
base-passwd                        base-passwd
bash                               bash
bash-completion                    bash-completion
bc                                 bc
                                    > bcache-tools
bind9-dnsutils                     bind9-dnsutils
bind9-host                         bind9-host
bind9-libs:amd64                   bind9-libs:amd64
binutils                           | bolt
binutils-common:amd64              <
binutils-x86-64-linux-gnu          <
bpfcc-tools                        bpfcc-tools
bpftrace                           bpftrace
bsdextrautils                      bsdextrautils
bsdutils                           bsdutils
btop                               <
btrfs-progs                        btrfs-progs
build-essential                    <
busybox-initramfs                  busybox-initramfs
busybox-static                     busybox-static
byobu                              byobu
bzip2                              <
ca-certificates                    ca-certificates
                                    > casper
                                    > cifs-utils
cloud-guest-utils                  cloud-guest-utils
                                    > cloud-init
cloud-initramfs-copymods           cloud-initramfs-copymods
cloud-initramfs-dyn-netconf        cloud-initramfs-dyn-netconf
command-not-found                  command-not-found
...

I can then go through and remove the packages from the left side that don't need to be there anymore.

Even more useful is to throw a grep < on the end of there and see only the extra installed packages.

diff -y <(dpkg-query -f '${binary:Package}\n' -W) <(cat ubuntu-24.04.3-live-server-amd64.manifest | cut -f 1 - ) | grep '<'

attr                                  <
binutils-common:amd64                 <
binutils-x86-64-linux-gnu             <
btop                                  <
build-essential                       <
bzip2                                 <
containerd.io                         <
cpp                                   <
cpp-11                                <
cpp-13                                <
cpp-13-x86-64-linux-gnu               <
cpp-x86-64-linux-gnu                  <
dialog                                <
dns-root-data                         <
docker-buildx-plugin                  <
docker-ce                             <
docker-ce-cli                         <
docker-ce-rootless-extras             <
docker-compose-plugin                 <
dos2unix                              <
dpkg-dev                              <
exiv2                                 <
fakeroot                              <
g++-13-x86-64-linux-gnu               <
g++-x86-64-linux-gnu                  <
gcc                                   <
gcc-11                                <
gcc-11-base:amd64                     <
gcc-13                                <
gcc-13-base:amd64                     <
gcc-13-x86-64-linux-gnu               <
gcc-x86-64-linux-gnu                  <

After removing the packages, they'll still show up in dpkg -l as rc "removed but config files still in place",

$ dpkg -l | grep ^rc
rc  powertop                              2.15-3build1                            amd64        diagnose issues with power consumption and management
rc  silversearcher-ag                     2.2.0+git20200805-1.1                   amd64        very fast grep-like program, alternative to ack-grep
rc  usbmuxd                               1.1.1-5~exp3ubuntu2.1                   amd64        USB multiplexor daemon for iPhone and iPod Touch devices

You can purge them completely as follows:

$ sudo apt purge $(dpkg -l | grep "^rc" | awk '{print $2}')

Reviving My Blog

2026-03-13T00:00:00Z

Meet the new boss, same as the old boss.

It's been a long time since I added entries to my blog.

2021 was the last time I updated it.

Part of this was a loss of motivation to write, the bigger part of this was the mountain of old posts stored as interlinked HTML files which I had exported from WordPress years ago.

I couldn't see a clean way to add new posts to the old structure w/o also possibly breaking links or having to create a whole new parallel blog, something like a /blog2, which would have annoyed my usual sensitivies.

Adding WordPress again was out of the question.

WordPress is kind of the worst. Its WYSIWYG editing hides all of the gory details of web development from noobs, but also makes it a mess to actually do useful things with the data you've put into it.

Plugins make it worse.

WordPress hides all of the content in a database and is overkill for a personal website. The notion of hitting a database every single time someone makes a web request against what should be a collection of static pages, this didn't jive with me anymore.

Getting the data out of WordPress the first time was annoying, and it left everything in a messy and hard-to-wrangle state of play.

I got lazy, I let the data sit, and I didn't want to do anything with it.

Five years passed.

Enter Claude

A month ago, I finally picked up a subscription to Claude Pro and have been mucking around exploring its capabilities.

I run it on Ubuntu inside of a VirtualBox instance, with read-only and read-write Shared Folders automounted, and these are then symlinked to whatever folder I want to work in.

I absolutely keep it away from my host system, too many horror stories make it clear this is a must.

I like to be precise in my use of language, and find the Claude Code interface at the terminal to be quite amenable to the way I think things through.

So, I asked it to convert a folder of all of my old posts from HTML to Markdown format, as a first step to roundtripping those Markdown files back out to clean, static HTML files.

I wanted an identical folder structure which would map out to the identical URL structure.

It exceeded my expectations, though there was some cleanup needed.

Stripping Noisy Highlighter

The raw exported HTML changes from this:

into this:

After a lot more fiddling and filtering, more commands to Claude to do this for me, the final output is really clean!

Consolidating Tags

WordPress encouraged tagging, but I went a little too specific, which then generated all of these tag/index.md files with links to one or two posts in them. Pointless.

All tags (21):
  #FirstWorldProblems
  3D Printing
  Android
  Arduino
  Berlin
  Brewing
  CSS
  Django
  ESP8266
  Food
  Hardware Hacks
  Localization
  Mechanical
  Python
  Quiz
  Riotboard
  Society
  Startups
  Tech
  Uncategorized
  emacs

Claude to the rescue to consolidate them:

When any of the following tags are seen ("#FirstWorldProblems", "3D Printing", "Android",
"Arduino", "CSS", "Django", "ESP8266", "Hardware Hacks", "Localization", "Mechanical",
"Python", "Riotboard", "Startups", "emacs") convert them to "Tech". Then remove any duplicate
"Tech" tags.

And a little more whittling down, leads to:

All tags (4):
  Food
  Society
  Tech
  Uncategorized

I'll fix the Uncategorized later.

Converting Blockquotes To Code Blocks

This would have taken me several hours, but Claude went through in about 10 minutes to convert and to assign types to log outputs and code blocks. This is a huge timesaver.

"Go through the Markdown files and analyze the contents of blockquotes, then set the
type of blockquote based on what you think the contents are, i.e. bash commands,
source code, etc."

And one more time on the Code Blocks types:

This would have taken me several hours, quite some time.

Converting Consecutive Monospace Blocks To Code Blocks

There are some more weird blocks that need proper conversion.

The source for this looks as follows, which is weird.

More Formatting Fixes

There were some files with weird, non-uniform formatting.

Format Filenames as Monospace

becomes

Summary

In the end, Claude cut the development time for this one-off tool down from probably days to one long evening and a few hours in the morning.

I wanted my writing back, in as simple a format as possible, with as low a cost as possible.

And I got it, no extras.

While this is an enormous time and cost saving for me (human life is an input cost), the even larger benefit is to get over the inertia of finally doing this.

Now I can add these Markdown files as input to a static site generator and preserve the link structure of the old blog, while adding new contents to the blog.

Claude's summary of its own work:

Filtering (files skipped entirely)
- Root index.html
- Anything under author/, category/, tag/
- Any path containing a page/ component (e.g. top-level page/ or nested YYYY/MM/page/)
- Month-level index pages (YYYY/MM/index.html)

HTML Preprocessing (before pandoc)
1. Extract Crayon Syntax Highlighter code blocks, replacing them with placeholders
2. Replace gallery <div> elements with plain <img> tags pointing to full-size images
3. Unwrap <a href="full"><img src="thumb"> links → bare <img src="full">
4. Convert <figure> tags → <img> + caption <p> (wrapped together with <br> for flush layout)
5. Strip all <img> attributes except src and alt

Conversion
6. Run pandoc --from=html --to=markdown --wrap=auto --columns=100

Markdown Postprocessing (after pandoc)
7. Restore Crayon placeholders as backtick code fences
8. Rejoin markdown links/images that pandoc word-wrapped across lines
9. Strip <span> tags from inside code fences
10. Strip {crayon="false"} and other WordPress attribute annotations from fence opening lines
11. Remove stray blockquote > markers between consecutive fences
12. Remove empty code fences
13. Trim leading/trailing blank lines inside fences
14. Remove WordPress <!--more--> link artifacts
15. Word-wrap long copyright/license comment blocks inside code fences
16. Remove lines consisting only of a backslash
17. Fix opening formatting marker spacing (e.g. ** separated from its content)
18. Ensure a blank line before every # header

Frontmatter & Output
19. Build YAML frontmatter: title, date, author: "Max Vilimpoc", tags
    - Tags normalised: tech sub-categories → "Tech", "Brewing" → "Food", "Quiz" → "Uncategorized",
      if "Tech" present all other tags are dropped
20. Write frontmatter + markdown body to output file

It's worth every penny.

Some of the steps even reflect a fundamental shift in what I could expect of the Internet as a platform.

For example, bandwidth is far more plentiful and screens are far higher resolution, so: no more thumbnail images in anchor links, just show the full sized image right there. Don't make people click through.

Having the source code as plaintext for running through a syntax highlighter later is also nice.

Postnotes: Trying Out `eleventy`

The current blog is now roundtripped back from Markdown to HTML using Eleventy.

It seems to do what I need, without too much fuss.

$ ./node.sh npx @11ty/eleventy --serve
[11ty] Writing ./_site/feed/feed.xml from ./content/eleventy-plugin-feed-blog-title-atom.njk (virtual)
[11ty] Writing ./_site/sitemap.xml from ./content/sitemap.xml.njk
[11ty] Writing ./_site/blog/firstpost/index.html from .//blog/firstpost.md (njk)
[11ty] Writing ./_site/blog/secondpost/index.html from .//blog/secondpost.md (njk)
[11ty] Writing ./_site/tags/index.html from ./content/tags.njk
[11ty] Writing ./_site/blog/index.html from .//blog.njk
[11ty] Writing ./_site/blog/thirdpost/index.html from .//blog/thirdpost.md (njk)
[11ty] Writing ./_site/blog/fifthpost/index.html from .//blog/fifthpost.md (njk)
[11ty] Writing ./_site/index.html from ./content/index.njk
[11ty] Writing ./_site/tags/another-tag/index.html from ./content/tag-pages.njk
[11ty] Writing ./_site/tags/number-2/index.html from ./content/tag-pages.njk
[11ty] Writing ./_site/tags/second-tag/index.html from ./content/tag-pages.njk
[11ty] Writing ./_site/tags/posts-with-two-tags/index.html from ./content/tag-pages.njk
[11ty] Writing ./_site/404.html from ./content/404.md (njk)
[11ty] Writing ./_site/about/index.html from ./content/about.md (njk)
[11ty] Writing ./_site/blog/fourthpost/index.html from .//blog/fourthpost/fourthpost.md (njk)
[11ty/eleventy-img] 1 image optimized (1 deferred)
[11ty] Copied 2 Wrote 16 files in 0.25 seconds (v3.1.2)
[11ty] Watching…
[11ty] Server at http://localhost:8080/

`prefixpath` Issues

Actually there is some fuss, with the interplay between the prefixpath and the absolute paths that my existing blog posts were using.

It kept doubling up the prefixpath with subfolders in content/ so I ended up with /blog/ paths everywhere.

The solution is to move the year-based sorting folders into the main content/ folder directly, so remove one level of hierarchy.

There were still some /blog/ paths, so I wrote a sed expression to remove them:

find _site -type f -exec sed -i 's|/blog|/blog|g' {} +

Image Processor Problem

The other issue was the image processor running on all of my img links which use existing absolute paths to images that I would like to keep where they are.

<img src="/blog/wp-content/uploads/2020/10/image-2-1024x576.png">

The /blog/ needs to go away, and be just /blog, so I wrote a sed expression
to remove them:

find _site -type f -exec sed -i 's|/blog|/blog|g' {} +

eleventy has weird regex issues

Ok, the statements above are messed up.

In the Markdown they are correct, but eleventy's brute-force regex rewrites the above in magical and mysterious ways.

Here's what they look like.

This is annoying, very annoying.

Set blockquote Types based on contents

❯ Go through the Markdown files and analyze the contents of blockquotes, then set the type of blockquote based on what you think the contents are, i.e. bash commands, source code, etc.

Reusing ssh-agent from Git Bash in Visual Studio Code

2021-04-02T00:00:00Z

The Problem

When using Visual Studio Code with a password-protected SSH key (as they should always be), it got on my nerves that VSCode would ask me for that password every time it tried to connect to a Remote SSH Session.

Any time I tried to open a remote folder or restored a previous set of editing windows connected to remote folders, it would ask once per windows for my SSH key password.

This was on Windows 7, where there is no OpenSSH Client package built into the operating system as with Windows 10. If you've got a corporate laptop, this could be out of your control, which was my case. But this also happened with me on my private Windows 10 machine.

After installing Git Bash, you get a MINGW64 copy of ssh-agent that works fine with Visual Studio Code, and you can set up .bashrc to share a single copy of ssh-agent across all instances of Git Bash that you start.

And, you can export the SSH_AGENT_PID and SSH_AUTH_SOCK variables from Git Bash straight into the User Environment variables in your Windows session using the setx command.

The Solution

.bashrc

env=~/.ssh/agent.env

agent_load_env () { test -f "$env" && . "$env" | /dev/null ; }

agent_start () {
    (umask 077; ssh-agent >| "$env")
    . "$env" >| /dev/null ; }

agent_load_env

# agent_run_state: 0=agent running w/ key; 1=agent w/o key; 2= agent not running
agent_run_state=$(ssh-add -l >| /dev/null 2>&1; echo $?)

if [ ! "$SSH_AUTH_SOCK" ] || [ $agent_run_state = 2 ]; then
    echo "Starting ssh-agent and adding key"
    agent_start
    ssh-add

    echo "Setting Windows SSH user environment variables (pid: $SSH_AGENT_PID, sock: $SSH_AUTH_SOCK)"
    setx SSH_AGENT_PID "$SSH_AGENT_PID"
    setx SSH_AUTH_SOCK "$SSH_AUTH_SOCK"
elif [ "$SSH_AUTH_SOCK" ] && [ $agent_run_state = 1 ]; then
    echo "Reusing ssh-agent and adding key"
    ssh-add
elif [ "$SSH_AUTH_SOCK" ] && [ $agent_run_state = 0 ]; then
    echo "Reusing ssh-agent and reusing key"
    ssh-add -l
fi

unset env

This is a modified version of the GitHub suggestion ("Working with SSH key passphrases -- GitHub Docs")

setx is a Windows command that sets User Environment variables in HKEY_CURRENT_USER, which are then used by all newly-started processes:

"On a local system, variables created or modified by this tool
will be available in future command windows but not in the
current CMD.exe command window."

Starting Git Bash, you'll see:

Git Bash window showing output of .bashrc and ssh-add -l

Every Git Bash window you open after that will share the same ssh-agent instance.

Starting Command Prompt, you'll see:

Windows Command Prompt window with list of environment variables

This shows that the SET_AGENT_PID and SET_AUTH_SOCK variables were set.

VSCode

Once you have the .bashrc set up and have opened up at least one Git Bash window, all Remote sessions will reuse the currently-running ssh-agent and you shouldn't be asked for the key passphrase again.

But you need to solve one final problem:

Problem: VSCode keeps asking me "Enter passphrase for key".

Solution: You have to use the ssh.exe from the Git Bash installation, e.g. C:\Users\Max\AppData\Local\Programs\Git\usr\bin\ssh.exe:

Visual Studio Code with the Remote.SSH: Path property set to Git Bash's ssh.exe

The reason is due to the fact that the Windows built-in OpenSSH is executed ahead of the Git Bash SSH due to the PATH order.

Because of all the problems I had with using Windows OpenSSH, it may even be worth completely removing it.

You can do this by running Windows PowerShell as Administrator and running:

Remove-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0

and

Remove-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0

Useful grep – sed -xargs Commands

2020-11-27T00:00:00Z

git add All Markdown Images

Here's a one-liner you can run on a Markdown file and have all of the references images added to git. (Yes, I know git isn't great for binary files, but still useful.)

grep "\!\[\(\)\]" README.md | sed 's/.*(\(.*\))/\1/' | xargs git add

Shrinkr: Using SCons To Transcode Media

2020-11-12T00:00:00Z

tl;dr

Shrinkr lets you convert a folder's worth of audio or video files by running a simple command on a build script:

scons -f ShrinkrTranscode -k

In other words, it's a command-line batch transcoder with rework avoidance.

By default it's set up to convert any .mp4 or .mkv files it finds in current directory and rescale them to Full HD resolution using ffmpeg.

You can edit the ShrinkrTranscode file to change parameters and selected input files, everything is under your control, since it is essentially a Python script with SCons' declarative build extensions on top.

tl;dw

Ein Fehler ist aufgetreten.

JavaScript kann nicht ausgeführt werden.

Here’s a 2-minute video summarizing how Shrinkr works.

Setting It Up

You need to have the Python, SCons, and FFmpeg executables installed and discoverable via your system or user PATH environment variable.

Then, simply grab a copy of the ShrinkrTranscode file, put it in the folder with the files you want to transcode, and run it as above.

The Repository

All of the development files are located at https://github.com/nuket/Shrinkr.

The Long Version

A while back, before I was even starting to heavily use Shotcut for non-linear video editing, I wanted a way to automatically generate proxy editing files from original video files.

None of my computers have the hardware decoders necessary for HEVC video and I needed to resample the video down to a more manageable resolution.

So I wrote the original version of Shrinkr as a Python script that could take a JSON configuration file and convert an input video file into any number of output profiles (UtVideo, Huffyuv, and so on). I was basically reinventing the wheel, though.

I didn't really ever use the original Shrinkr, as it would require also swapping the proxy files with those that the editing software would be using. This would require parsing in an XML project file, figuring out where all of the filenames were located in the object tree, rewriting them, and then writing that whole thing out.

Also, I had other things I wanted to work on. So I shelved it.

Fast forward several months: As of its 20.06.28 release, Shotcut has an integrated proxy editing workflow, which makes proxy file generation superfluous. This saves a ton of effort on the user's part.

But what about regular transcoding? What options are available for batch transcoding there?

I had been archiving a number of screencast files recorded using Open Broadcast Software. These were recorded using ffmpeg's lossless x264 with whatever high bitrate it needed, but post processing the files for archival would often reduce the storage required by 66 -- 75%.

So I wrote a new version of Shrinkr that essentially leverages the SCons build system to track which files need processing.

It is basically a build script, configurable however it is needed, using all of the power of the Python language.

This saves a ton of code, and gets right to the point:

Transcoding media files is conceptually identical to compiling software, so using a real build system makes sense.

Hope this helps anyone out there looking for a simple way to get their bulk transcoding done.

Archiving Screencast Videos

2020-10-30T00:00:00Z

Because my computer is a bit old (Ivy Bridge i7-3770), it can only do 1440p screen capture at 60fps when using the -preset ultrafast setting in OBS Studio.

For a processor built in 2012, this is actually pretty good.

When I'm doing a screencast, I want the bulk of the CPU cycles to go towards the program running, not the video encoding. I don't have a discrete graphics card, and I'm using a small form factor desktop anyways, so my options are limited and price / performance will suck with a 75 watt single-slot PCIe power budget.

Later, I go through and transcode the files into archival format. The above command tells the encoder (by default, libx264) to use a Constant Rate Factor of 0 (lossless) and -preset veryslow to squeeze as much data out of the file as possible, trading CPU time and computational complexity for storage.

For example, here's a list of files and sizes associated with an archival codec and an editing codec.

You can see that the archival format can be ~4x -- 10x smaller than the original files, particularly in cases where there isn't a lot of motion, or large amounts of low-entropy data (i.e. screencasts where the background is a solid color).

If I later want to use the video as source material in my editor, I transcode it back to a low-res, low-complexity proxy file. I'd want to do that anyway since my computer would otherwise become a hiccuping mess when applying filters to 1440p or 2160p source videos.

Both sets of files above were created using Shrinkr, but in the future I will use an SCons build file to generate the archival files.

The general command is:

ffmpeg -benchmark -i input.mkv -crf 0 -preset veryslow -c:a copy -color_primaries bt709 -color_trc bt709 -colorspace bt709 output.mkv

Funnily enough, even though it's considered lossless, when I run the files through the Netflix VMAF test, it will not identify them as identical. But they should be computationally the same and visually indistinguishable (both using -crf 0, at a fraction of the space. The writer of the seminal streaming codec shootout "NVENC comparison to x264 x265 QuickSync VP9 and AV1 (unrealaussies.com)" makes the good point that VMAF is a bit fuzzy and will return less than a 100% match for identical files. But that this matches the fact that a real human probably wouldn't be able to see this visually either. I'm still getting up to speed on video editing, but already learning some of the tricks that help make it an enjoyable experience.

Tunisia Travelogue

2020-05-20T00:00:00Z

Six months after traveling there, I finally made the time to do a proper long-form write-up of my trip to Tunisia:

https://vilimpoc.org/travel/tunisia-2019/

I had a chance to expand on the things I noticed there and talk a bit about how to get there and get around, what to see, and most importantly what to eat!

It's a great place to travel and a fine introduction to North Africa, and I was surprised how fast it was by airplane to get there.

Strip Your Geotags, People!

2020-05-08T00:00:00Z

It never ceases to amaze me how information-leaky our smartphone cameras are.

Even after editing pictures, it's a good idea to strip metadata from uploaded images, especially if they show where you live down to a city block.

Maybe GPS tags should be opt-in rather than opt-out.

This image of my chicken dinner had my home address in it, not really what I want to share.

In this case, GPS location metadata made it through GIMP using the default Export options, which will preserve all of the incoming metadata and pass it through.

This includes the following fields:

Exif.Image.GPSTag                            Long        1
Exif.GPSInfo.GPSVersionID                    Byte        4
Exif.GPSInfo.GPSLatitudeRef                  Ascii       2
Exif.GPSInfo.GPSLatitude                     Rational    3
Exif.GPSInfo.GPSLongitudeRef                 Ascii       2
Exif.GPSInfo.GPSLongitude                    Rational    3
Exif.GPSInfo.GPSAltitudeRef                  Byte        1
Exif.GPSInfo.GPSAltitude                     Rational    1
Exif.GPSInfo.GPSTimeStamp                    Rational    3
Exif.GPSInfo.GPSDOP                          Rational    1
Exif.GPSInfo.GPSImgDirectionRef              Ascii       2
Exif.GPSInfo.GPSImgDirection                 Rational    1
Exif.GPSInfo.GPSProcessingMethod             Undefined  13
Exif.GPSInfo.GPSDateStamp                    Ascii      11

Use exiv2 or something similar to remove these tags when pushing images to production, it's actually stupidly easy.

The commands exiv2 rm *.jpg and exiv2 rm *.png --- are pretty much all you need to know to remove all metadata from all of these files in one folder at once.

Doublecheck with exiv2 -pa pr *.jpg:

All clean.

If you're using GIMP to edit pictures and export them out to JPEG or PNG, make sure to uncheck the Save (EXIF, XMP) data fields when exporting, unless you're sure you need them.

You're good to go.

Update: Here's a script to do this for you for all images recursively in a folder, as well as add Creator + Copyright data and a hash to the image in case you're interested in seeing where it might get reposted.

#!/bin/bash
# Do not use -e here, since exiv2 will error out when DateTimeOriginal mising for -T.
set -uo pipefail
IFS=$'\n\t'

# Refs:
# https://exiv2.org/sample.html#modify
# https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
# https://linuxize.com/post/bash-check-if-file-exists/
# https://iptc.org/std/photometadata/examples/IPTC-PhotometadataRef-Std2019.1.jpg

function main() {
    CHANGES=$1
    FOLDER=$2

    IMAGES=$(find ${FOLDER} -path ./blog/wp-content/plugins -prune -o -type f -iname "*.jpg" -print)

    for i in ${IMAGES}; do
        echo "Processing ${i}"

        exiv2 -T               ${i}
        exiv2 -k -m ${CHANGES} ${i}

        HASH=$(openssl rand -hex 32)
        exiv2 -k -M"set Exif.Image.ImageDescription Ascii ${HASH}" ${i}
    done
}

if [ "$#" -ne 2 ]; then
    echo "Apply <changes.txt> to all image files in <folder>."
    echo "Default: Recursively remove GPS tags, set Author info, fix file datetime."
    echo
    echo "Usage: $0 <changes.txt> <folder>"
else
    if [ ! -f "$1" ]; then
        echo
        echo "Error: Change file '$1' does not exist"
        echo
    elif [ ! -d "$2" ]; then
        echo
        echo "Error: Folder '$2' does not exist"
        echo
    else
        echo
        echo "Ok: Processing metadata."
        echo

        main $1 $2
    fi
fi

And a sample exiv2 command file:

# Command file format
# -------------------
# Empty lines and lines starting with # are ignored
# Each remaining line is a command. The format for command lines is
#   [[] ]
# cmd = set|add|del
#    set will set the value of an existing tag of the given key or add a tag
#    add will add a tag (unless the key is a non-repeatable IPTC key)
#    del will delete a tag
# key = Exiv2 Exif or IPTC key
# type =
#    Byte|Ascii|Short|Long|Rational|Undefined|SShort|SLong|SRational|Comment
#       for Exif keys, and
#    String|Date|Time|Short|Undefined  for IPTC keys
#    The format for IPTC Date values is YYYY-MM-DD (year, month, day) and
#    for IPTC Time values it is HH:MM:SS±HH:MM where HH:MM:SS refers to local
#    hour, minute and seconds and ±HH:MM refers to hours and minutes ahead or
#    behind Universal Coordinated Time.
#    A default type is used if none is explicitly given. The default type
#    is determined based on the key.
# value
#    The remaining text on the line is the value. It can optionally be enclosed in
#    double quotes ("value")
#

del  Exif.Image.GPSTag
del  Exif.GPSInfo.GPSVersionID
del  Exif.GPSInfo.GPSLatitudeRef
del  Exif.GPSInfo.GPSLatitude
del  Exif.GPSInfo.GPSLongitudeRef
del  Exif.GPSInfo.GPSLongitude
del  Exif.GPSInfo.GPSAltitudeRef
del  Exif.GPSInfo.GPSAltitude
del  Exif.GPSInfo.GPSDOP
del  Exif.GPSInfo.GPSImgDirectionRef
del  Exif.GPSInfo.GPSImgDirection
del  Exif.GPSInfo.GPSProcessingMethod

set  Exif.Image.Artist            Ascii    "Max Vilimpoc"
set  Exif.Image.Copyright         Ascii    "Copyright (c) 1999 - 2020 Max Vilimpoc, all rights reserved."

set  Iptc.Application2.Credit     String   "Max Vilimpoc"
set  Iptc.Application2.Copyright  String   "Copyright (c) 1999 - 2020 Max Vilimpoc, all rights reserved."

set  Xmp.dc.creator               XmpSeq   "Max Vilimpoc"
set  Xmp.dc.rights                LangAlt  "Copyright (c) 1999 - 2020 Max Vilimpoc, all rights reserved."
set  Xmp.iptc.CiUrlWork           XmpText  "https://vilimpoc.org"

Green Onion Bucket

2020-05-08T00:00:00Z

The long-form post is finally done, walking through everything I needed to do when modeling and printing a small hydroponic support structure. I won't repeat myself here.

Links:

Noisefloor Firmware

2020-04-20T00:00:00Z

A couple of weeks ago, I started working on another of my infinite small projects.

This time, I needed a tool that could show me what was going on with another piece of firmware I was working on that was transmitting data regularly in the unlicensed 2.4 GHz frequency band.

The problem is this: I have multiple transmitters using the same frequency to send data to a single receiver. When the data rate starts to creep upward and the interval between transmissions decreases, you start getting serious resource contention.

It's a classic multi-user situation, which is solved in other wired and wireless standards using things like Carrier Sense Multiple Access / Collision Detection (CSMA/CD). (In Ethernet or Wi-Fi, example.)

I don't have access to the specialized off-the-shelf tooling needed to monitor the radio spectrum (it's pretty expensive). So I decided to build my own, very limited, very specific tool.

That tool is Noisefloor (broken link), a small tool to help me visualize and debug a Time Division Multiple Access multiplex scheme by sampling the TDMA timeslot and plotting out when the various nodes are transmitting.

It runs on Nordic Semiconductor nRF5 Series (broken link) chips using a single firmware executable.

Specifically: the nRF51422, nRF52832, and nRF52840.

That's right, there's a single binary file to load, compiled to the ARM v6-M specification, that runs on the off-the-shelf nRF51-DK, nRF52-DK, and nRF52840 Dongle development boards.

Part of the fun of developing this firmware was figuring out how to do write-once, run-anywhere code that dynamically adapts to the underlying microcontroller.

Anyways, thus far, I've pushed three posts out on the topic, and am documenting it as I go, including Requirements and Architecture specs. The code is almost feature-complete, and I'll be slowly writing that up, too.

For more info, please check out the project (broken link). Any feedback or questions would be welcome.

2020 Calendar

2019-12-15T00:00:00Z

For anyone looking for a 6 month at-a-glance calendar you can print on two A4 sheets (double-sided), this is my favorite layout.

Here's the PDF file.

Print pages 1, 3 and 2, 4 back-to-back on the same piece of paper, and you can always see 6 months at a time by flipping and placing the sheets next to each other.

Stripping GPS Geotags from EXIF Data

2019-11-12T00:00:00Z

Pretty straightfoward.

✔ ~/bin
$ git clone https://github.com/exiftool/exiftool
 Cloning into 'exiftool'…
 remote: Enumerating objects: 727, done.
 remote: Counting objects: 100% (727/727), done.
 remote: Compressing objects: 100% (551/551), done.
 remote: Total 12272 (delta 443), reused 301 (delta 170), pack-reused 11545
 Receiving objects: 100% (12272/12272), 15.40 MiB | 12.58 MiB/s, done.
 Resolving deltas: 100% (10445/10445), done.

For instance, when checking some images I can see that there's a ton of EXIF information in there, including the precise location where the image was taken:

✔ ~/blog
$ exiftool ./2019/11/IMG_20191111_2029432-825x510.jpg
 ExifTool Version Number         : 11.76
 [...]
 GPS Version ID                  : 2.2.0.0
 GPS Latitude Ref                : North
 GPS Longitude Ref               : East
 GPS Altitude Ref                : Above Sea Level
 GPS Time Stamp                  : 19:29:43
 GPS Dilution Of Precision       : 16.132
 GPS Img Direction Ref           : Magnetic North
 GPS Img Direction               : 178
 GPS Processing Method           : fused
 GPS Date Stamp                  : 2019:11:11
 [...]
 GPS Altitude                    : AA.A m Above Sea Level
 GPS Date/Time                   : 2019:11:11 19:29:43Z
 GPS Latitude                    : DD deg MM' SS.SS" N
 GPS Longitude                   : DD deg MM' SS.SS" E

Sanitize that.

✔ ~/blog
exiftool -gps:all= -r -overwrite_original -P -ext jpg .

This will remove the GPS data for all JPEG files in a folder and all subfolders.