It appears that, with some effort, you do not need more than an 8-bit chip - despite the 32-bit requirement - 25-year old RAM and half a gigabyte of storage. It's not what you would call a particularly fast system, but it is astonishing that Grinberg got it to work at all.
He used an Atmel ATmega1284p RISC-based microcontroller with 128 KB ISP flash memory, 4 KB EEPROM, and 16 KB SRAM that runs at 20 MHz off-the shelf, but was overclocked to 24 MHz in Grinberg's case. The chip delivers a total of about 24 MIPS. To support the booting process and store Ubuntu, the developer added a 1 GB SD card as well as a 30-pin SDRAM SIMM that was common in 286-computers in the late 1980s and delivers a data throughput of about 300 KB/s.
Grinberg programmed an ARM emulator for the ATmega1284p to boot Linux (kernel 2.6.34), which decreased the effective emulated clock speed of the chip dramatically and ended up at about 6.5 KHz. The boot process took about two hours. the developer noted that the system is "somewhat usable". Typed commands deliver replies within a minute, he said. The overall result is the "cheapest, slowest, simplest to hand assemble, lowest part count, and lowest-end Linux PC," he wrote in a post detailing the system.