I booted my Series 3 Chromebook this afternoon to a fun challenge: no init to run. This post is a short story of how I worked my way back in without resorting to using an install/rescue medium.
'cros_legacy' seems to be the default init specified in the kernel command line and 'daisy' is defaulted to when you remove this, but neither existed in my filesystem:
$this->bbcode_second_pass_code('', '/bin/sh: cros_legacy: No such file or directory
...kernel panic, attempted to kill init...')
If you rapidly mash keys right after pressing Ctrl+d you can get access to the u-boot derived bootloader:
$this->bbcode_second_pass_code('', 'Model: Google Snow
In: mkbp-keyb
Out: lcd
Err: lcd
Net: No ethernet found.
Hit any key to stop autoboot: 0
SMDK5250 #')
Most of us have (hopefully!) used this to remove the 'debug' keyword from our kernel command line but we can also do some evil through here. Namely: telling the kernel to use something else as init, such as a shell:
$this->bbcode_second_pass_code('', ' # env set common_bootargs /bin/sh ')
It's worth considering the commonly used 'break' kernel command-line argument which normally drops you to a shell to rescue things. Unfortunately this argument is handled by the init system, not by the kernel, so it's not any use here.
Sidenote: If you want to see what other variables you have to play with: try an env print. By some dark magic all of the variables fit exactly onto one 768 pixel high screen-height, otherwise the lack of scrolling keys on this hardware would leave you in the dark.
Sidenote: Variables can either be normal values or lists of commands: there is no distinction in this u-boot derived environment. You can recognise the ones intended as command lists by the use of semi-colons.
I now run the variable that's already setup to boot from the first partition of my internal flash (your choice may vary):
$this->bbcode_second_pass_code('', ' # run mmc0_boot
... booting ...
/bin/sh: /bin/sh: cannot execute binary file
* kernel panic *')
What? Have I accidentally installed x86 binaries? Luckily not -- it seems that the init file is trying to be intepreted by a shell rather than executed by the kernel. You can't interpret a normal binary like a shell script. But I also don't know of any shell scripts lying around on my laptop that will come in handy to run a rescue shell or similar.
My first attempt: write a shell script to a usb stick using 'dd'. It will show up as /dev/sda to the laptop's kernel.
$this->bbcode_second_pass_code('', '$ echo -e '#!/bin/sh \nexec sh -i' > file
$ dd if=file of=/dev/myusbstick ')
$this->bbcode_second_pass_code('', '# env set common_bootargs /dev/sda
# run mmc0_boot
*boot messages*
*detect USB device...*
....could not find /dev/sda... (to run as init)
Kernel panic, attempted to kill init
')
This almost worked! Unfortunately the kernel was just slightly too fast in trying to run init, only just detecting my USB stick and not yet discovering it has a block device at the other end of it. I confirmed this by telling the kernel to run 'startx' as init, which although guaranteed to crash (by default runs an xterm, I only have urxvt installed) kept the system running just a few split seconds long enough to see the kernel messages for enumerating /dev/sda.
When in doubt, use a more forceful hammer
My chromebook installation would be littered with shell scripts installed as components of various programs and utilities in various packages. I was sure some of them could somehow be exploited to give me access to a rescue shell, but I didn't know which ones. Luckily I have a desktop running (x86) Arch which in turn has a very similar selection of package-installed shell-scripts.
$this->bbcode_second_pass_code('', '$ file /bin/* | grep shell | less
*many, many files*
')
Most of these however are completely useless: without any command line arguments passed to them they give you a usage or error message and then quit. There is probably a way of giving init command line arguments, but my DuckDuck-foo was not good enough to find it.
Time for some automation!
$this->bbcode_second_pass_code('', ' for exec in $(file /bin/* | grep shell | cut -f1 -d':'); do echo "### Trying $exec"; sh $exec; done ')
A few seconds later my screen fills with a vim session (copied here from my x86 desktop, not the Chromebook):
$this->bbcode_second_pass_code('', 'From: valentine
To: bug-bash@gnu.org
Subject: [50 character or so descriptive subject here (for reference)]
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' -DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/bin' -DSTANDARD_UTILS_PATH='/usr/bin' -DSYS_BASHRC='/etc/bash.bashrc' -DSYS_BASH_LOGOUT='/etc/bash.bash_logout'
uname output: Linux Shyarenris 3.17.6-1-ARCH #1 SMP PREEMPT Sun Dec 7 23:43:32 UTC 2014 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu
Bash Version: 4.3
Patch Level: 30
Release Status: release
Description:
[Detailed description of the problem, suggestion, or complaint.]
Repeat-By:
[Describe the sequence of events that causes the problem
to occur.]
Fix:
[Description of how to fix the problem. If you don't know a
fix for the problem, don't include this section.]
~
~
~
~
~
~
~
~
~
"/tmp/bbug.2566/bbug1" 26L, 1246C 1,1 All
')
I've hit gold! A shell script called bashbug runs a session of something along the lines of your default editor (vi if not fiddled with). A working text editor is more useful than a working operating system. Especially in vi/vim, where you can execute commands such as /bin/bash.
Summary of the solution:
$this->bbcode_second_pass_code('', 'Press power button
Hit Ctrl+D at scary screen
Mash spacebar until u-boot prompt appears
# env set common_bootargs /usr/bin/bashbug
# run mmc0_boot
Wait for vi to open up a pre-written bug-report template (thankyou bash devs!)
Enter in :!/bin/sh
Profit!
')
I have not yet fixed my system, but who cares! This was fun I can add it to my /etc/issue along with all of my other catastrophic failures over the last year and a bit:
$this->bbcode_second_pass_quote('', 'W')elcome to Clusterlizard, chamber \l. Please praise your selected deity(s)
Summary of interesting failure stories since December 2013:
- 'debug' on kernel command line -> systemd becomes very verbose -> random race conditions due to text output delays
- filesystem death on SSD by power-cycling too rapidly too many times when trying to debug a problem
- btrfs failure from trying to compile firefox ("ran out of inodes", even though btrfs does not use inodes)
- systemd update -> required kernel support for xattrs on folders -> boot hung
- cracked the screen by carrying in my bag
- systemd update -> prevented kernel from loading userspace wifi firmware
Happy birthday Clusterlizard! 2014-12