
 About
 *****

 Behold: the glorious, self booting ZIP archive!

 An archive that can be loaded/run by a multiboot compliant boot loader,
 including qemu direct kernel boot:

 $ qemu-system-i386 -kernel boot.zip


 It can also be dumped directly onto a disk and booted by a PC BIOS, as it
 contains a valid boot sector and stub loader that implements the necessary
 parts of the multiboot protocol itself:

 $ qemu-system-i386 -fda boot.zip


 Or on a hard disk, the boot code uses the INT 13h parameters for the BIOS
 disk it was loaded from:

 $ qemu-system-i386 -hda boot.zip


 The archive has no hidden parts, no gaps between entries, no obvious polyglot
 trickery.

 The source code for the ZIP archive is contained in the archive itself, the
 shell script `mk.sh` builds the whole thing from scratch:

 $ chmod +x mk.sh
 $ ./mk.sh

 Zlib is required to compile some utility programs.


 Besides qemu, Bochs and 86box, it has been successfully tested on real
 hardware (Pentium MMX, 486DX2, Am386DX-40).


 On the nature of the ZIP Archives
 *********************************

                       ,------------------.
              ,-------|-----------------. |
    .--------|--------|---------------. | |
    V________V________V__________   __|_|_| ________
   |H|      |H|      |H|      | /   \|H|H|H|        |
   |D| File |D| File |D| File | \   /|D|D|D| Footer |
   |R|______|R|______|R|______|_/   \|R|R|R|________|
                                     A        |
                                     `--------'
                                     |_____|
                                        |
                                 central directory

 It is well known that the ZIP format is designed to be read from the back,
 allowing a different format (read from the front) to be combined into a
 polyglot file.

 In short: an archive contains a number of compressed blobs, each with a "local
 file header" in front. At the end, there is a central directory with a full
 listing and back references to the local headers, followed by a footer that
 points to the start of the directory. This allows constructing the archive in
 one pass and fast seeking to files within the archive.


 Because of a number of design flaws and resulting foot guns (e.g. there is a
 variable sized comment field in the footer), modern implementations typically
 prefer to do a linear scan from the front anyway, hopping from header to
 header, and rejecting any archive where this doesn't work.


 For the self booting ZIP archive, the challenge now becomes to pack
 *everything* into ZIP file entries and leave no gaps, creating a genuine
 ZIP archive that just so happens to be bootable.


 Embedding a Multiboot kernel in a ZIP archive
 *********************************************

 This part is trivial. The multiboot specification requires a header with a
 magic value, at a 4 byte aligned location _somewhere_ within the first 8k of
 the kernel image.

 The specification says that the kernel image can be an executable format
 like ELF, but the header can also just specify the memory layout manually.

 The compliant loader works out the file offsets, loads the thing into memory,
 and jumps into it, with the CPU in protected mode, 32 bit code, flat segments,
 A20 enabled and an informational structure set up for the kernel.


 All that we need to do to make a ZIP multiboot bootable, is to create an
 uncompressed file, containing a flat binary and a multiboot header. We need
 to micro manage the location so the addresses line up, and adjust the header
 after the fact, to encompass the full archive.


 Making it BIOS bootable
 ***********************

 The PC BIOS boot process is fairly simplistic. It iterates over the disk
 drives, loads the first sector at 0x7c00. If the last two bytes are 0xaa55
 (encoded as little endian), it jumps to the first byte.


 All we need to do is to create a flat binary that is stored at the beginning
 of the archive and has the magic signature at offset +510. The BIOS will load
 the first sector and execute it.

 The problem is this: To satisfy modern archivers, we MUST have a ZIP local
 header at offset 0, the BIOS will now EXECUTE THE HEADER!

 We need to craft a ZIP local file header that is both valid, and only consists
 of 8086 opcodes without negative side effects. At least up to a field where we
 can squeeze a jump instruction in and skip the rest.

 We can fill the rest of the sector with 16 bit setup code (CPU & memory
 detection, A20 enable, protected mode setup), load the rest of the archive
 via INT 13h into high memory, setup the multiboot structures ourselves and
 jump there.


 Putting it all together
 ***********************

 What this project pieces together looks something like this:

      __________________   _                  __
   0 |  ZIP local file  |   |                   |
     |______header______|   |                   |
     |_Multiboot_header_|   |                   |
     |                  |    \ Loaded by BIOS   |
     |   16 bit boot    |    / to 0000:7c00     |
     |   sector code    |   |                   |
     |__________________|   |                   |
+510 |____BIOS_MAGIC____|  _|                   |
+512 |                  |   |                   |
     |   16 bit code    |   |                   |
     |__________________|    \  Loaded by us    |
     |                  |    /  to 0000:7e00    |
     |   32 bit code    |   |                   |
     |. . . . . . . . . |  _|                   |
     |                  |                       |
     |   32 bit code    |                       |
     |                  |                       |
      >================<                        |
     |  ZIP local file  |                       |  Loaded by multiboot loader
     |______header______|                        > or by the 16 bit code to
     |                  |                       |  0x00100000 in high memory
     |       DATA       |                       |
      >================<                        |
     |  ZIP local file  |                       |
     |______header______|                       |
     |                  |                       |
     |       DATA       |                       |
      >================<                        |
     |                  |                       |
     |/\/\/\/\/\/\/\/\/\|                       |
                                                |
      /\/\/\/\/\/\/\/\/\                        |
     |                  |                       |
      >================<                        |
     |     Central      |                       |
     |    Directory     |                       |
      >================<                        |
     |                  |                       |
     |      Footer      |                       |
     |__________________|                     __|


 As far as a ZIP archiver is concerned, the first part is a completely normal,
 uncompressed file in the archive.

 If a multiboot compliant loader (e.g. GRUB, qemu) sees the file, it will find
 the multiboot magic, load the entire thing into _high memory_, and jump
 directly to the 32 bit entry point. The environment is 32 bit protected mode,
 flag segments, A20 enabled. We load our *own* GDT, setup a stack (the
 multiboot header tells where the archive ends) and jump into the C code.

 In the "high" level C code, we dump the information from the bootloader,
 scan the archive in memory to access files.


 If the file is dumped onto a disk, the BIOS loads the first 512 bytes, sees
 the BIOS MAGIC, starts executing the header. The 16 bit loader itself has
 additional code past the first sector. It performs the following steps:
  - Initialization (setup segments, clear scratch memory)
  - Check that we have at least an i386, so we can actually run 32 bit code
  - Query disk geometry for LBA -> C/H/S mapping
  - Load the rest of the boot code
  - Detect available memory (INT 12h, INT 15h 88/E801/E820)
  - Turn on the A20 gate
     - Try BIOS INT 15h
     - Try talking to the keyboard controller directly
     - Try SYSCTL A register
  - Set up 16 bit unreal mode, so we can actually access high memory
  - Load the rest of the archive into HIMEM (size is in the multiboot header)
  - Enter 32 bit protected mode, cobble the rest of the multiboot info struct
    together and jump to the 32 bit entry point (see above).



 This might sound like a lot, but all of the assembly code, including
 the 32 bit entry point, i.e. everything before we enter the C code, currently
 fits neatly into 817 bytes. And that _includes_ 48 bytes of ZIP header,
 32 bytes multiboot header and 40 bytes of GDT, the boot signature and some
 padding bytes.


 Some linker script magic is required to tie it all together into one binary
 blob and to share the multiboot header & GDT between the lowmem and HIMEM
 code.


 Structure of the Code
 *********************

 All of the low level boostrap code, along with the linker script, is in
 the `setup` directory.

 You might want to look at the linker script (`setup.ld`) first, to clarify a
 few things. It specifies several magic symbols that the assembly uses,
 including scratch memory layout at 0000:0600.

 The linker script also adds the BIOS boot signature AND THE GDT (in case you
 were wondering where the hell the GDT was defined).


 Following this, you might want to look at things in the following order:

  - entry16.S
     Contains the executable ZIP header, the multiboot header,
     the 16 bit point for the BIOS that directs the system bring up.

  - disk.S
     Utilities for BIOS disk I/O

  - mem.S
     Memory detection

  - a20.S
     A20 bring up, didn't fit into the boot sector anymore :-(

  - mbload.S
     Contains the code for loading the archive and jumping into highmem.
     Disk sectors are loaded into a bounce buffer and copied up. Lastly,
     the 32 bit code is entered with magic values and multiboot info
     struct in EAX/EBX.

  - entry32.S
     Contains the 32 bit entry point that is to be loaded into HIMEM. This code
     can either be reached via mbload.S or directly from a multiboot compliant
     boot loader.



 The "high" level multiboot kernel code lives in the `multiboot` directory:

  - multiboot.c
     Contains the main function that is called from entry32.S

  - console.c/console.h
     A simplistic framebuffer console implementation

  - string.S
     Some libc helper functions from string.h


 The low level and high level code are compiled into a single blob with a ZIP
 header in the front.

 To puzzle together the ZIP archive, three helper programs are built from files
 in the `util` sub directory:

  - fixhdr.c
     A program that mmap()s an input file and either finds and fixes the
     multiboot header (adjusting load size), or updates the checksum
     in a ZIP local file header (crc32 for the payload data).

     If the multiboot header is updated, the enclosing ZIP local file header
     needs to be found, its checksum updated and the same fixup has to be
     carried over to the central directory at the end, that contains a copy of
     the checksum.

  - zipglue.c
     Given several files with ZIP local headers in the front, concatenate them
     into a single blob, collect the local headers as we go, generate a central
     directory and footer once everything is done.

  - zippack.c
     Compress an input file and prefix it with a ZIP local file header. Path in
     the ZIP file can be specified, compression can be disabled.

 Additional headers, shared between the components are in `common`.
