Skip to content

Commit

Permalink
cleanup makefile, readme, and warnings
Browse files Browse the repository at this point in the history
  • Loading branch information
@kevinaboos
kevinaboos committed Aug 31, 2020
1 parent 4ebe09a commit cf45c46
Show file tree
Hide file tree
Showing 15 changed files with 155 additions and 156 deletions.
36 changes: 16 additions & 20 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ APP_CRATE_NAMES += getopts unicode_width
### Most targets are PHONY because cargo itself handles whether or not to rebuild the Rust code base.
.PHONY: all \
check_rustc check_xargo check_captain \
clean run run_pause iso build userspace cargo \
clean run run_pause iso build cargo \
simd_personality_sse build_sse simd_personality_avx build_avx \
$(assembly_source_files) \
gdb doc docs view-doc view-docs
Expand All @@ -162,7 +162,6 @@ check_captain:


### This target builds an .iso OS image from all of the compiled crates.
### It skips building userspace for now, but you can add it back in by adding "userspace" to the line below.
$(iso): build check_captain
# after building kernel and application modules, copy the kernel boot image files
@mkdir -p $(GRUB_ISOFILES)/boot/grub
Expand Down Expand Up @@ -276,16 +275,16 @@ $(NANO_CORE_BUILD_DIR)/boot/$(ARCH)/%.o: $(NANO_CORE_SRC_DIR)/boot/arch_$(ARCH)/



## (This is currently not used in Theseus, since we don't run anything in userspace)
## This builds all userspace programs
userspace:
@echo -e "\n======== BUILDING USERSPACE ========"
@$(MAKE) -C userspace all
## copy userspace binary files and add the __u_ prefix
@mkdir -p $(GRUB_ISOFILES)/modules
@for f in `find $(ROOT_DIR)/userspace/build -type f` ; do \
cp -vf $${f} $(GRUB_ISOFILES)/modules/`basename $${f} | sed -n -e 's/\(.*\)/__u_\1/p'` 2> /dev/null ; \
done
# ## (This is currently not used in Theseus, since we don't run anything in userspace)
# ## This builds all userspace programs
# userspace:
# @echo -e "\n======== BUILDING USERSPACE ========"
# @$(MAKE) -C old_crates/userspace all
# ## copy userspace binary files and add the __u_ prefix
# @mkdir -p $(GRUB_ISOFILES)/modules
# @for f in `find $(ROOT_DIR)/old_crates/userspace/build -type f` ; do \
# cp -vf $${f} $(GRUB_ISOFILES)/modules/`basename $${f} | sed -n -e 's/\(.*\)/__u_\1/p'` 2> /dev/null ; \
# done



Expand Down Expand Up @@ -436,7 +435,6 @@ view-book: book
clean:
cargo clean
@rm -rf build
#@$(MAKE) -C userspace clean



Expand Down Expand Up @@ -529,18 +527,18 @@ help:
###################################################################################################
QEMU_MEMORY ?= 512M
QEMU_FLAGS := -cdrom $(iso) -no-reboot -no-shutdown -s -m $(QEMU_MEMORY) -serial stdio

## multicore
QEMU_FLAGS += -smp 4
QEMU_CPUS ?= 4
QEMU_FLAGS += -smp $(QEMU_CPUS)

## QEMU's OUI dictates that the MAC addr start with "52:54:00:"
MAC_ADDR ?= 52:54:00:d1:55:01

## Add a disk drive, a PATA drive over an IDE controller interface.
QEMU_FLAGS += -drive format=raw,file=random_data2.img,if=ide

# QEMU_FLAGS += -drive format=raw,file=DISK_IMAGE.img,if=ide
## Add a disk drive, a SATA drive over the AHCI interface.
## We don't yet have SATA support in Theseus.
# QEMU_FLAGS += -drive id=my_disk,file=random_data2.img,if=none -device ahci,id=ahci -device ide-drive,drive=my_disk,bus=ahci.0
# QEMU_FLAGS += -drive id=my_disk,file=DISK_IMAGE.img,if=none -device ahci,id=ahci -device ide-drive,drive=my_disk,bus=ahci.0

## Read about QEMU networking options here: https://www.qemu.org/2018/05/31/nic-parameter/
ifeq ($(net),user)
Expand Down Expand Up @@ -601,7 +599,6 @@ loadable: run

### builds and runs Theseus in QEMU
run: $(iso)
# @qemu-img resize random_data2.img 100K
qemu-system-x86_64 $(QEMU_FLAGS)


Expand All @@ -622,7 +619,6 @@ gdb:
### builds and runs Theseus in Bochs
bochs : export override THESEUS_CONFIG += apic_timer_fixed
bochs: $(iso)
# @qemu-img resize random_data2.img 100K
bochs -f bochsrc.txt -q


Expand Down
137 changes: 21 additions & 116 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
# Theseus OS

Theseus is a new runtime-morphable OS with a variety of design goals, but primarily we tackle the problem of *state spill*, the harmful yet ubiquitous phenomenon described in our [research paper from EuroSys 2017 here](http://kevinaboos.web.rice.edu/statespy.html).
Theseus is a new OS written from scratch in [Rust](https://www.rust-lang.org/) to experiment with novel OS structure, better state management, and how to shift OS responsibilities like resource management into the compiler.

We have designed and built Theseus from scratch using Rust to completely rethink state management in an OS, with the intention of avoiding state spill or mitigating its effects to the fullest extent possible.

More details are provided in Theseus's [documentation](#Documentation), especially its book.
We are continually working to improve the OS, including its fault recovery abilities for higher system availability without redundancy, as well as easier and more arbitrary live evolution and runtime flexbility.
Though still an incomplete prototype, we envision that Theseus will be useful for high-end embedded systems or edge datacenter environments.
See our [published papers](http://kevinaboos.web.rice.edu/publications.html) for more information about Theseus's design principles and implementation philosophy, as well as our goal to avoid the phenomenon of *state spill* or mitigate its effects as much as possible.
Also, see Theseus's [documentation](#Documentation) for more.



## Building Theseus

Currently, we support building and running Theseus on the following host OSes:
* Linux, 64-bit Debian-based distributions like Ubuntu, tested on Ubuntu 16.04 and 18.04.
* Windows, using the Windows Subsystem for Linux (WSL), tested on the Ubuntu version.
* MacOS, tested on versions High Sierra (10.13) and Catalina (10.15.2), but likely works on others.
* Linux, 64-bit Debian-based distributions like Ubuntu, tested on Ubuntu 16.04, 18.04, 20.04.
* Windows, using the Windows Subsystem for Linux (WSL), tested on the Ubuntu version of WSL and WSL2.
* MacOS, tested on versions High Sierra (10.13) and Catalina (10.15.2).


### Setting up the build environment
Expand Down Expand Up @@ -54,29 +55,29 @@ To build and run Theseus in QEMU, simply run:
Run `make help` to see other make targets.


#### Note: Rust compiler versions
Because we use the Rust nightly compiler (not stable), the Theseus Makefile checks to make sure that you're using the same version of Rust that we are. We were inspired to add this safety check when we failed to build other Rust projects put out there on Github because they used an earlier version of the nightly Rust compiler than what we had installed on our systems. To avoid this undiagnosable problem, we force you to use a specific version of `rustc` that is known to properly build Theseus. This version is upgraded as often as possible to align with the latest Rust nightly, but this is a best-effort policy.
### Note: Rust compiler versions
Because we use the Rust nightly compiler (not stable), the Theseus Makefile checks to make sure that you're using the same version of Rust that we are. We were inspired to add this safety check when we failed to build other Rust projects put out there on Github because they used an earlier version of the nightly Rust compiler than what we had installed on our systems. To avoid this undiagnosable problem, we force you to use a specific version of `rustc` that is known to properly build Theseus; we also use the standard `rust-toolchain` file to ensure this. The Rust version is upgraded as often as possible to align with the latest Rust nightly, but this is a best-effort policy.

So, if you see a build error about the improper version of `rustc`, follow the instructions printed out at the end of the error message.
As such, if you see a build error about the improper version of `rustc`, follow the instructions printed out at the end of the error message.


## Using QEMU
QEMU allows us to run Theseus quickly and easily in its own virtual machine, completely segregated from the host machine and OS.
To exit Theseus in QEMU, press `Ctrl+Alt` (`Ctrl+Alt+G` on Mac OS), which releases your keyboard focus from the QEMU window. Then press `Ctrl+C` in the terminal window that you ran `make run` from originally to kill QEMU.
QEMU allows us to run Theseus quickly and easily in its own virtual machine.
To exit Theseus in QEMU, press `Ctrl+Alt` (or `Ctrl+Alt+G` on some systems, e.g., Mac OS), which releases your keyboard and mouse focus from the QEMU window. Then press `Ctrl+C` in the terminal window that you ran `make run` from originally to kill QEMU, or you can also quit QEMU using the GUI `(x)` button.

To investigate the state of the running QEMU entity, you can switch to the QEMU console by pressing `Ctrl+Alt+2`. Switch back to the main window with `Ctrl+Alt+1`. On Mac, manually select `VGA` or `compact_monitor0` under `View` from the QEMU menu bar.
To investigate the hardware/machine state of the running QEMU VM, you can switch to the QEMU console by pressing `Ctrl+Alt+2`. Switch back to the main window with `Ctrl+Alt+1`. On Mac, manually select `VGA` or `compact_monitor0` under `View` from the QEMU menu bar.

### KVM Support
While not strictly required, KVM will speed up the execution of QEMU.
To install KVM, run the following command:
`sudo apt-get install kvm`.
To enable KVM support, add `host=yes` to your make command, e.g.,
`make run host=yes`
Currently this option is only supported on native Linux hosts.
This option is only supported on native Linux hosts.


## Debugging
GDB has built-in support for QEMU, but it doesn't play nicely with OSes that run in long mode. In order to get it working properly with our OS in Rust, we need to patch it and build it locally. The hard part has already been done for us ([details here](http://os.phil-opp.com/set-up-gdb.html)), so we can just quickly set it up with the following commands.
GDB has built-in support for QEMU, but it doesn't play nicely with OSes that run in long mode. In order to get it working properly with our OS in Rust, we need to patch it and build it locally. The hard part has already been done for us ([details here](https://os.phil-opp.com/set-up-gdb/)), so we can just quickly set it up with the following commands.

First, install the following packages:
`sudo apt-get install texinfo flex bison python-dev ncurses-dev`
Expand All @@ -86,13 +87,13 @@ Then, from the base directory of the Theseus project, run this command to easily

After that, you should have a `rust-os-gdb` directory that contains the `gdb` executables and scripts.

Then, simply run `make debug` to build and run Theseus in QEMU, which will pause the OS's execution until we attach our patched GDB instance.
Then, simply run `make run_pause` to build and run Theseus in QEMU, which will pause the OS's execution until we attach our patched GDB instance.
To attach the debugger to our paused QEMU instance, run `make gdb` in another terminal. QEMU will be paused until we move the debugger forward, with `n` to step through or `c` to continue running the debugger.
Try setting a breakpoint at the kernel's entry function using `b nano_core::nano_core_start` or at a specific file/line, e.g., `b scheduler.rs:40`.


## Documentation
Theseus, like the Rust language itself, includes two forms of documentation: a "book" for high-level overviews of design concepts, and the source-level documentation within the code itself. The former is a work in progress but may still be useful, while the latter is more useful for developing in Thesues.
Theseus, like the Rust language itself, includes two forms of documentation: the main source-level documentation within the code itself, and a "book" for high-level overviews of design concepts. The latter is a work in progress but may still be useful, while the former is more useful for developing on Thesues.

Once your build environment is fully set up, you can generate Theseus's documentation in standard Rust docs.rs format.
To do so, simply run:
Expand Down Expand Up @@ -125,106 +126,10 @@ Sometimes RLS just doesn't want to behave. In that case, try the following steps
You can see other installed toolchains with `rustup toolchain list`.


## License
The source code is licensed under the MIT License. See the LICENSE-MIT file for more.


# Other stuff

[How to set up PXE to boot Theseus over the network](book/src/pxe.md)


## Loading Theseus Through PXE
The following instructions are a combination of [this](https://www.ostechnix.com/how-to-install-pxe-server-on-ubuntu-16-04/) guide on OSTechNix to set up PXE for Ubuntu and [this](https://wellsie.net/p/286/) guide by Andrew Wells for using an arbitrary ISO with PXE.

PXE can be used to load Rust onto a target computer that is connected by LAN to the host machine used for development. To set up the host machine for PXE, first make the Theseus ISO by navigating to the directory Theseus is in and running:
`make iso`

Then, you will need to set up a TFTP and DHCP server which the test machine will access.
### Setting up the TFTP Server
First, install all necessary packages and dependencies for TFTP:
`sudo apt-get install apache2 tftpd-hpa inetutils-inetd nasm`
Edit the tftp-hpa configuration file:
`sudo nano /etc/default/tftpd-hpa`
Add the following lines:
```
RUN_DAEMON="yes"
OPTIONS="-l -s /var/lib/tftpboot"
```
Then, edit the inetd configuration file by opening the editor:
`sudo nano /etc/inetd.conf`
And adding:
`tftp dgram udp wait root /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /var/lib/tftpboot`

Restart the TFTP server and check to see if it's running:
`sudo systemctl restart tftpd-hpa`
`sudo systemctl status tftpd-hpa`

If the TFTP server is unable to start and mentions an in-use socket, reopen the tftp-hpa configuration file,set the line that has `TFTP_ADDRESS=":69"` to be equal to `6969` instead and restart the TFTP server.

### Setting up the DHCP Server
First, install package for DHCP server:
`sudo apt-get install isc-dhcp-server`

Then run `ifconfig` to view available networking devices and find the network device name, e.g., `eth0`.

Edit the `/etc/default/isc-dhcp-server` configuration file and add the network device name from the step above to "INTERFACES". For me, this looks like `INTERFACES="eth0"`.

Configure an arbitrary IP address that will be used in the next step:
`sudo ifconfig <network-device-name> 192.168.1.105`
This command might have to be done each time the computer being used as a server is restarted.

Edit the `/etc/dhcp/dhcpd.conf` file by uncommenting the line `authoritative;` and adding a subnet configuration such as the one below:
```
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.20 192.168.1.30;
option routers 192.168.1.1;
option broadcast-address 192.168.1.255;
default-lease-time 600;
max-lease-time 7200;
}
allow booting;
allow bootp;
option option-128 code 128 = string;
option option-129 code 129 = text;
next-server 192.168.1.105;
filename "pxelinux.0";
```

Restart the DHCP server and check to see if it's running:
`sudo systemctl restart isc-dhcp-server`
`sudo systemctl status isc-dhcp-server`

### Loading the Theseus ISO Into the TFTP Server
In order for the TFTP server to load Theseus, we need the Theseus ISO and a memdisk file in the boot folder. To get the memdisk file first download syslinux which contains it.
`wget https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-5.10.tar.gz`
`tar -xzvf syslinux-*.tar.gz`

Then navigate to the memdisk folder and compile.
`cd syslinux-*/memdisk`
`make memdisk`

Next, make a TFTP boot folder for Theseus and copy the memdisk binary into it along with the Theseus ISO:
`sudo mkdir /var/lib/tftpboot/theseus`
`sudo cp /root/syslinux-*/memdisk/memdisk /var/lib/tftpboot/theseus/`
`sudo cp /Theseus/build/theseus-x86_64.iso /var/lib/tftpboot/theseus/`

Navigate to the PXE configuration file:
`sudo nano /var/lib/tftpboot/pxelinux.cfg/default`
And add Theseus as a menu option by adding the following:
```
label theseus
menu label Theseus
root (hd0,0)
kernel theseus/memdisk
append iso initrd=theseus/theseus-x86_64.iso raw
```
Finally, restart the DHCP server one more time and make sure it's running:
`sudo systemctl restart isc-dhcp-server`
`sudo systemctl status isc-dhcp-server`

On the target computer, boot into the BIOS, turn on Legacy boot mode, and select network booting as the top boot option. Once the target computer is restarted, it should boot into a menu which displays booting into Theseus as an option.

### Subsequent PXE Uses
After setting up PXE the first time, you can run `make pxe` to make an updated ISO, remove the old one, and copy the new one over into the TFTP boot folder. At that point, you should be able to boot that new version of Theseus by restarting the target computer. If there are issues restarting the DHCP server after it worked the first time, one possible solution may be to confirm that the IP address is the one you intended it to be with the command from earlier:
`sudo ifconfig <network-device-name> 192.168.1.105`
# License
Theseus's source code is licensed under the MIT License. See the LICENSE-MIT file for more.
Loading

0 comments on commit cf45c46

Please sign in to comment.