See the stages below for an assignment outline.
Wiki Answers Due: 10-30-2012
Design Documents Due: 11-6-2012
Preliminary Patches Due: 11-11-2012
Final Patches Due: 11-21-2012
Code Reviews Due: 12-3-2012, 7 PM
In this assignment you will be implementing a set of file- and process-related system calls. When you have completed the first coding part of the assignment, your operating system will be able to run multiple copies of a single user-space program, and allow that program to perform basic file I/O. You will make OS/161 able to run real programs! Once you have completed the final coding parts of the assignment, you will also add the ability to run a variety of programs concurrently.
A substantial part of this assignment is understanding how OS/161 works and determining what code is required to implement the required functionality. Expect to spend at least as long browsing and digesting OS/161 code as actually writing and debugging your own code.
Recall that you must have your path correctly set to
undertake all CS 134 assignments. If you haven't done so already, put
the appropriate line in your .login or .bashrc.
Once we have chosen our patch for locks and condition variables, you will no longer need your code from Assignment 2. In the first week of the assignment, you will only be reading OS/161 code so it does not matter which code base you examine. Once the Assignment 3 code base is available, you can remove your Assignment 2 files. If you wish to save what you have done, you could run something along the lines of
cd ~/courses/cs134/your-group-name
mkdir hw2-changes
mv hw2/handin hw2-changes/
mv hw2/src/*/{thread,synch}.{c,h} hw2-changes/
before removing the hw2 directory.
For this assignment, once the hw3 directory is available,
svn copy it and run setup in the usual way,
then configure and build the kernel using the SYSCALLS
configuration file (see previous assignments for details).
Recall that the setup script builds all of the
“userland” programs and libraries (such as
/sbin/reboot). This assignment includes an additional
program, /testbin/hw3test. After you've run
setup and built the kernel, running this program (using
the p command from the menu) will yield the following
results:
Unknown syscall 6 Unknown syscall 6 Unknown syscall 6 ... Unknown syscall 0When your assignment is complete, the results will be much more satisfying.
In the previous assignment, you may have thought of your solutions to the cats and mice problem as separate programs, but they were really just functions that were linked into the kernel itself and thus ran inside the kernel, in kernel mode. We used that approach because the kernel wasn't yet able to run meaningful user-space code. In this assignment, you'll be making OS/161 deal with true user-space programs for the first time.
At present, OS/161's support for user-space programs is limited and
incomplete. The only working system call available to user-space code
is reboot. Moreover, the kernel itself doesn't currently
understand what a process is or maintain any per-process state. In
this assignment, you will remedy this issue by
read, write)
fork)
The first step is to read and understand the existing parts of
OS/161. At present, OS/161 can run one user-level C program at a
time—of course, the only programs that work are reboot,
halt, and poweroff, which all (only) use the
reboot system call. Understanding how the kernel runs
these simple programs should help you in planning what parts of the
code you will need to modify.
The code-reading component of this assignment is meant to direct your attention to those parts of OS/161 that are particularly relevant to this assignment.
kern/userprog/
kern/userprog/loadelf.c
dumbvm code does not provide what is normally
meant by virtual memory—although there is
translation between the addresses that executables
“believe” they are using and physical
addresses, there is no mechanism for providing more memory
than exists physically. The dumbvm system
also leaks memory, so expect OS/161 to run out of memory
after running a few programs.)
kern/userprog/runprogram.c
runprogram,
which is responsible for running a program from the kernel
menu. It is a good base for writing the execv
system call, but only a base—you will need to
determine what more is required for execv
that runprogram does not need to worry
about. Additionally, once you have designed your process
system, runprogram should be altered to start
processes properly within this framework; for example, a
program started by runprogram should have the
standard file descriptors available while it's running.
kern/userprog/uio.c
lib/copyinout.c.
kern/arch/mips/mips/
syscall.c handles traps that happen to be
syscalls. Understanding the C code in this directory is
key to being a real operating-systems junkie.
kern/arch/mips/mips/trap.c
mips_trap is the key function for returning
control to the operating system. This is the C function
that gets called by the assembly exception handler.
md_usermode is the key function for returning
control to user programs. kill_curthread is
the function for handling broken user programs; when the
processor is in user mode and hits something it can't
handle (say, a bad instruction), it raises an exception.
There's no way to recover from this, so the OS needs to
kill off the process. Part of this assignment will be to
write a useful version of this function.
kern/arch/mips/mips/syscall.c
mips_syscall is the function that delegates
the actual work of a system call to the kernel function
that implements it. Notice that reboot is
the only case currently handled. You will also find a
function, md_forkentry, which is a stub where
you will place your code to implement the
fork system call. It should get called from
mips_syscall.
lib/crt0/
mips-crt0.S,
which contains the MIPS assembly code that receives
control when a user-level program is started. It calls
main. Your execv implementation
will interface with this code, so be sure to check what
values it expects to appear in what registers and so
forth.
lib/libc/
lib/libc/errno.c
errno.
lib/libc/syscalls-mips.S
lib/libc/syscalls.S
syscalls-mips.S at
compile time and is the actual file assembled to put into
the C library. The names of the system calls are placed
in this file using a script, callno-parse.sh,
that reads them from the kernel's header files. This
approach avoids having to make a second list of the system
calls.
As with the previous assignment, the Wiki component is not graded in the conventional sense—completion of this part of the assignment is covered under the course's participation requirement. Every member of the class should should post or improve the answer to about two of the questions below on the course's Wiki and understand the answers to all of the questions. Some questions are easier than others, so if you have answered an easy one, you should try to correct or amplify a more difficult one.
In class, I may ask you about the answers to these questions, or ask you closely related questions whose answers you should know from answering the questions below.
UIO_USERISPACE
and UIO_USERSPACE?UIO_SYSSPACE instead?
struct uio that is used to read in a
segment be allocated on the stack in
load_segment? (I.e., where does the memory read
actually go?)
runprogram, why is it important to call
vfs_close before going to user mode?
md_usermode and
mips_usermode?
copyin and copyout
defined? What about memmove?copyin and copyout be
implemented as simply as memmove?
userptr_t
mips_trap set curspl to
SPL_HIGH “manually,” instead of using
splhigh?
mips_syscall carefully, not by
looking somewhere else.)
struct trapframe, and what is
the trapframe used for?trapframe that is passed into
mips_syscall stored?
struct thread defined? What does
this structure contain?
errno. How does errno get set?
md_forkentry function exists to
“return” from the fork() system call
in a newly created child process. Why is return in quotes?
When you write this function, what code will you use as your
starting point? Where did the trapframe come
from? How will md_forkentry come to be called?
SYSCALL macro?
vfs_open and vfs_close
used? What other vfs_* calls are relevant?
path argument to vfs_open is
not const. Investigate, and determine whether
there are any potential gotchas with using
vfs_open.
VOP_READ and VOP_WRITE?
How are they used?VOP_TRYSEEK do?
The first week of this assignment is devoted to design, with the remainder for testing, documenting, and cleaning up your code—although you can write some ancillary code even during the first week. Your first goal is to work out how you are going to achieve the objectives of this assignment, so that both members of the team will have a clear idea of what it is you are trying to achieve when you begin coding. At the end of the design phase, you will bring your design documents to class, ready to explain your design to others (ideally you should have electronic versions too). After class you may use your own design as is, or borrow from someone else's to use in synthesizing your final design.
You are required to provide an implementation plan and time
budget in the file handin/schedule.txt. This requirement
is intended to help you—if you are not disciplined in
managing your time, you will have severe difficulties completing this
assignment.
In this part, you will need to design and implement support for the following system calls:
open, read, write,
lseek, close, dup2
fork, _exit
kern/include/kern/callno.h
include/unistd.h
The manual pages in the OS/161 distribution (available in the
src/man/bin directory of your source tree, and on-line
on the CS134 Web site)
contain a description of correct return values for various errors. If
there are conditions that can happen that are not listed in the manual
page, return the most appropriate error code from
kern/errno.h. If none seem particularly appropriate,
consider adding a new one.
If you're adding an error code for a condition for which Unix has a
standard error-code symbol, use the same symbol if possible. Consult
the Unix manual pages to learn about Unix error codes (see the
intro page in Section 2 of the manual; use man
man to find out how to get to Section 2). Note that if you add
an error code to kern/include/kern/errno.h, you need to
add a corresponding error message to the file
src/lib/libc/strerror.c.
The OS/161 kernel already knows how to open files and character devices, and read and write from them. The problem is that this functionality is not exposed to user programs—no system calls are currently implemented that provide these facilities. Your task is to implement the necessary POSIX-style interface.
The open, read, write,
lseek, close, and dup2 calls
all manipulate file descriptors. From a user-code perspective, file
descriptors are small integers returned by the operating system, which
signify open files. On a POSIX system, the convention is that for any
given process, the first three file descriptors (0, 1, and 2) are
considered to be standard input (stdin), standard output
(stdout), and standard error (stderr). In
your solution, programs run directly from the OS/161 menu system
should start out with these file descriptors attached to the console
device (“con:”), but your implementation must
allow programs to use dup2 to change them to point
elsewhere.
Although these system calls may seem to be tied to the filesystem, in fact, they are really about manipulation of file descriptors, or process-specific filesystem state. A large part of this assignment is designing and implementing a system to track this state. You must determine what information is specific only to the process, what is specific to the file descriptor, and how the two relate. Don't rush your design. Think carefully about the state you need to maintain, how to organize it, and when and how it has to change. Important questions include:
forks, and what about afterwards?
dup2 pose any complications for your design?
dup2 does?
Remember that you will not actually have to write any low-level file system code to implement these system calls. You will use the existing VFS layer to do most of the work. Your job is to construct the subsystem that implements the interface expected by user-level programs by invoking the appropriate VFS and vnode operations.
For consistency, please place most of your implementation of file-related system calls in the following files:
kern/include/file.h
kern/userprog/file.c
fork system call. Your implementation should be
the same as that described in the fork man page, except
that it should return 1 to the parent (rather than the child's PID).
The amount of code to implement fork and
_exit is quite small at this stage; the main challenge is
to understand what needs to be done and where. In particular, you
should design and implement the file-related system calls with
fork in mind.
Some hints:
mips_usermode in
kern/arch/mips/mips/trap.c
kern/include/addrspace.h,
particularly as_copy
thread_fork function in
kern/thread/thread.c
Currently, if user-space code generates a fatal exception, the kernel
panics. This placeholder solution is obviously not acceptable, and
needs to be fixed now that you'll be running actual programs in user
space. You'll need to replace kill_curthread with a new
version that properly handles such issues more sanely. You should, of
course, try to write it in as simple a manner as possible. Keep in
mind that essentially nothing about the current thread's user-space
state can be trusted if it has suffered a fatal exception—it
must be taken off the processor in as judicious a manner as possible,
but without returning execution to the user level.
/testbin/hw3test to test your system calls.
Example output (where most of the system calls except
_exit are implemented) is shown below:
OS/161 kernel [? for menu]: p /testbin/hw3test Operation took 0.000212160 seconds OS/161 kernel [? for menu]: ********** * File Tester ********** * write() works for stdout ********** * write() works for stderr ********** * opening new file "test.file" * open() got fd 3 * writing test string * wrote 45 bytes * writing test string again * wrote 45 bytes * closing file ********** * opening old file "test.file" * open() got fd 3 * reading entire file into buffer * attempting read of 500 bytes * read 90 bytes * attempting read of 410 bytes * read 0 bytes * reading complete * file content okay ********** * testing lseek * reading 10 bytes of file into buffer * attempting read of 10 bytes * read 10 bytes * reading complete * file lseek okay * closing file ********** * testing fork * Forked, in parent * Forked, in child Unknown syscall 0
In the final part of the assignment, you will implement the following system calls:
getpid
execv, waitpid, and _exit
A pid, or process ID, is a unique number that identifies a process (as
such, it has much in common with a file descriptor; both are integers
that let user-space programs refer to kernel data safely). The
implementation of getpid is not terribly challenging, but
pid allocation and reclamation are the important concepts that you
must implement. It is not okay for your system to crash because over
the lifetime of its execution you've used every possible pid
once. Design your pid system, implement all the tasks associated with
pid maintenance, and only then implement getpid.
In this part, you must revise fork so that a newly
created process is given a new PID and that pid is returned to the
parent of the process. You will want to think carefully
through the design of fork and consider it together with
execv to make sure that each system call is performing
the correct functionality.
Although execv is “only” a system call, it is
perhaps the most challenging part of this assignment. It is
responsible for taking a process and making it execute a new program.
Essentially, it must replace the existing address space with a brand
new one for the new executable (created by calling
as_create in the current dumbvm system) and
then run it. While this is similar to starting a process straight out
of the kernel (as runprogram does), it's not quite that
simple. Remember that this call is coming out of user space, into the
kernel, and then returning back to user space. You must manage the
memory that travels across these boundaries very carefully. (Also,
notice that runprogram doesn't take an argument
vector—but arguments must of course be handled correctly in
execv).
Although it may seem simple at first, waitpid requires a
fair bit of design. Read the specification carefully to understand the
semantics, and consider these semantics from the ground up in your
design. You may also wish to consult the Unix manual page, although you
should keep in mind that you are not required to implement all the
things that Unix's waitpid supports—nor is the Unix
parent/child model of waiting the only valid or viable possibility.
Your revised implementation of _exit is also intimately
connected to the implementation of waitpid. They are
essentially two halves of the same mechanism. In all likelihood, your
code for _exit will be simple and your code for
waitpid more complex—but it's perfectly viable to
design it the other way around. If you find both are becoming
extremely complicated, it may be a sign that you should rethink your
design.
The system calls you have in OS/161 do not allow as full-featured a
shell as Unix, but they are nevertheless sufficient to write
a simple and useful command-line interface. This optional
part of the assignment is recommended for your own sense of fun and
achievement, but is not assessed. You may adapt code you wrote for
105, or collaborate more widely than with just your partner. Skeleton
code for a shell is in bin/sh/sh.c.
Remember that because OS/161 implements a subset of the standard POSIX
API, you can develop your shell under BSD, Linux, MacOS X, or Solaris
and then compile it for OS/161. But keep in mind that OS/161 is
really basic right now. No signals. No malloc.
And no, you're not allowed to write those missing parts to make your
shell better/easier. Yet.
To assist you in organizing and budgeting your time, this assignment is completed in stages.
On Tuesday, November 6, you will need to bring your (printed) design
documents to class. These documents will show how you plan to
implement file I/O and processes. You do not need to have thought
through your methods for handling process IDs or implementing
waitpid, but it would be good if you have.
Your documents must include:
open, close, fork, read, fork, and _exit)
As usual, you will make your changes to OS/161 available in the form
of a patch. This patch is essentially a work in progress and need not
be extensively commented or have explanatory text. It should,
however, provide a complete working implementation of
open, read, write,
lseek, close, dup2,
fork, and _exit.
You should generate your preliminary patch by running
cd ~/cs134/hw3 handin/makepatch > handin/prelim-syscalls.patch svn add handin/prelim-syscalls.patch svn commit
Your preliminary patch is due on the night of Sunday, November 11.
Your final patch will include code implementing all coding parts of the assignment.
You should generate your final patch by running
cd ~/cs134/hw3 handin/makepatch > handin/syscalls.patch svn add handin/syscalls.patch svn commit
You should also document your patch in handin/syscalls.txt.
The final patch is due the night of Wednesday, November 21.
char *filename = "/bin/cp";
char *args[4];
pid_t pid;
args[0] = "cp";
args[1] = "file1";
args[2] = "file2";
args[3] = NULL;
pid = fork();
if (pid == 0)
execv(filename, argv);
which will load the executable file /bin/cp, install it
as a new process, and execute it. The new process will then find
file1 on the disk and copy it to file2.
Some questions to think about:
cat command, and the cat command
starts to read from file descriptor 0, what will happen if the
shell also tries to read from the same file descriptor? What
would you like to happen?
Finally, remember to keep it simple. Both open
and fork are allowed to return errors saying that you've
opened too many files at once or forked too many processes at once,
which, in turn, means that you are allowed to use fixed-sized tables.
After all assignments have been submitted, we will perform code reviews on the submissions.