CS133 Lab 4: SimpleDB Transactions

Exercise 1. The Buffer Pool will be responsible for granting and releasing page-level locks; in this exercise you will write the methods that acquire and release locks in the BufferPool. More details on using the LockManager class are below, but at a high level, you will need to do the following:

• Modify getPage() in BufferPool to block and acquire the desired lock for a transaction as described above. As needed, it should catch a DeadlockException and re-throw that exception as a TransactionAbortedException. If you are using the LockManager class, you'll accomplish this by adding a call to LockManager.acquireLock() at the very beginning of your BufferPool.getPage() like this (assuming your Lock Manager instance is called lockmgr):

  try {
        lockmgr.acquireLock(tid, pid, perm);
  } catch (DeadlockException e) { 
        throw new TransactionAbortedException(); // caught by caller, 
                                                 // who calls transactionComplete()
  }
  



• Implement releasePage(). This method is primarily used for testing, and at the end of transactions. If you are using the LockManager class, uncomment the call to releaseLock() in BufferPool and complete LockManager.releaseLock(), described below.


• Implement holdsLock() so that logic in Exercise 2 can determine whether a page is already locked by a transaction. If you are using the LockManager class, uncomment the call to holdsLock() in BufferPool and complete LockManager.holdsLock(), described below.

Here are some further implementation hints and details if you are using the provided LockManager class.

1. You won't need to add more code to acquireLock() until Exercise 5, but you may wish to look at it briefly now as it uses methods you will implement in this exercise.


2. LockManager constructor: Your constructor should create whatever data structure(s) you will be using to represent your lock table. The design of this is entirely up to you. You may decide to use multiple data structures, create a helper class, etc. As a design guideline, you should ensure your data structure(s) allow you to answer these questions:
   • Given a transactionId, which pages does it have locked?
   • Given a page Id, which transactions hold a lock on the page?
   • Given a page, which Permissions is it locked with?
   If you create a helper class for this and want to be able to check equality of instances of that class, be sure to implement its equals() method.


3. lock(): To get a lock, this method first checks if the given transaction can acquire a lock using locked(), which you will implement next. Right now you should just add code to lock() to update your lock table assuming it's okay (this is the "else" case in the code).


4. locked(): This method returns a boolean indicating whether a transaction is "locked out" from acquiring a lock on the given page with the given permissions. Logic for this method appears in the code in comments above the method. Be careful with == vs. equals and be sure to check Java documentation for whatever data structures you are using for your lock table to see when methods can return null.


5. holdsLock(): Simple method used by Buffer Pool to determine whether the given transaction has any type of lock on the given page.


6. releaseLock(): Release whatever lock the given transaction has on the given page, updating your lock table. Used for testing (used by BufferPool.releasePage()) and to help at the end of transactions, as you'll see in Exercise 4.

You may need to implement the next exercise before your code passes the unit tests in LockingTest.

Note: if it seems like LockingTest is hanging forever before it even runs any of its tests, the problem is likely happening in the setup() for LockingTest! Check out what happens there. In particular, does your implementation for locked() allow a transaction to get a lock it already has? And allow a transaction to get a lock on a page if no other transaction holds a lock on that page?

Debugging tip: When running tests with ant, note that the printing of standard output will be delayed until the tests complete. If this is making debugging hard, you could try temporarily commenting out parts of the unit tests.

Exercise 2. Ensure that you acquire and release locks throughout SimpleDB. Some (but not necessarily all) actions that you should verify work properly:

1. Reading tuples off of pages during a SeqScan (if you implemented locking in BufferPool.getPage(), this should work correctly as long as your HeapFile.iterator() uses BufferPool.getPage().)


2. Inserting and deleting tuples through BufferPool and HeapFile methods. Note that your implementation of HeapFile.insertTuple() and HeapFile.deleteTuple(), as well as the implementation of the iterator returned by HeapFile.iterator() should access pages using BufferPool.getPage(). Double check that that these different uses of BufferPool.getPage() pass the correct permissions object (e.g., Permissions.READ_WRITE or Permissions.READ_ONLY).


3. Marking dirty pages. You may also wish to double check that your implementation of BufferPool.insertTuple() and BufferPool.deleteTupe() call markDirty() on any of the pages they access (you should have done this when you implemented this code in Lab 2, but we did not test for this case.)


4. You will also want to think about acquiring and releasing locks in the following situations:
   • Adding a new page to a HeapFile. When do you physically write the page to disk? Are there race conditions with other transactions (on other threads) that might need special attention at the HeapFile level, regardless of page-level locking? You may need to synchronize the part of your code where you write a blank page to disk, e.g.,
     synchronized (this) { // code to write blank page }
   
   
   • Looking for an empty slot into which you can insert tuples. Most implementations scan pages looking for an empty slot, and will need a READ_ONLY lock to do this. Surprisingly, however, if a transaction t finds no free slot on a page p, t may immediately release the lock on p. Although this apparently contradicts the rules of two-phase locking, it is ok because t did not use any data from the page, such that a concurrent transaction t' which updated p cannot possibly effect the answer or outcome of t.

At this point, your code should pass the unit tests in LockingTest.
Code hanging? See the notes at the end of Exercise 1.

Exercise 3. Implement the necessary logic for page eviction without evicting dirty pages. You will need to modify the evictPage method in BufferPool. In particular, it must never evict a dirty page. If your eviction policy currently chooses a dirty page for eviction, you will have to find a way to evict an alternative page. In the case where all pages in the buffer pool are dirty, you should throw a DbException.

This functionality is not tested until you've completed Exercise 4.

Exercise 4. You will now implement transactionComplete in BufferPool.java to finish a transaction, adhering to a FORCE policy and Strict 2PL.

1. transactionComplete(tid,commit): This method should first deal with dirty pages in the buffer pool for committing or aborting to adhere to a FORCE policy. If the xact is committing (i.e., commit==true ), you should flush dirty pages associated with the xact to disk. Else, if the xact is aborting, you should throw away its changes to pages in the Buffer Pool (e.g., by using its "before image" or even just removing it from the Buffer Pool)..
   
   It should then release all locks associated with that transaction. If you are using the LockManager class, you can do this by uncommenting the call to releaseAllLocks and implementing that method in the Lock Manager as described next.
   
   Note that there is another version of transactionComplete that takes a single argument; you do not need to add any code there.


2. releaseAllLocks(tid): This method in the Lock Manager should update your lock table data structure(s) to release all locks held by the given xact. Be careful: Java doesn't like it if you are iterating over a collection while you are also removing elements from that collection; you may end up seeing a ConcurrentModificationException.

At this point, your code should pass the TransactionTest unit test and the AbortEvictionTest system test. You may find the TransactionTest system test illustrative, but it will likely fail until you complete the next exercise.

Exercise 5. Implement deadlock detection and resolution in BufferPool.java. If you are using the Lock Manager class, you will be checking for deadlocks (and possibly throwing a DeadLockException) in acquireLock().

You will want to check for a deadlock whenever a transaction attempts to acquire a lock and finds another transaction is holding the lock (note that this by itself is not a deadlock, but may be symptomatic of one). E.g., for the waits-for-graph approach, you could check if a cycle of transactions waiting has formed. Please describe your approach to dealing with deadlock in the lab writeup.

Aborting a transaction
To abort a transaction in the Lock Manager's acquireLock, you can simply throw a DeadlockException which should be caught by BufferPool.getPage() and re-thrown as a TransactionAbortedException as described in Exercise 1. This TransactionAbortedException will be caught by the code executing the transaction. (e.g., TransactionTest.java), which calls transactionComplete() to clean up after the transaction. You are not expected to automatically restart a transaction which fails due to a deadlock -- you can assume that higher level code in the unit tests will take care of this.

Unit testing
We have provided some (not-so-unit) tests in DeadlockTest. They are actually a bit involved, so they may take more than a few seconds to run (depending on your policy). If they seem to hang indefinitely, then you probably have an unresolved deadlock. These tests construct simple deadlock situations that your code should be able to escape. The tests will print TransactionAbortedExceptions corresponding to the deadlocks it successfully resolved to the console.

Note that there are two timing parameters near the top of DeadLockTest.java; these determine the frequency at which the test checks if locks have been acquired and the waiting time before an aborted transaction is restarted. You may observe different performance characteristics by tweaking these parameters if you use a timeout-based detection method.

In addition to DeadlockTest, your code should now should pass the TransactionTest system test (which may also run for quite a long time, but timing out at 10 minutes). Note that if you used a timeout approach to deadlock detection, you might not be able to pass all the sub-tests.

Debugging tip: TransactionTest runs actual queries--see the run method in XactionTester. So if DeadlockTest passes but TransactionTest does not, the issue may lie with the query plan operators that are used by this test, namely, Insert.java and Delete.java. If your implementation of those operators suppresses TransactionAbortedExceptions because it catches all exceptions, that could be the issue. Feel free to consult Lab 2 solution code.

[untar your tar.gz file]
[replace build.xml and test]
$ ant test
$ ant systemtest