1. Create an EC2 account if you do not have one.
2. Login to EC2
3. Select the Amazon EC2 tab in the Management Console page
4. 
   Create a new instance (by clicking on Launch Instance Button), select Linux image. Create a Key Pairs for authentication purpose. Save the pem file in your local directory and change its permission to 400. (You do not need to do this key pairs creation if you already have it)
5. After the instance is created, note its instance ID
6. Point your browser to here and search for “m-lab” to locate m-lab’s NPAD dataset. Note the snapshot ID (for Linux).
7. Go back to the EC 2 Management Console and select SnapShots (on the left panel under ELASTIC BLOCK STORES), under the viewing drop down list, select Public Snapshots and search for the M-Lab’s NAPD’s Linux snapshot ID.
8. Create on the Create Volume button to create a volume on your instance that you have previously created. (e.g. /dev/sdz)
9. Select the Instances option (on the left panel under INSTANCES), select the checkbox of the instance that you have previously created, click on Instance Actions and select Connect option. This will tell you how to connect to this instance using ssh.
10. On your local machine, in the directory where your pem file is stored, enter the ssh command accordingly
11. After the ssh is completed, mkdir data; mount /dev/sdz data
12. Now data directory contains M-Lab’s NPAD data. (You can use scp to copy it accordingly).

Addendum

As I was using scp, I wish to have multiple clients downloading the massive data. I was forced to distribute the pem file. Instead of doing this, (I got this idea from my boss) we installed apache server on the Amazon EC2 instance and mounted the data volume to the document root of web server. Now, we can use wget to download the data.

And I have started to use google chrome after its linux’s release is out. I always wanted to create a “hello world” chrome extension. So tonight (Friday night), I was able to find some quiet time to create a google-chrome extension to show recent earthquakes in CA and NV.

This extension contains 3 things. (place them in a directory.)

1. manifest.json
2. icon.png
3. popup.html

This is the manifest.json

{
  "name": "Recent Earthquakes in CA and NV",
  "version": "1.0",
  "description": "Recent Earthquakes (> 3.0) in California-Nevada.",
  "browser_action": {
    "default_icon": "icon.png",
     "popup": "popup.html"
  },
  "permissions": [
    "http://quake.usgs.gov/", "tabs"
  ]
}

here is my icon.png

and the popup.html

<style>
body {
    min-width:500px;
    background-color: #FFFFCC;
}
div {
    font-size: 13px;
    font-weight: bold;
}

span.mag {
    font-size: 10px;
    color: red;
}
span {
    font-size: 10px;
    color: black;
}

a {
    font-size: 10px;
}
</style>

<script>
var hostUrl = 'http://quake.usgs.gov';
var req = new XMLHttpRequest();
req.open(
    "GET",
    "http://quake.usgs.gov/recenteqs/Quakes/quakes0.htm",
    true);
req.onload = showTable;
req.send(null);

function showUrl(anchor) {
    var urlLocation = '' + anchor.href;
    chrome.tabs.create({url:urlLocation});
}

function showTable() {
    var title = document.createElement("div");
    title.innerText = 'Recent Earthquakes in CA and NV';
    document.body.appendChild(title);
    var hr = document.createElement("hr");
    document.body.appendChild(hr);

    var body = req.responseText;
    var i = body.indexOf('<A HREF="/recenteqs/Maps/');
    body = body.substring(i);
    i = body.lastIndexOf('</PRE>');
    body = body.substring(0, i);

    i = body.indexOf('<STRONG>');
    while (i != -1) {
        body = body.substring(i);
        i = body.indexOf('<A HREF="');
        j = body.indexOf('">');
        var map = hostUrl + body.substring(i+9, j);

        j = body.indexOf('<FONT COLOR="#CC0000"');
        i = body.indexOf('>', j);
        body = body.substring(i+2);
        i = body.indexOf('<A ');
        var mag = body.substring(0, i-1);

        i = body.indexOf('<A HREF="');
        j = body.indexOf('">');
        var details = hostUrl + body.substring(i+9, j);

        body = body.substring(j+2);
        i = body.indexOf('</A>');
        var detailsLoc = body.substring(0, i);

        body = body.substring(i+5);
        i = body.indexOf('</FONT>');
        var mapLoc = body.substring(0, i);

        i = body.indexOf('</STRONG>');
        body = body.substring(i+9);
        i = body.indexOf('<STRONG>');

        var span = document.createElement("span");
        span.setAttribute('class', 'mag');
        span.innerText = 'Mag ' + mag;

        var detailLink = document.createElement("a");
        detailLink.setAttribute('href', details);
        detailLink.setAttribute('onClick', 'showUrl(this)');
        detailLink.appendChild(span);
        document.body.appendChild(detailLink);

        var span = document.createElement("span");
        span.innerText = ' . . ';
        document.body.appendChild(span);

        var mapLink = document.createElement("a");
        mapLink.setAttribute('href', map);
        mapLink.setAttribute('onClick', 'showUrl(this)');
        mapLink.innerText = mapLoc;
        document.body.appendChild(mapLink);

        var br = document.createElement("br");
        document.body.appendChild(br);
    }
}
</script>

Next, go to the chrome extension page and switch to developer mode to load unpack extension. Basically, just point to the directory that I have created. And this is how it looks like.

Followings are the key things that we have done (I wrote about 80% of the SDK so it is ok to blame me if things are not working)

1. Improve scalability.
   We use indexes in LDAP server to locate policies for evaluation. This fast and heuristic approach helps us to eliminate most of unrelated policies and retrieve the related ones rapidly. Our latest test shows that we can handle over 1 million policies.
2. Improve performance.
   We use multi-threading, re-entrance read-write lock (Java concurrency package) and LDAP indexes to speed up policy evaluation. The performance has improved significantly as we benchmark it against the previous policy evaluation engine. We are in the process of tuning the caching system. And better results would be expected.
3. REST interfaces
   We have REST interfaces for policy evaluation and management. This means that non Java (such as PHP and Python) clients can make policy evaluation requests. State of art, Jersey is used for our REST implementation; and JSON is used too.
4. User friendly UI
   My co-worker has developed a nice set of entitlements services UI using icefaces
5. XACML support
   Able to import and export XACML.

Currently, we are working with technical writers on documenting entitlement services. You should be able to see these document early next year when OpenSSO Express 9 is shipped.

Well, year 2009 is coming to an end. It has been a fun year working on entitlements service (among my other doings). Sleepless nights, long meetings, ranting and hanging my head on the keyboard (just kidding) are part of the fun . I hope that 2010 will be more exciting.

Gee, I have sent over 9000 emails to OpenSSO’s email aliases for the past 4 years.
That’s 2381 emails per year. About 9 emails per day (excluding weekends). So, 1 email per hour! (8-9 hours work day).

Here is where you can find the source code for entitlements service.

• console resources: opensso/products/federation/openfm/web
• console source: opensso/products/federation/openfm/source
• entitlement APIs and backend implementation: opensso/products/amserver/com/sun/identity/entitlement
• CLI: opensso/products/amserver/com/sun/identity/cli/entitlement

Entitlements Service is released part of OpenSSO Express 8.0.

There are two types of datastore in OpenSSO server, namely the configuration datastore and user datastore. As the name suggest, the former stores the configuration data that are required by OpenSSO server to operate properly. The latter stores users related information, such as role, group and user entries.

It can be done through Command Line Interface or Administration Console.

The Command Line Interface way.

1. Output the current server configuration XML
   ./ssoadm get-svrcfg-xml -u amadmin -f /tmp/fampass -s \
   http://owen1.red.iplanet.com:8080/opensso -o /tmp/serverconfig.xml

2. Encrypt new password
   ./ampassword -e /tmp/newpassword

3. edit /tmp/serverconfig.xml. replace admin password with the new encrypted password.
4. Output the current server configuration XML
   ./ssoadm set-svrcfg-xml -u amadmin -f /tmp/fampass -s \
   http://owen1.red.iplanet.com:8080/opensso -X /tmp/serverconfig.xml

The Administration Console Interface way.

1. Login as amadmin
2. select Configuration tab
3. select Sites and Servers tab
4. Choose the server
5. select Directory Configuration tab
6. set the password

This new client runtime requirement shall be in our next official release i.e. OpenSSO Express 8 which is scheduled to released in a couple of months from now.

1. OpenSSO Community Day 3.0 Sunday (1:00 PM until 7:00 PM) May 31st 2009, Moscone Center, SF, CA.
   

   Open discussions on all OpenSSO related features.

2. CommunityOne West June 1-3 2009, Moscone Center, SF, CA.
   

   Hands On Lab. Web Application Security with OpenSSO.
   Monday June 1, 1:40 – 3:30 PM by Himanshu Vijay and Baby Sunil.

   Pragmatic Identity 2.0: Invoking Identity Services with a Simplified REST/ROA Architecture.
   Monday June 1, 11:50 AM – 12:40 PM by Daniel Raskin,

   Deep Dives. Identity Management with OpenSSO: Deploy an Identity Management Solution in 4 hours Learn how to build an identity management solution based on OpenSolaris, Open DS, and Sun OpenSSO Express 7.
   Wednesday, June 3, Morning Session. by Mrudul Uchil and David Goldsmith.

3. JavaOne Conference June 2-5, 2009, Moscone Center, SF, CA.
   

   BOF-5275 – Using and Participating in the OpenSSO Project
   Tuesday night, June 02, 9:30 PM – 10:20 PM hosted by Sean Brydon, Pat Patterson and Aravindan Ranganathan.

   TS-5295 Designing and Building Security into REST Applications
   Wednesday, June 03, 2:50 PM – 3:50 PM by Sean Brydon, Aravindan Ranganathan, Paul Bryan.

   TS-4012 – Pragmatic Identity 2.0: Simple, Open, Identity Services Using REST
   Thursday, June 04, 10:50 AM – 11:50 AM by – Pat Patterson and Ron Ten-Hove.

   LAB-6727 – Web Application Security with OpenSSO: From Simple Log-In to Single Sign-On to Federation
   Thursday, June 4, 1:30 – 3:00 pm by Pat Patterson, Himanshu Vijay and Baby Sunil.

   BOF-4903 – A RESTful approach to identity-based web services
   Thursday, June 04, 7:30 PM – 8:20 PM by Hubert Le Van Gong and Marc Hadley.

Remember to set the values of $EIGHT_DOT_ZERO and $CURRENT

#!/usr/bin/perl -w

use strict;

my $EIGHT_DOT_ZERO = '/home/dennis/workspace/opensso8.0';
my $CURRENT = '/home/dennis/workspace/opensso1';

my %eightdotXMLs;
my %currentXMLs;

getServiceXMLs(
    "$EIGHT_DOT_ZERO/opensso/products/amserver/xml/services",
     \%eightdotXMLs);
getServiceXMLs(
    "$EIGHT_DOT_ZERO/opensso/products/federation/openfm/xml/services",
    \%eightdotXMLs);
getServiceXMLs(
    "$CURRENT/opensso/products/amserver/xml/services",
     \%currentXMLs);
getServiceXMLs(
    "$CURRENT/opensso/products/federation/openfm/xml/services",
     \%currentXMLs);

foreach (keys %currentXMLs) {
    my $name = $_;
    my $rev = $currentXMLs{$_};

    if (! defined $eightdotXMLs{$name}) {
        print "$name ($rev) \n";
        }
    }
}

sub getServiceXMLs {
    my $base = shift;
    my $hash = shift;
    opendir(DIR, $base);
    foreach (readdir DIR) {
        my $f = $_;
        if (($f !~ /^\./) && ($f =~ /\.xml$/)) {
            getRev("$base/$f", $hash);
        }
    }
    closedir DIR;
}

sub getRev {
    my $file = shift;
    my $hash = shift;
    my $f = $file;
    $f =~ s/.+\///;

    my $buff = '';

    open(FILE, $file);
    while () {
        chomp;
        $buff .= $_;
    }
    close FILE;
    if ($buff =~ /<Schema .+?revisionNumber="(.+?)"/) {
        ${%{$hash}}{$f} = $1;
    } else {
        ${%{$hash}}{$f} = 0;
    }
}
 

I was at Borders on Saturday and was reading Clean Code by Uncle Bob. It mentioned that quality of code is measured by WTF/minute. Look at this Cartoon to understand what he meant.

Many of us have encountered this before. i.e. looking at a piece of code and go WTF, WTF, WTF …. It is very difficult to review code in cases like this. Comments were lacking or missing; methods were over 100 lines long (btw, the suggested length of a method is 24); and code was not well indented.

This book is very easy to read because many of the things written in it were common sense. For instance, “bad code functions too”; bad code causes organizational loss because of defects and difficulty in maintaining it (now why people are ignoring this?); have small classes and methods; etc.

Uncle Bob has done a good job in writing this book. Do get a copy of it if you are a software developer AND want to write clean code