Project Green Duckling

Will use ulimit to change the max fd, so that i don’t have to wait 10 minutes before things get screwed. Stay tuned.

When you try to do connect slice williams_gush on more than 900 nodes, most of the nodes are going to be marked as failed, though they can be ssh’ed into manually. Bizarre!

To address the problem that Gush may run of file descriptors, I have started the Crazy Stress Test in which all the planetlab nodes are added to williams_gush. It took forever to add all the nodes (more than 900 of them), so I had to write a little hack to handle-plcdb.pl.

Read more…

Gush now supports the latest stable version of SFA/SFI/GeniWrapper. It also works if the user is not a PI (e.g. me). A few caveats though. In my case, my private key requires a pass phrase, so when I’m running the GeniWrapper as a standalone Python script, it prompts for my PEM Pass Phrase incessantly, because it’s trying to make several XML-RPC calls that require my credentials. Furthermore, if GeniWrapper is run inside Gush, the PEM Passphrase prompt kills Gush, which freezes and accepts no input (not even ^C or ^D). I think it has to do with how Gush handles I/O of external processes.

As a temporary solution, any passphrase needs to be removed from the private key. Otherwise, everything works fine.

Development resumes now, after almost a month of inactivity. Expect to see more upgrades of Gush and related stuff. Right now, I’m going to integrate SFA/GeniWrapper into Gush. And finally its documentation is available at http://svn.planet-lab.org/wiki/SFAGuide.

This article explains how to create an SSH (Secure Shell) Tunnel over HTTP/HTTPS with SOCKS 5 proxy. This article assumes that you have access to an OpenSSH server, and that you’re running a *nix operating system, e.g. Mac OS, Linux, FreeBSD.

Specially dedicated to E.Y. who has struggled for months behind the Great Firewall.

First, you need SSH access to any server. Go to the terminal and enter:

ssh -vv -CND 1080 yourUserName@yourServer.com

You’ll be prompted to enter the password. Then, some messy texts are printed out. When it’s ready, you’ll see (varies across different flavors of Linux):

debug1: Entering interactive session.

Now you need to set up Firefox. Go to Edit > Preferences and open up “Firefox Preferences”. Then proceed to Advanced > Network, as shown. Click on “Settings”.

Firefox Preference

In “Connection Settings” window, check “Manual proxy configuration”. In “SOCKS Host”, enter “localhost” with port number 1080. Make sure that “SOCKS v5″ is checked.

Configuring the proxy in Firefox

Next, we need to override the default DNS settings. We can force Firefox to resolve domain names using the SOCKS v5 proxy, rather than using the DNS servers provided by the DHCP by default.

In Firefox’s address bar, enter “about:config”.

Firefox may warn you that you’re about to make changes that will probably upset the browser. Ignore it and proceed.

In the “Filter” text box, enter “dns”. You are may see this:

Configure remote DNS in Firefox

Look for the line that says “network.proxy.socks_remote_dns”. By default, it is set to false. Double click on that line so that the value becomes true, as shown:

Configure remote DNS in Firefox

Your Firefox is now correctly set up. All web traffic has to go through your SSH server, and all packets are encrypted so you don’t need to worry about being sniffed.

If you’re not convinced, open up http://www.ip-adress.com in Firefox. Instead of your usual IP, you’ll see your server’s IP address. What is more, your address has changed: your location is shown to be that of the server! Yay! Done!

Reference

“Access Facebook Through The Great Firewall”, <http://rejon.org/2009/07/access-facebook-through-the-great-firewall-second-line-ssh-tunnel/>

I’ve been reading this: http://dejant.blogspot.com/2007/08/chat-program-in-c.html

More coming up.

Using Emulab Resources in Gush

This example illustrates how to add Emulab resources (in particular, nodes) into Gush. We assume that you are already familiar with the basic configurations of Emulab and Gush experiments. We also assume that the working directory of Gush is ~/gush.

Configuring Emulab for Gush

Set up an Emulab experiment as you normally would. Ensure that the experiment is swapped in because Gush does not do this automatically. This example uses two Emulab nodes. The NS file is as follows:

source tb_compat.tcl
set ns [new Simulator]
 
set devbox0 [$ns node]
tb-set-hardware $devbox0 pc
tb-set-node-os $devbox0 "PLAB-DEVBOX-v42"
 
set devbox1 [$ns node]
tb-set-hardware $devbox1 pc
tb-set-node-os $devbox1 "PLAB-DEVBOX-v42"
 
$ns run

Next we need to set up the SSL Certificate which enables Gush to authenticate with Emulab. In the Emulab web interface, go to “My Emulab”. In the “Profile” section, click on “Generate SSL Cert”. Supply the necessary information and download the SSL Certificate. Save the file as emulab.pem in the directory ~/gush/emulab/.ssl. Create one if the directory does not exist.

This SSL Certificate consists of two sections: the RSA private key and the certificate. The private key is encrypted with a pass phrase. We need to remove the pass phrase so that it may work correctly in Gush. Refer to this step-by-step guide on the removal of the PEM pass phrase.

Configuring Gush for Emulab

Add the following section to ~/gush/directory.xml:

<resource_manager type="emulab">
    <user>yh1</user>
    <port>15888</port>
 
    <EmulabProjectID>Gush</EmulabProjectID>
    <EmulabExperimentID>gush</EmulabExperimentID>
</resource_manager>

Running the experiment

Start Gush. If the above is configured correctly, Gush should say that it has learned about the Emulab resources. To verify that the Emulab nodes have indeed been added to the resource pool, do the following:

gush> info nodes
There are 10 known nodes:
[ P         ] williams_gush@planetlab1.ucsd.edu:15001(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab5.williams.edu:15001(pref=0) (Disconnected.)
[   U       ] yh1@devbox0.gush.Gush.emulab.net:15888(pref=0) (Disconnected.)
[   U       ] yh1@devbox1.gush.Gush.emulab.net:15888(pref=0) (Disconnected.)

Try connecting to the Emulab nodes:

gush> connect pat emulab 2
Initiated connections to 2 of 2 hosts.
gush> yh1@devbox1.gush.gush.emulab.net:15888 has joined the mesh.
yh1@devbox0.gush.gush.emulab.net:15888 has joined the mesh.

And run a very simple experiment:

gush> shell 'hostname'
gush> yh1@devbox0.gush.gush.emulab.net:15888,15413: devbox0.gush.gush.emulab.net
 
yh1@devbox1.gush.gush.emulab.net:15888,15314: devbox1.gush.gush.emulab.net

The same emulab node appears twice within Gush, after I added a dummy EmulabResourceManager. For the record,

gush> info nodes
There are 10 known nodes:
[ P   C     ] williams_gush@planetlab1.ucsd.edu:15001(pref=10) (Finished.)
[ P   C     ] williams_gush@planetlab2.ucsd.edu:15001(pref=10) (Finished.)
[ P   C     ] williams_gush@planetlab3.ucsd.edu:15001(pref=10) (Finished.)
[ P   C     ] williams_gush@planetlab1.williams.edu:15001(pref=10) (Idle.)
[ P   C     ] williams_gush@planetlab2.williams.edu:15001(pref=10) (Idle.)
[ P   C     ] williams_gush@planetlab3.williams.edu:15001(pref=10) (Idle.)
[ P       F ] williams_gush@planetlab4.williams.edu:15001(pref=-25) (Failed.)
[ P   C     ] williams_gush@planetlab5.williams.edu:15001(pref=10) (Idle.)
[   U       ] yh1@devbox0.gush.Gush.emulab.net:15420(pref=0) (Connecting.)
[     C     ] yh1@devbox0.gush.gush.emulab.net:15420(pref=10) (Idle.)
gush> info mesh
Mesh:
williams_gush@planetlab1.ucsd.edu:15001: connected; (fds=9,9 last_heard=1248969258)
williams_gush@planetlab2.ucsd.edu:15001: connected; (fds=7,7 last_heard=1248969258)
williams_gush@planetlab3.ucsd.edu:15001: connected; (fds=6,6 last_heard=1248969258)
williams_gush@planetlab1.williams.edu:15001: connected; (fds=15,15 last_heard=1248969258)
williams_gush@planetlab2.williams.edu:15001: connected; (fds=12,12 last_heard=1248969258)
williams_gush@planetlab3.williams.edu:15001: connected; (fds=11,11 last_heard=1248969258)
williams_gush@planetlab4.williams.edu:15001: failed; (fds=18,17 last_heard=0)
williams_gush@planetlab5.williams.edu:15001: connected; (fds=14,14 last_heard=1248969258)
yh1@devbox0.gush.gush.emulab.net:15420: connected; (fds=16,16 last_heard=1248969258)
yh1@devbox0.gush.gush.emulab.net:15420: connected; (fds=16,16 last_heard=1248969258)
Summary: 9 mesh members; 0 nodes connecting.
gush>

Emulab generates an emulab.pem file which is by default encrypted with a PEM passphrase. It consists of two sections: the RSA private key (encrypted) and the certificate. We need to remove the PEM passphrase from the RSA private key for the file to work correctly in Gush.

Here is an example of what emulab.pem looks like:

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,552F11FEB2C33DB6
 
KGO5Rj9QcaRD6dfiuTHWnvAcwhER2yknQjUFTkB5/zz1JqMChAkH3q6nwpCehOVF
....
aUzRSY3KpBSCoKjpe0dcZkTHPoRqyCmcU75gioSYtbD2wWcvhGFsPA==
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDADCCAmkMAiRtMA0GCSqGSIb3DQEBBAUAMIG4MQswCQYDVQQGEwJVUzENMAsG
...
i6vWkw==
-----END CERTIFICATE-----

Before we proceed, please backup the original emulab.pem.

Open emulab.pem in a text editor and copy the certificate section into a new file called emulab.crt, which looks like this:

-----BEGIN CERTIFICATE-----
MIIDADCCAmkMAiRtMA0GCSqGSIb3DQEBBAUAMIG4MQswCQYDVQQGEwJVUzENMAsG
...
i6vWkw==
-----END CERTIFICATE-----

Next, we need to strip the RSA private key off its passphrase. In the terminal, run

openssl rsa -in emulab.pem -out rsa.key

Now that rsa.key contains no passphrase, we need to combine it with emulab.crt into a new Emulab SSL certificate.

cat rsa.key emulab.crt > emulab.pem

Finally, the passphrase-free emulab.pem can pose a security risk. This is the least we can do:

chmod 400 emulab.pem