http://www.CookingWithSteven.com

pingtest.py

A python program that uses threading to ping a large number of hosts.

Usage: python pingtest.py [input file] [number of threads]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

import subprocess
import threading
import sys
 
 
badHosts = []
hostQueue = []
lock = threading.Lock()
debug = False
count = 0
 
def parseHost(line):
    if line.find('node name') >= 0:
        return line[line.find('"') + 1 : len(line) - 4]
 
    return None
 
 
def ping(host):
    ret = subprocess.call("ping -c 1 %s" % host,
                          shell=True,
                          stdout=open('/dev/null', 'w'),
                          stderr=subprocess.STDOUT)
 
    return ret == 0
 
 
def popQueue():
    global lock
    global hostQueue
 
    host = None
 
    lock.acquire()
    if len(hostQueue) > 0:
        host = hostQueue.pop()
    lock.release()
 
    return host
 
 
 
def dequeue():
    global lock
    global badHosts
    global hostQueue
    global count
 
    while True:
        host = popQueue()
        if not host:
            return
 
        if not ping(host):
            lock.acquire()
            badHosts.append(host)
            lock.release()
 
        lock.acquire()
        count -= 1
        if debug:
            print "%d left" % count
        lock.release()
 
 
def main():
    global debug
    global badHosts
    global hostQueue
    global count
 
    try:
        file = sys.argv[1]
        n = int(sys.argv[2])
    except:
        print "Usage: python pingtest.py [input file] [number of threads] [optional debug]"
        return
 
    if len(sys.argv) > 3:
        debug = (sys.argv[3] == 'debug')
 
    f = open(file, 'r')
    lines = f.readlines()
    f.close()
 
    threads = []
    count = 0
 
    for line in lines:
        host = parseHost(line)
        if host:
            hostQueue.append(host)
            count += 1
 
    for i in range(n):
        t = threading.Thread(target = dequeue)
        t.start()
        threads.append(t)
 
    for t in threads:
        t.join()
 
    if debug:
        print "Writing to file..."
 
    f = open('static-list-of-pingable-nodes.txt', 'w')
    for line in lines:
        host = parseHost(line)
        if host:
            if not host in badHosts:
                f.write(line)
        else:
            f.write(line)
    f.close()
 
    if debug:
        print "Done. Nodes written to static-list-of-pingable-nodes.txt"
 
if __name__ == '__main__':
    main()

The following lines were added to the beginning of handle-plcdb.pl as a temporary hack, so that all nodes that are returned by the Perl script are pingable.

1
2
3
4

if ( -f "static-list-of-pingable-nodes.txt" ) {
    system("cat static-list-of-pingable-nodes.txt");
    exit(0);
}

As a temporary solution, any passphrase needs to be removed from the private key. Otherwise, everything works fine.

Specially dedicated to E.Y. who has struggled for months behind the Great Firewall.

First, you need SSH access to any server. Go to the terminal and enter:

ssh -vv -CND 1080 yourUserName@yourServer.com

You’ll be prompted to enter the password. Then, some messy texts are printed out. When it’s ready, you’ll see (varies across different flavors of Linux):

debug1: Entering interactive session.

Now you need to set up Firefox. Go to Edit > Preferences and open up “Firefox Preferences”. Then proceed to Advanced > Network, as shown. Click on “Settings”.

Firefox Preference

In “Connection Settings” window, check “Manual proxy configuration”. In “SOCKS Host”, enter “localhost” with port number 1080. Make sure that “SOCKS v5″ is checked.

Configuring the proxy in Firefox

Next, we need to override the default DNS settings. We can force Firefox to resolve domain names using the SOCKS v5 proxy, rather than using the DNS servers provided by the DHCP by default.

In Firefox’s address bar, enter “about:config”.

Firefox may warn you that you’re about to make changes that will probably upset the browser. Ignore it and proceed.

In the “Filter” text box, enter “dns”. You are may see this:

Configure remote DNS in Firefox

Look for the line that says “network.proxy.socks_remote_dns”. By default, it is set to false. Double click on that line so that the value becomes true, as shown:

Configure remote DNS in Firefox

Your Firefox is now correctly set up. All web traffic has to go through your SSH server, and all packets are encrypted so you don’t need to worry about being sniffed.

If you’re not convinced, open up http://www.ip-adress.com in Firefox. Instead of your usual IP, you’ll see your server’s IP address. What is more, your address has changed: your location is shown to be that of the server! Yay! Done!

Reference

“Access Facebook Through The Great Firewall”, <http://rejon.org/2009/07/access-facebook-through-the-great-firewall-second-line-ssh-tunnel/>

More coming up.

This example illustrates how to add Emulab resources (in particular, nodes) into Gush. We assume that you are already familiar with the basic configurations of Emulab and Gush experiments. We also assume that the working directory of Gush is ~/gush.

Configuring Emulab for Gush

Set up an Emulab experiment as you normally would. Ensure that the experiment is swapped in because Gush does not do this automatically. This example uses two Emulab nodes. The NS file is as follows:

source tb_compat.tcl
set ns [new Simulator]
 
set devbox0 [$ns node]
tb-set-hardware $devbox0 pc
tb-set-node-os $devbox0 "PLAB-DEVBOX-v42"
 
set devbox1 [$ns node]
tb-set-hardware $devbox1 pc
tb-set-node-os $devbox1 "PLAB-DEVBOX-v42"
 
$ns run

Next we need to set up the SSL Certificate which enables Gush to authenticate with Emulab. In the Emulab web interface, go to “My Emulab”. In the “Profile” section, click on “Generate SSL Cert”. Supply the necessary information and download the SSL Certificate. Save the file as emulab.pem in the directory ~/gush/emulab/.ssl. Create one if the directory does not exist.

This SSL Certificate consists of two sections: the RSA private key and the certificate. The private key is encrypted with a pass phrase. We need to remove the pass phrase so that it may work correctly in Gush. Refer to this step-by-step guide on the removal of the PEM pass phrase.

Configuring Gush for Emulab

Add the following section to ~/gush/directory.xml:

<resource_manager type="emulab">
    <user>yh1</user>
    <port>15888</port>
 
    <EmulabProjectID>Gush</EmulabProjectID>
    <EmulabExperimentID>gush</EmulabExperimentID>
</resource_manager>

Running the experiment

Start Gush. If the above is configured correctly, Gush should say that it has learned about the Emulab resources. To verify that the Emulab nodes have indeed been added to the resource pool, do the following:

gush> info nodes
There are 10 known nodes:
[ P         ] williams_gush@planetlab1.ucsd.edu:15001(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab5.williams.edu:15001(pref=0) (Disconnected.)
[   U       ] yh1@devbox0.gush.Gush.emulab.net:15888(pref=0) (Disconnected.)
[   U       ] yh1@devbox1.gush.Gush.emulab.net:15888(pref=0) (Disconnected.)

Try connecting to the Emulab nodes:

gush> connect pat emulab 2
Initiated connections to 2 of 2 hosts.
gush> yh1@devbox1.gush.gush.emulab.net:15888 has joined the mesh.
yh1@devbox0.gush.gush.emulab.net:15888 has joined the mesh.

And run a very simple experiment:

gush> shell 'hostname'
gush> yh1@devbox0.gush.gush.emulab.net:15888,15413: devbox0.gush.gush.emulab.net
 
yh1@devbox1.gush.gush.emulab.net:15888,15314: devbox1.gush.gush.emulab.net