Wednesday, June 23, 2010

SSH login without password


First log in on A as user a and generate a pair of authentication keys.

Do not enter a passphrase:











a@A:~> ssh-keygen -t rsa Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa):
Created directory '/home/a/.ssh'. Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/a/.ssh/id_rsa. 
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is: 3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A 

Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh b@B's password:  

Finally append a's new public key to b@B:.ssh/authorized_keys and enter b's password one last time:

a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys' b@B's password:  

From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B hostname B 

A note from one of our readers: Depending on your version of SSH you might also have to do the following changes:

  • Put the public key in .ssh/authorized_keys2
  • Change the permissions of .ssh to 700
  • Change the permissions of .ssh/authorized_keys2 to 640

Tuesday, June 22, 2010

Voice Over IP - Per Call Bandwidth Consumption

Introduction

One of the most important factors to consider when you build packet voice networks is proper capacity planning. Within capacity planning, bandwidth calculation is an important factor to consider when you design and troubleshoot packet voice networks for good voice quality.

This document explains voice codec bandwidth calculations and features to modify or conserve bandwidth when Voice over IP (VoIP) is used.

VoIP – Per Call Bandwidth

These protocol header assumptions are used for the calculations:

  • 40 bytes for IP (20 bytes) / User Datagram Protocol (UDP) (8 bytes) / Real-Time Transport Protocol (RTP) (12 bytes) headers.

  • Compressed Real-Time Protocol (cRTP) reduces the IP/UDP/RTP headers to 2or 4bytes (cRTP is not available over Ethernet).

  • 6 bytes for Multilink Point-to-Point Protocol (MP) or Frame Relay Forum (FRF).12 Layer 2 (L2) header.

  • 1 byte for the end-of-frame flag on MP and Frame Relay frames.

  • 18 bytes for Ethernet L2 headers, including 4 bytes of Frame Check Sequence (FCS) or Cyclic Redundancy Check (CRC).


Codec Information

Bandwidth Calculations

Codec & Bit Rate (Kbps)

Codec Sample Size (Bytes)

Codec Sample Interval (ms)

Mean Opinion Score (MOS)

Voice Payload Size (Bytes)

Voice Payload Size (ms)

Packets Per Second (PPS)

Bandwidth MP or FRF.12 (Kbps)

Bandwidth w/cRTP MP or FRF.12 (Kbps)

Bandwidth Ethernet (Kbps)

G.711 (64 Kbps)

80 Bytes

10 ms

4.1

160 Bytes

20 ms

50

82.8 Kbps

67.6 Kbps

87.2 Kbps

G.729 (8 Kbps)

10 Bytes

10 ms

3.92

20 Bytes

20 ms

50

26.8 Kbps

11.6 Kbps

31.2 Kbps

G.723.1 (6.3 Kbps)

24 Bytes

30 ms

3.9

24 Bytes

30 ms

33.3

18.9 Kbps

8.8 Kbps

21.9 Kbps

G.723.1 (5.3 Kbps)

20 Bytes

30 ms

3.8

20 Bytes

30 ms

33.3

17.9 Kbps

7.7 Kbps

20.8 Kbps

G.726 (32 Kbps)

20 Bytes

5 ms

3.85

80 Bytes

20 ms

50

50.8 Kbps

35.6 Kbps

55.2 Kbps

G.726 (24 Kbps)

15 Bytes

5 ms

60 Bytes

20 ms

50

42.8 Kbps

27.6 Kbps

47.2 Kbps

G.728 (16 Kbps)

10 Bytes

5 ms

3.61

60 Bytes

30 ms

33.3

28.5 Kbps

18.4 Kbps

31.5 Kbps

G722_64k(64 Kbps)

80 Bytes

10 ms

4.13

160 Bytes

20 ms

50

82.8 Kbps

67.6Kbps

87.2 Kbps

ilbc_mode_20(15.2Kbps)

38 Bytes

20 ms

NA

38 Bytes

20 ms

50

34.0Kbps

18.8 Kbps

38.4Kbps

ilbc_mode_30(13.33Kbps)

50 Bytes

30 ms

NA

50 Bytes

30 ms

33.3

25.867 Kbps

15.73Kbps

28.8 Kbps

Explanation of Terms

Codec Bit Rate (Kbps)

Based on the codec, this is the number of bits per second that need to be transmitted to deliver a voice call. (codec bit rate = codec sample size / codec sample interval).

Codec Sample Size (Bytes)

Based on the codec, this is the number of bytes captured by the Digital Signal Processor (DSP) at each codec sample interval. For example, the G.729 coder operates on sample intervals of 10 ms, corresponding to 10 bytes (80 bits) per sample at a bit rate of 8 Kbps. (codec bit rate = codec sample size / codec sample interval).

Codec Sample Interval (ms)

This is the sample interval at which the codec operates. For example, the G.729 coder operates on sample intervals of 10 ms, corresponding to 10 bytes (80 bits) per sample at a bit rate of 8 Kbps. (codec bit rate = codec sample size / codec sample interval).

MOS

MOS is a system of grading the voice quality of telephone connections. With MOS, a wide range of listeners judge the quality of a voice sample on a scale of one (bad) to five (excellent). The scores are averaged to provide the MOS for the codec.

Voice Payload Size (Bytes)

The voice payload size represents the number of bytes (or bits) that are filled into a packet. The voice payload size must be a multiple of the codec sample size. For example, G.729 packets can use 10, 20, 30, 40, 50, or 60 bytes of voice payload size.

Voice Payload Size (ms)

The voice payload size can also be represented in terms of the codec samples. For example, a G.729 voice payload size of 20 ms (two 10 ms codec samples) represents a voice payload of 20 bytes [ (20 bytes * 8) / (20 ms) = 8 Kbps ]

PPS

PPS represents the number of packets that need to be transmitted every second in order to deliver the codec bit rate. For example, for a G.729 call with voice payload size per packet of 20 bytes (160 bits), 50 packets need to be transmitted every second [50 pps = (8 Kbps) / (160 bits per packet) ]

Bandwidth Calculation Formulas

These calculations are used:

  • Total packet size = (L2 header: MP or FRF.12 or Ethernet) + (IP/UDP/RTP header) + (voice payload size)

  • PPS = (codec bit rate) / (voice payload size)

  • Bandwidth = total packet size * PPS

Sample Calculation

For example, the required bandwidth for a G.729 call (8 Kbps codec bit rate) with cRTP, MP and the default 20 bytes of voice payload is:

  • Total packet size (bytes) = (MP header of 6 bytes) + ( compressed IP/UDP/RTP header of 2 bytes) + (voice payload of 20 bytes) = 28 bytes

  • Total packet size (bits) = (28 bytes) * 8 bits per byte = 224 bits

  • PPS = (8 Kbps codec bit rate) / (160 bits) = 50 pps

    Note: 160 bits = 20 bytes (default voice payload) * 8 bits per byte

  • Bandwidth per call = voice packet size (224 bits) * 50 pps = 11.2 Kbps


Codec

Voice Payload Size (ms)

Voice Payload Size (Bytes)

Comments

G.711

20 ms (default)

160 Bytes

Notice that the codec bit rate is always maintained. For example: G.711 codec = [240 bytes * 8(bits/bytes)] / 30 ms = 64 Kbps

30 ms

240 Bytes

G.729

20 ms (default)

20 Bytes

30 ms

30 Bytes

G.723

30 ms (default)


Each codec sample produces 10 bytes of voice payload.
  Valid sizes are: 10, 20, 30, 40, 50, 60, 70, 80, 90,
   100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230 
  Any other value within the range will be rounded down to nearest valid size.
   <10-230> Choose a voice payload size from the list above

Impact of Changing Voice Payload Sizes

The number of codec samples per packet is another factor that determines the bandwidth and delay of a VoIP call. The codec defines the size of the sample, but the total number of samples placed in a packet affects how many packets are sent per second.

When you increase the voice payload size the VoIP bandwidth reduces and the overall delay increases. The following example illustrates this:

  • G.729 call with voice payload size of 20 bytes (20 ms): (40 bytes of IP/UDP/RTP headers + 20 bytes voice payload)* 8 bits per byte * 50 pps = 24 Kbps

  • G.729 call with voice payload size of 40 bytes (40 ms): (40 bytes of IP/UDP/RTP headers + 40 bytes voice payload) * 8 bits per byte * 25 pps = 16 Kbps

Note: L2 headers are not considered in this calculation.

Note: The calculations show that while the payload size is doubled, the number of packets per second required is subsequently cut in half.

Note: As defined in the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) G.114 specifications, the recommended one-way overall delay for voice is 150 ms. For a private network, 200 ms is a reasonable goal, and 250 ms must be the maximum.

Voice Activity Detection

With circuit-switched voice networks, all voice calls use 64 Kbps fixed-bandwidth links regardless of how much of the conversation is speech and how much is silence. With VoIP networks, all conversation and silence is packetized. With Voice Activity Detection (VAD), packets of silence can be suppressed.

Over time and as an average on a volume of more than 24 calls, VAD can provide up to a 35 percent bandwidth savings. The savings are not realized on every individual voice call, or on any specific point measurement. For the purposes of network design and bandwidth engineering, VAD must not be taken into account, especially on links that carry fewer than 24 voice calls simultaneously. Various features such as music on hold and fax render VAD ineffective. When the network is engineered for the full voice call bandwidth, all savings provided by VAD are available to data applications.

VAD also provides Comfort Noise Generation (CNG). Because you can mistake silence for a disconnected call, CNG provides locally generated white noise so the call appears normally connected to both parties. G.729 Annex-B and G.723.1 Annex-A include an integrated VAD function, but otherwise performs the same as G.729 and G.723.1, respectively.

RTP Header-Compression or Compressed RTP (cRTP)

bwidth_consume.gif

All VoIP packets are made up of two components: voice samples and IP/UDP/RTP headers. Although the voice samples are compressed by the Digital Signal Processor (DSP) and can vary in size based on the codec used, these headers are a constant 40 bytes in length. When compared to the 20 bytes of voice samples in a default G.729 call, these headers make up a considerable amount of overhead. With cRTP, these headers can be compressed to two or four bytes. This compression offers significant VoIP bandwidth savings. For example, a default G.729 VoIP call consumes 24 Kb without cRTP, but only 12 Kb with cRTP enabled.

Because cRTP compresses VoIP calls on a link-by-link basis, both ends of the IP link need to be configured for cRTP.



Tuesday, April 27, 2010

Linux readdir example code to check /proc/ process dir.

NAME

readdir, readdir_r - read directory

SYNOPSIS



#include <sys/types.h>
#include <dirent.h>

struct dirent *readdir(DIR *dirp);
int readdir_r(DIR *dirp, struct dirent *entry, struct dirent **result);

Code Example.


/* just compile and run below c code, it will list all the process pid dir from /proc system. */

#include
#include
#include
#include
#include
#include

int main ()
{
DIR *fd_proc;
struct dirent *dirp;
int fd_stat, cur_pid, cnt;
char buf[1000], pdir[50], statfile[50], procname[50];
int pid;

sprintf (pdir, "/proc/");

if (!(fd_proc = opendir (pdir)))
{
printf( "Unable to open dir = %s, errno \n", pdir);
return -1;
}
while ((dirp = readdir (fd_proc)) != NULL)
{
printf(" %s, \n",dirp->d_name);

pid = atoi (dirp->d_name);
if (pid <= 0)
continue;
sprintf (statfile, "/proc/%s/stat", dirp->d_name);
if ((fd_stat = open (statfile, O_RDONLY)) < 0)
{
printf ( "Unable to open stat = %s, errno %d\n", dirp->d_name, errno);
closedir (fd_proc);
continue ;
}
}
printf("Hello World");
return 1;
}


DESCRIPTION

Do man readdir.

Monday, November 23, 2009

Internet checksum

Short description of the Internet checksum

IP checksum definition

The IP checksum is the 16 bit one's complement of the one's complement sum of all 16 bit words in the header.

One question many people may ask is "What is the 1's complement sum ?". This is because all computers utilize the 2's complement representation and the 1's complement is not used. The following gives a short introduction.

2's complement fixed point integers (8-bit)

Binary Decimal Hex
0000 0000 0 00
0000 0001 1 01
0000 0010 2 02
0000 0011 3 03
1111 1111 -1 FF
1111 1110 -2 FE
1111 1101 -3 FD

Let's add two intergers:
-3 + 5 = 2
FD + 05 = 01 02
Discarding the carry (01) gives the correct result.

1's complement fixed point integers (8-bit)

Binary Decimal Hex
0000 0000 0 00
0000 0001 1 01
0000 0010 2 02
0000 0011 3 03
1111 1111 -0 FF
1111 1110 -1 FE
1111 1101 -2 FD
1111 1100 -3 FC

Add the same numbers:
-3 + 5 = 2
FC + 05 = 01 01
Adding the carry (01) to the LSB (01) gives the correct result:
01 + 01 = 02

So, the 1's complement sum is done by summing the numbers and adding the carry (or carries) to the result..

Simple Internet checksum example

Suppose we have an 8-bit, 2's complement, machine and send the packet

FE 05 00

where 00 is the checksum field.

Let's calculate and verify the Internet checksum.

FE + 05 = 01 03

This is the result of the normal (2's complement) addition. The 1's complement sum requires the addition of the carry to the 8-bit word (even though we will not get the same result)

03 + 01 = 04

so the 1's complement sum of FE + 05 is 04.

The 1's complement of the 1's complement sum (Internet checksum) will be

~04 = FB

and the packet will be sent as

FE 05 FB

Now, at the receiving end we add all the received bytes, including the checksum (again using the 2's complement representation)

FE + 05 + FB = 01 FE

The 1's complement sum is

FE + 01 = FF = -0

which checks that the transmission was OK (see below).

A more complex example (32-bit machine)

As shown in RFC 1071, the checksum calculation is done in the following way:

(1) Adjacent octets to be checksummed are paired to form 16-bit integers, and the 1's complement sum of these 16-bit integers is formed.

(2) To generate a checksum, the checksum field itself is cleared, the 16-bit 1's complement sum is computed over the octets concerned, and the 1's complement of this sum is placed in the checksum field.

(3) To check a checksum, the 1's complement sum is computed over the same set of octets, including the checksum field. If the result is all 1 bits (-0 in 1's complement arithmetic), the check succeeds.

Packet

01 00 F2 03 F4 F5 F6 F7 00 00

(00 00 is the checksum field)

Form the 16-bit words

0100 F203 F4F5 F6F7

Calculate 2's complement sum

0100 + F203 + F4F5 + F6F7 = 0002 DEEF (store the sum in a 32-bit word)

Add the carries (0002) to get the 16-bit 1's complement sum

DEEF + 002 = DEF1

Calculate 1's complement of the 1's complement sum

~DEF1 = 210E

We send the packet including the checksum 21 0E

01 00 F2 03 F4 F5 F6 F7 21 0E

At the receiving

0100 + F203 + F4F5 + F6F7 + 210E = 0002 FFFD
FFFD + 0002 = FFFF

which checks OK.

Comments
It may look awkword to use a 1's complement addition on 2's complement machines. This method however has its own benefits.

Probably the most important is that it is endian independent. Little Endian computers store hex numbers with the LSB last (Intel processors for example). Big Endian computers put the LSB first (IBM mainframes for example). When carry is added to the LSB to form the 1's complement sum (see the example) it doesn't matter if we add 03 + 01 or 01 + 03. The result is the same.

Other benefits include the easiness of checking the transmission and the checksum calculation plus a variety of ways to speed up the calculation by updating only IP fields that have changed.

Using the select() and poll() methods (Linux\Unix)

Takeaway: The select() and poll() methods are used for multiplexing network sockets. Find out how to use these functions, and learn about their limitations.


The select() and poll() methods can be a powerful tool when you’re multiplexing network sockets. Specifically, these methods will indicate when a procedure will be safe to execute on an open file descriptor without any delays. For instance, a programmer can use these calls to know when there is data to be read on a socket. By delegating responsibility to select() and poll(), you don’t have to constantly check whether there is data to be read. Instead, select() and poll() can be placed in the background by the operating system and woken up when the event is satisfied or a specified timeout has elapsed. This process can significantly increase execution efficiency of a program. (If you are more concerned with performance than portability, we discuss some alternatives to select() and poll()toward the end of the article.)

As you will see, select() and poll() are very similar in functionality. Often, implementations of select() and poll() are mapped onto each other. For instance, in the Apache Portable Runtime, a core component of Apache 2.0, a portable interface is provided that mimics the poll() semantics. On platforms that do not have a native poll() implementation, the poll() semantics are mapped onto select(). On FreeBSD, the libc_r implementation of select() is merely a thin wrapper around the poll() system call.

select() description
The Single UNIX Specification, version 2 (SUSv2) defines select() as follows:
int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *errorfds, struct timeval *timeout);

It takes these parameters:
  • · int nfds - The highest file descriptor in all given sets plus one
  • · fd_set *readfds - File descriptors that will trigger a return when data is ready to be read
  • · fd_set *writefds - File descriptors that will trigger a return when data is ready to be written to
  • · fd_set *errorfds - File descriptors that will trigger a return when an exception occurs
  • · struct timeval *timeout - The maximum period select() should wait for an event

The return value indicates the number of file descriptors (fds) whose request event has been satisfied.

You can’t modify the fd_set structure by changing its value directly. The only portable way to either set or retrieve the value is by using the provided FD_* macros:
  • · FD_ZERO(fd_set *) - Initializes an fd_set to be empty
  • · FD_CLR(int fd, fd_set *) - Removes the associated fd from the fd_set
  • · FD_SET(int fd, fd_set *) - Adds the associated fd to the fd_set
  • · FD_ISSET(int fd, fd_set *) - Returns a nonzero value if the fd is in fd_set

Upon return from select(), FD_ISSET() can be called for each fd in a given set to identify whether its condition has been met.

With the timeout value, you can specify how long select() will wait for an event. If timeout is NULL, select() will wait indefinitely for an event. If timeout's timeval structures are set to 0, select() will return immediately rather than wait for any event to occur. Otherwise, timeout defines how long select() will wait. SUSv2 indicates that all compliant implementation will support a timeout of at least 31 days. Check out Listing A for a select() example.

poll() description
The poll() method attempts to consolidate the arguments of select() and provides notification of a wider range of events. The SUSv2 defines poll() as follows:
int poll(struct pollfd fds[ ], nfds_t nfds, int timeout);

It takes these parameters:
  • · struct pollfd fds[ ] - An array of pollfd structures
  • · nfds_t nfds - The number of file descriptors set in fds[ ]
  • · int timeout - How long poll() should wait for an event to occur (in milliseconds)

The return value indicates how many fds had an event occur.

A pollfd struct typically includes the following members:
  • · int fd - Indicates which fd to monitor for an event
  • · short events - A bitwise field that represents which events will be monitored
  • · short revents - A bitwise field that represents which events were detected in a call to poll()

The SUSv2 specification details the precise value and meanings of the event’s bitfield. In comparison to select(), poll() allows a greater degree of flexibility in determining what type of events can be processed. In addition to the read, write, and error notifications, poll() also supports explicit recognition of out-of-band and high-priority data.

Unlike select(), poll()'s timeout is a simple integer that represents how long poll() will wait for an event. A special value—usually -1 or the constant INFTIM, which is used by many older systems—specifies that poll() will wait forever for an event. Like select(), a 0 timeout value indicates that poll() should return immediately.

In Listing B, the select() example in Listing A is shown as poll().

Alternatives to select() and poll()
Even with the advantage of having multiplexed event notification via select() or poll(), other implementations exist that provide better performance. However, these implementations are not standardized across platforms. You must weigh the potential performance benefits of using one of these specialized implementations against the loss of portability. We will examine two of these alternatives: Solaris' /dev/poll and FreeBSD's kqueue.

As you will see, both implementations gain their key performance benefits by leveraging the fact that, in the real world, developers continually call select() or poll() with the same fds. To eliminate overhead when passing the same arguments each time, fds that are examined can be cached. This approach also works well when a large number of fds are monitored, because some select() and poll() implementations have scalability issues.

Solaris's /dev/poll
In Solaris 7, Sun introduced the /dev/poll device. To use /dev/poll, you first open /dev/poll as you would for a normal file. Then, you construct pollfd structures in a manner similar to the normal poll() call. These pollfd structures are then written to the open /dev/poll file descriptor. For the lifetime of this open handle, /dev/poll will now return events according to that pollfd structure. (Note that a special POLLREMOVE in the events field of the pollfd structure will remove that fd from /dev/poll's list.) A program retrieves the information from /dev/poll by calling a special ioctl (DP_POLL) and dvpoll structure. By using the dvpoll structure, the events that have occurred can be determined.

Linux support and other resources
A patch exists to add /dev/poll support for the Linux 2.4 series, but it has not currently been accepted into mainstream Linux kernel trees. On the FreeBSD site, you can check man pages for Solaris, Red Hat, and other operating systems.

FreeBSD's kqueue
Introduced in FreeBSD 4.1, FreeBSD's kqueue API is designed to address a wider range of notifications than any of the other alternatives presented here. The kqueue API provides several generic filters that allow mimicking of the poll() semantics (EVFILT_READ and EVFILT_WRITE). However, it also allows notification of file system changes (EVFILT_VNODE), process state changes (EVFILT_PROC), and delivery of signals (EVFILT_SIGNAL). For more information on kqueue, download Jonathan Lemon’s paper (PDF format) from the BSDCon 2000, "Kqueue: A generic and scalable event notification facility."

To build on the introduction to select() and poll() presented in this article, I highly recommend Advanced Programming in the UNIX Environment by W. Richard Stevens.