Linux System Programming (Part – 2)

Hi Guys, Its been a lot of time since my last post. But anyway we are again here for more knowledge sharing. Now we move on to C program Storage Classes.

A storage class defines the scope (visibility) and life-time of variables and/or functions within a C Program.There are four types of storage classes categorized in C:

1. Auto Variables
2. Extern Variables
3. Static Variables
4. Register Variables

Auto Variables
The Auto storage class is the default storage class for all local variables. All the variables declared inside the function which are local to that are Auto Variables.
And Auto Variables are stored in Stack section of C program segment space (Because all functions are stored in Stack section).

void main()
{
int var;
auto int var;          //Both are same.
}

It has garbage value as initial/ default value. Scope is local.

Extern Variables
The extern storage class is to give a reference of global variable and it is visible to ALL the program files. The extern keyword is used before a variable to tell compiler that the variable is declared somewhere else.
The extern declaration does not allocate storage for variables, but all it does is point the variable name at a storage location that has been previously defined. You can use extern with a function also to use it from other location.

It has zero value as initial/ default value. Scope is global.

Compile below examples using $ gcc File1.c File2.c and then run ./a.out

Result : 5

Using extern in same file

Result : 10

Static Variables
The static keyword instructs the compiler to persist local variable during the life-time of the program. Instead of creating and destroying a variable each time it comes into and goes out of scope.

Static is initialized only once and remains into existence till the end of the program. The static modifier may also be applied to global variables. When this is done, it causes that variable’s scope to be restricted to the file in which it is declared. Static variables are stored in Data section of C program segment space.

The “static” keyword before a function name makes it static. Access to static functions is restricted to the file where they are declared. Another reason for making functions static can be reuse of the same function name in other files.

Result : i is 4 and counter is 2
i is 5 and counter is 1
i is 6 and counter is 0

Register Variables
register keyword inform the compiler to store the variable in register instead of memory. It has the faster access than normal variable.
It can’t have the unary ‘&’ operator applied to it because it does not have a memory location at all.

register int num;

That’s it for the current Article. I will come back with more topics on Linux Programming. Any Query ? please ask in the comment section.

Shared memory in Linux : Continued

Mmap is 2nd method of shared memory to reduce the transactional overhead associated with these external read and write operations. You can map files directly to the memory to speed up disk operations.Processes using shared memory areas must set up a signal or semaphore control method to prevent access conflicts and to keep one process from changing data that another is using.

Use the shmat services under the following circumstances:

• For 32-bit application, eleven or fewer files are mapped simultaneously, and each is smaller than 256MB.
• When mapping files larger than 256MB.
• When mapping shared memory regions which need to be shared among unrelated processes (no parent-child relationship).
• When mapping entire files.

Use mmap under the following circumstances:

• Portability of the application is a concern.
• Many files are mapped simultaneously.
• Only a portion of a file needs to be mapped.
• Page-level protection needs to be set on the mapping.
• Private mapping is required.

Shared memory in Linux

Shared memory is a accessible region so that more than one process can access that region fast. When write access is allowed for more than one process, semaphore can be used to prevent inconsistencies and collisions.
There are different system call for accessing, controlling shared memory segment.

First we need to access shared memory segment :
shmget : shmget() returns the identifier of the shared memory segment associated with the value of the argument key.
int shmget(key_t key, size_t size, int shmflg);
Size : Shared memory segment size.
Shmflag : flags , whether to create a new segment , or fail when segment already exists.
A valid segment identifier, shmid, is returned on success, -1 on error.

shmat : shmat()  attaches  the shared memory segment identified by shmid to the address space of the calling process.
void *shmat(int shmid, const void *shmaddr, int shmflg);
If shmaddr is NULL, the system chooses a suitable (unused) address at which to attach the segment.
If SHM_RDONLY is specified in shmflg, the segment is attached for reading and the process must have read permission for the segment.
On success shmat() returns the address of the attached shared memory segment; on error (void *) -1 is return.

shmdt : shmdt()  detaches  the  shared  memory  segment  located  at  the address specified by shmaddr from the address space of the calling process.
int shmdt(const void *shmaddr);
On success shmdt() returns 0; on error -1 is returned

Shmctl : shmctl() performs the control operation specified by cmd on the shared memory segment whose identifier is given in shmid.
int shmctl(int shmid, int cmd, struct shmid_ds *buf);

Note : After a fork(2) the child inherits the attached shared memory segments.
After an execve(2) all attached shared memory segments are detached from the process.

Example : We have two process which will share a common memory segment. one  server process will write to that particular region and then wait for client process to read that content. Client process will read the data from shared memory segment.

SERVER PROCESS

ashwani@ash:~/C-Programming$ cat shm_server.c 
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdio.h>
#include<stdlib.h>

#define SHMSZ 30

main()
{
    char c;
    int shmid;
    key_t key;
    char *shm, *s;

    /*
     * We'll name our shared memory segment
     * "5678".
     */
    key = 1234;

    /*
     * Create the segment.
     */
    if ((shmid = shmget(key, SHMSZ, IPC_CREAT | 0666)) < 0) {
        perror("shmget");
        exit(1);
    }

    /*
     * Now we attach the segment to our data space.
     */
    if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {
        perror("shmat");
        exit(1);
    }

    /*
     * Now put some things into the memory for the
     * other process to read.
     */
    s = shm;

    for (c = 'a'; c <= 'z'; c++)
        *s++ = c;
    *s = NULL;

    /*
     * Finally, we wait until the other process 
     * changes the first character of our memory
     * to '*', indicating that it has read what 
     * we put there.
     */
    while (*shm != '*')
        sleep(1);

    exit(0);
}

CLIENT PROCESS
ashwani@ash:~/C-Programming$ cat shm_client.c 
/*
 * shm-client - client program to demonstrate shared memory.
 */
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdio.h>
#include<stdlib.h>

#define SHMSZ 30

main()
{
    int shmid;
    key_t key;
    char *shm, *s;

    /*
     * We need to get the segment named
     * "5678", created by the server.
     */
    key = 1234;

    /*
     * Locate the segment.
     */
    if ((shmid = shmget(key, SHMSZ, 0666)) < 0) {
        perror("shmget");
        exit(1);
    }

    /*
     * Now we attach the segment to our data space.
     */
    if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {
        perror("shmat");
        exit(1);
    }

    /*
     * Now read what the server put in the memory.
     */
    for (s = shm; *s != NULL; s++)
        putchar(*s);
    putchar('\n');

    /*
     * Finally, change the first character of the 
     * segment to '*', indicating we have read 
     * the segment.
     */
    *shm = '*';

    exit(0);
}
OUTPUT :
ashwani@ash:~/C-Programming$ ./a.out 
abcdefghijklmnopqrstuvwxyz

What is Struct hack OR Dyanamic array in C ?

In C Programming , we cannot declare memory for an array at dynamic time, but there is a way to hack this ….

You can do this by declaring an empty array in a structure and define storage for that array at running time.

ashwani@ash:~/inter$ cat dynamic_array.c 
#include<stdio.h>
#include<stdlib.h>
struct abc
{
    int id;
    int records[];
};
struct abc *abc_ptr;
int main()
{
 	abc_ptr = malloc(sizeof(struct abc) + (1000 * sizeof(int)));
}

How to find duplicate element from a list of unsorted integers using Hash table ?

ashwani@ash:~/inter$ cat find_duplicate.c 
#include<stdio.h>
int find_unique(int *p, int count)
{
    static int b[50], i = 0;
	int count1 = 0;
	while(count1 < count)
	{
		i =(*p) % 1;
		if(b[*p] != 0)
			printf("duplicate value is %d\n", *p);
		else
			b[*p] = 1;
		p++;
		count1++;
	}
}
int main()
{
	int count = 0;
	int a[] = {5,6,7,1,4,5,11,22,33,44,55,66,77,2,22};	
	count  = sizeof(a)/4;
	find_unique(a,count);
}
ashwani@ash:~/inter$ ./a.out 
duplicate value is 5
duplicate value is 22

The best solution may be doing XOR of the number to find a duplicate element without wasting space, 
but above mentioned solution does this thing in O(n) time in a single traversal.

What is the difference using a library function or a system call?

Libarary functions: usually are just a wrapper function around your system call, they may or may not use system call depanding on the requirement of the function. if library function required something from kernel, then they have to use system call for context switch from user space to kernel space.

C libarary functions are usually buffered I/O functions, they have their own buffer in user space to speed up the read and write requests but when we compare with system call they directly process the request to kernel.

e.g. For system call, one process does write request two times (w.e. write system call) then there are two context switch happens from user space to kernel space consuming more CPU. But in case of library functions (fwrite), when two write request happens by a process, library functions doesn’t call write system call immendiatly, they will buffer the data and once buffer will full from one or more than one system call they will request for write system call and only one context switch will happen…

Additionally library functions check for resource limits, return value and some other tasks also.

Interaction with hardware using IOCTL

int ioctl(int d, int request, …);

d: First argument d must be an open file descriptor.
Request : Second argument is a  device-dependent  request  code.An  ioctl()  request  has  encoded  in it whether the argument is an in parameter or out parameter, and the size of the argument argp in bytes.
Third argument  is  an  untyped  pointer  to memory.it usually passed as an argument. if there are more than one argument needs to pass, address of a structure gets passed.it can be any type.

Basic example of ioctl call is Cd rom eject.

root@ash:/home/ashwani/IOCTL# cat ioctl-cdrom.c 
#include  <fcntl.h> 
#include  <linux/cdrom.h> 
#include  <sys/ioctl.h> 
#include  <sys/stat.h> 
#include  <sys/types.h> 
#include  <unistd.h> 
#include<stdio.h>
int main  (int argc, char* argv[]) 
{
    int status;
    /* Open a file descriptor to the device specified on the command line.  */ 
    int fd = open ("/dev/cdrom", O_RDONLY| O_NONBLOCK); 
    /* Eject the CD-ROM.  */ 
    ioctl(fd, CDROMEJECT,0);
    /* Close the file descriptor.  */ 
    close  (fd); 
    return 0; 
}

ioctl values are listed in <linux/cdrom.h>. 
some examples are: 
        CDROMPAUSE		Pause Audio Operation
	CDROMRESUME		Resume paused Audio Operation
	CDROMPLAYMSF		Play Audio MSF (struct cdrom_msf)
	CDROMPLAYTRKIND		Play Audio Track/index (struct cdrom_ti)
	CDROMREADTOCHDR		Read TOC header (struct cdrom_tochdr)
	CDROMREADTOCENTRY	Read TOC entry (struct cdrom_tocentry)
	CDROMSTOP		Stop the cdrom drive
	CDROMSTART		Start the cdrom drive
	CDROMEJECT		Ejects the cdrom media

Usually IOCTL system call is used for writing operations for your device driver.

Like you want to Write some user space utility that interacts with your device driver , do some operation or get some information from the device driver and give this information to the user then IOCTL system call is the most preferred way to accomplish this task.

Like above example mentioned you can write some operation at the device driver and give some Macro to the user in form of header file so that user can perform some operation.

How a debugger works internally ?

We know how to use some debugger like ptrace or GDB (GNU debugger) but we don’t how these debugger internally Works.

A debugger process, spawns a process to be traced and controls its execution with the “ptrace system call”, setting
and clearing break points, and reading and writing data in its virtual address space. means from debugger process address space we need to access memory of a process that is being debugged.

PTRACE system call :

The  ptrace()  system  call provides a means by which a parent process may observe and control the execution of another process, and examine and change its core image and registers.

long ptrace(enum __ptrace_request request,  pid_t pid,  void *addr,  void *data);

Four parameters are :

• request : It determines the action to be performed whether it is reading traced process registers or altering values in its memory.e.g. PTRACE_TRACEME, tells the kernel that calling process wants its parent to debug itself.
• pid : It specifies pid of the traced process.
• addr : address in child memory (optional)
• data :  (optional argument)

Pseudo code for Debugger Process :

/* 1. Firstly debugger process spawns a child process and then child invokes Ptrace system call and as a result kernel sets a 
trace bit in the child process table entry and then child execs the program being traced. 
kernel executes the exec call as usual but finds that trace bit is set then sends a trap signal to the child.
Kernel checks for signal when returning from exec system call finds the trap signal which it has sent to itself and executes code
for process tracing as a special case for handling signals.
since trace bit is set in process table entry, child awakens the parent process from sleep in the wait system call and does a 
context switch
*/ 

if((pid == fork()) == 0)   
{
      /* CHILD Traced process */
      ptrace(0,0,0,0);
      exec("name of the traced process here");
}
/* 2. When the traced process awakens the debugger, the debugger returns from wait , reads user input commands, and
 converts them to a series of ptrace calls to control the child (traced) process.  */
 
/* debugger process continues here */
for(;;)
{
      wait((int *) 0)
      read("input for tracing instructions");
      ptrace(cmd,pid,...);
      if(quitting trace)
           break;
}

Breakpoints are not part of ptrace API services. But we can alter the memory, and receive the Traced process signals.

Then How breakpoint works:

1. The debugger reads (ptrace peek) the binary instruction stored at this address, and saves it in its data structures.
2. It writes an invalid instruction at this location. What ever this instruction, it just has to be invalid.
3. When the Traced process reaches this invalid instruction, the it won’t be able to execute it (because it’s invalid).
4. In modern multitask OS’s, an invalid instruction doesn’t crash the whole system, but it gives the control back to the OS kernel, by raising an interruption (or a fault).
5. This interruption is translated by Linux into a SIGTRAP signal, and transmitted to the process … or to it’s parent, as the debugger asked for.
6. The debugger gets the information about the signal, and checks the value of the Traced process instruction pointer (i.e., where the trap occurred). If the IP address is in its breakpoint list, that means it’s a debugger breakpoint (otherwise, it’s a fault in the process, just pass the signal and let it crash).
7. Now that the Traced process is stopped at the breakpoint, the debugger can let its user do what ever s/he wants, until it’s time to continue the execution.
8. To continue, the debugger needs to :
   1. write the correct instruction back in the Traced process memory,
   2. single-step it (continue the execution for one CPU instruction, with ptrace single-step) and
   3. write the invalid instruction back (so that the execution can stop again next time).
   4. let the execution flow normally.

How Fork System Call Works? What is shared between parent and child process ?

In Unix, Fork system call is the only way for user to create a new process.

PID = fork();

It creates two process having identical copies of their user level context except returned PID.kernel allocates child process as a PID=0.

How Fork works:

Kernel does the following steps:

1. Firstly it checks for available kernel resources (memory or disk) to complete the fork process successfully.

2. Kernel finds a slot for child process in the process table to start construct the context of child process and confirms that user already don’t have too many process running.
Here it gives a unique PID number to the child process. Kernel gives PID number according to next higher free ID. When id no. reaches max, it again start from 0.

3. Kernel initializes the child process table slot, copying various field from the parent slot.
The kernel assigns the parent process ID field in the child slot, putting the child in· the process tree structure, and initializes various scheduling parameters, such as the initial priority value, initial CPU usage, and other timing fields.

4. The kernel now adjusts reference counts for files with which the child process is automatically associated
* the child process resides in the current directory of the parent process The number of processes that currently access the directory increases by 1.
* if the parent process had ever executed the chroot system call to change its root, the child process inherits the changed root and increments its inode reference count.
* The kernel searches the parent’s user file descriptor table for open files known to the process and increments the global file table reference count associated with each open file. Not only does the child process inherit access ri ghts to open files, but it also shares access to the files with the parent process because both processes manipulate the same file table entries.

5. Kernel allocates memory for the child process u area , regions, and auxiliary page tables, duplicates every region in the parent process and attaches every region to the child process.

6. The kernel then :creates a dummy context layer (2) for the child process, containing the saved register context for context layer ( 1 ) . It sets the program counter and other registers in the saved register context so that it can “restore” the child context.

7. When the child context is ready, the parent completes its part of fork by changing the child state to “ready to run Gn memory) ” and by returning the child process ID to the user.

So from the above What is similar between parent and child process :

Both processes share files that the parent had open at the time of the fork , and the file table reference count for those files is one greater than it had been.
Similarly, the Child process has the same current directory and changed root (if applicable) as the parent, and the inode reference count of those directories is one greater than it had been.
The processes have identical copies of the text, data, and (user) stack regions.
the system implementation determine whether the processes can share a physical copy of the text region.

Difference between parent and child process :
Neither process can access the variables of the other process.
PID of parent and child process are different.
Process locks, text locks, and data locks are not inherited by the child process.
The child process utime, stime, cutime, and cstime subroutines are set to 0.
Any pending alarms are cleared in the child process.
The set of signals pending for the child process is initialized to an empty set

In Linux we use the concept of COW (Copy on Write) :

In case of COW , parent process and child process share the same process address space, until one of the process don’t write , if some process write, then we create for copying of address space.

What is System Call ? What happens when invoke system call from user space ?

System Call : Whenever we wanted to do some operation from the Kernel (like accessing hard disk), we can not execute the operation directly from a user space (No Permissions). To interact user space functions to kernel space functions, we need System Call.

How System Call works ?

Whenever we invoke a C library function from user space then a list of predefined steps happened:

• The C compiler uses a predefined library of functions that have the names of the system calls,
  thus resolving the system call references in the user program to what would otherwise be undefined names.
• It looks in the system call table to find the  system call entry no. for invoked system call.
• It determines no. of parameters we are passing in a System Call.
• Copy all the parameters from user space to Uarea.
• Save Current context of the process for return purpose.
• Then it invokes Operating System Trap (for Intel 0x80 instruction) to switch user space mode to kernel space. With this instruction it passing System call number and parameters to the kernel in the form of Specific register or on the Stack.
• From the above, kernel determines the specific system call user space is invoking.
• The kernel calculates the ( user) address of the first parameter to the system call by adding  ( or subtracting, depending on the direction of stack growth ) an offset to the user stack pointer, corresponding to the number of parameters to the system call
• Finally kernel calls the appropriate System call routine i.e. a interrupt handler.
• After executing the system call routine, Kernel determines if there was some error ?
  if True : It set the “carry” bit for the PS register and copying the error number into the register 0 location
  if no : Kernel clears the carry bit in PS register and copy the return value of system call into register 0 and 1.
• Finally return the value to the user program.

Above we mentioned about u-area, what it is ?

In addition to the text, data, and stack segment, the OS also maintains for each process a region called the u area (User Area ,in linux in the form of task_struct structure). The u area contains information specific to the process (e.g. open files, current directory, signal action, accounting information) and a system stack segment for process use. If the process makes a system call (e.g., the system call to write in the function in main ), the stack frame information for the system is stored in the system stack segment.