Understanding Regular Expressions

Details

Category: Linux Tools & Utilities

Published on Wednesday, 16 May 2012 15:59

Introduction

Learning and understanding Regular Expressions may not be as straight forward as learning ls command. However, learning Regular Expressions and effectively implementing them in your daily work will doubtlessly reward your learning effort by greater work efficiency and time savings. Regular Expressions is a topic which can easily fill up entire 1000 pages long book. In this article, we only try to explain the basics of Regular Expressions in a concise, non-geeky and example driven manner. Therefore, if you ever wanted to learn Regular Expression basics now you have a viable chance.

The intention of this tutorial is to cover a fundamental core of Basic Regular Expressions and Extended Regular Expressions. For this, we will use a single tool,and that will be the GNU grep command. GNU/Linux operating system and its grep command recognizes three different types of Regular Expressions:

• Basic Regular Expressions (BRE)
• Extended Regular Expressions (ERE)
• Perl Regular Expressions (PRCE)

The difference between Basic Regular Expressions and Extended Regular Expressions well be explained momentarily.

What is a Regular Expression

Regular Expression provides an ability to match a "string of text" in a very flexible and concise manner. Where a "string of text" can be further defined as a single character, word, sentence or particular pattern of characters. Well known abbreviations for "Regular Expression" include regex and regexp.

Simple Regular Expression example

The simplest building block of any regular expression is a character. We can use grep to find for any particular character from within a text of any given non-binary file. For example, here is a content of our regex.txt sample file:

$ cat regex.txt 
grep stands for:                                                                   
global                                                                             
regular                                                                            
expression                                                                         
print

Read more: Understanding Regular Expressions

C development on Linux - Building a program - X.

Details

Category: Linux Development

Published on Saturday, 21 April 2012 19:36

Introduction

After all that theory and talking, let's start by building the code written through the last nine parts of this series. This part of our series might actually serve you even if you learned C someplace else, or if you think your practical side of C development needs a little strength. We will see how to install necessary software, what said software does and, most important, how to transform your code into zeros and ones. Before we begin, you might want to take a look at our most recent articles about how to customize your development environment:

• Introduction to VIM editor
• Introduction to Emacs
• Customizing VIM for development
• Customizing Emacs for development

Building your program

Remember the first part of our C Development series? There we outlined the basic process that takes place when you compile your program. But unless you work in compiler development or some other really low level stuff, you won't be interested how many JMP instructions the generated assembler file has, if any. You will only want to know how to be as efficient as possible. This is what this part of the article is all about, but we are only scratching the surface, because of the extensiveness of the subject. But an entry-level C programmer will know after reading this everything needed to work efficiently.

The tools

Besides knowing exactly what you want to achieve, you need to be familiar with the tools to achieve what you want. And there is a lot more to Linux development tools than gcc, although it alone would be enough to compile programs, but it would be a tedious task as the size of your project increases. This is why other instruments have been created, and we'll see here what they are and how to get them. I already more than suggested you read the gcc manual, so I will only presume that you did.

make

Imagine you have a multi-file project, with lots of source files, the works. Now imagine that you have to modify one file (something minor) and add some code to another source file. It would be painful to rebuild all the project because of that. Here's why make was created: based on file timestamps, it detects which files need to be rebuilt in order to get to the desired results (executables, object files...), named targets. If the concept still looks murky, don't worry: after explaining a makefile and the general concepts, it will all seem easier, although advanced make concepts can be headache-inducing.

Read more: C development on Linux - Building a program - X.

Learning Linux Commands: nice & renice

Details

Category: Linux Commands

Published on Friday, 30 March 2012 10:59

Introduction

User's ability to attach a priority value to its own process upon execution determines whether you are being nice to your fellow users on the same system. Are you being nice or you simply abuse system resources for no apparent reason? In this article you will learn how to manage your processes in terms of how much processing power they consume and how to change a priority value of your processes using nice & renice Linux command.  We will start with some basic theory about what is process, process scheduling, how to fork new process and then we move to nice command and explain how to change process priority value.

What is process

In simple words a process is a naming convention used by Linux to assume role of a running program. A process is a collection of rules by which any particular program makes use of assigned processor time, memory and I/O resources. Each process running on a Linux system has its own Process ID ( PID ) by which it can be monitored and administered.

Linux kernel is designed to collect various information about each process. These include, but not limited to:

• process status ( runnable, sleeping, zombie or stopped )
• process execution priority ( niceness)
• information about used resources
• owner of the process
• what network ports and files had each particular process opened
• and more...

Now that we have some idea on what the process is we can go ahead and create some process. To do this simply open your terminal and execute yes command in background and redirecting its output to /dev/null:

$ yes > /dev/null &
[1] 5997

After that use ps -l command we cant extract the information about our yes process:

$ ps -l
F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
0 S  1000  5830  3283  0  80   0 -  6412 wait   pts/0    00:00:00 bash
0 R  1000  5997  5830 99  80   0 -  1757 -      pts/0    00:00:09 yes
0 R  1000  5998  5830  0  80   0 -  2399 -      pts/0    00:00:00 ps

 From the above we can read to following information:

• F - FLAG: the process did not start with superuser privileges. Otherwise we would see number 4 or sum of 1 and 4. Check man ps for more info.
• S - STATE: process is currently running
• UID - User ID of user who initiated the process. UID is actual an alias for EUID ( Effective User ID )
• PID - Process ID of our yes command is 5997
• PPID - Parent Process ID. This is a process ID from which our yes command had been forked from. In this case it is bash with PID 5830.
• C - Integer value of Processor utilization in % .
• PRI - Process priority. The higher value the lower priority.
• NI - Nice value with range from -20 to 19.  The higher value the more nice you are to other users in other words the higher value the lower priority.

Process scheduling

This section can be skipped if you do not feel like to go into more details about process priority and scheduling. Here we will attempt to describe Linux process with some easy to understand rudimentary summarization as this topic can span multiple pages and that would be the only introduction.

From our point of view we need to understand that Linux ( Linux kernel version >= 2.6 ) scheduler is preemptive. This is a ability which allows kernel to preemptively choose to execute higher priority task over the ones with a lower priority.  Furthermore, kernel separates priority lists  to real-time tasks and user tasks raging from 1 - 100 and 101 - 140 respectively.

 Furthermore, Linux kernel assigns to higher priority tasks longer quantum time and lower priority tasks shorter quantum time this is approximately 200 ms and 10 ms, respectively. In other words every task is allowed to execute only if it has some remaining time slice. Therefore, shorter time slice to execute causes process to stay shorter time in active runqueue and thus consume less resources. Once the process time slice is empty, process is moved to expired runqueue where it's priority is re-calculate and then moved again to active runqueue. This relationship is illustrated on the diagram on your right. It is important to mention that both  where both active and expired runqueues contain lists of tasks sorted by their priority.

Read more: Learning Linux Commands: nice & renice

Customizing vim for development

Details

Category: Linux Tools & Utilities

Published on Friday, 30 March 2012 09:20

Introduction

Of course, we wouldn't have had it any other way: we wanted to be fair, as pledged, so here is the vim article, which is a counterpart of our last one on how to make your editor the perfect programming environment. So you must have the following profile for this article to be really useful to you: you know your way around programming, so you subsequently know what you would like in an editor, and you also know your way around vim, preferably more than what we talked about in the article dedicated to it. If you read the customizing emacs article, you already have a good idea on how this article is going to be structured. If you were directed here from somewhere else, here's what we're gonna do: we'll take some popular programming language (space permitting) and show you how to tweak vim so it will became more fit for coding in that language.

The languages

Although vim is written entirely in C, there is something named vimscript that makes creating/editing settings, sort of like Elisp in emacs, although this is a loose comparison. Please remember that whatever will be talked about here is only about vim. Not BSD vi, not some vi extension for another editor, just vim. That is because although you can learn the basics on, say, nvi, the things that interest us (since you already know the basics) will only work on vim. Of course, some recent version, not older than 7.3.x. Many things will probably work on 7.x or maybe even 6.x, but there's no guarantee.

Just as before, a little advice: although this is influenced by personal preference, experience says it works; namely, install scripts/addons/color schemes directly from the source, regardless if your distro offers it as well. That's because many maintainers tend to package stuff with respect to their personal preference, which might or might not be in concordance with yours. Installing such addons is as simple as copying a file to a location, nothing more. And, for your convenience, we'll tell you how to install via your package manager anyway.

The distributions I have available to me at this point are Debian, Fedora, Gentoo and Arch. I will do a search for the 'vim' keyword on each of them and give you some tips and pointers on what you can install, then we'll go language-specific.

Read more: Customizing vim for development

Customizing emacs for development

Details

Category: Linux Tools & Utilities

Published on Wednesday, 28 March 2012 14:43

Introduction

First and foremost, a word of warning: while the previous articles were centered around the beginner, this article is for more advanced users, that already "speak" a programming language or two, and want to customize their editor towards it being ideal for the task. So you are expected to be more or less proficient in the ways of emacs, to be able to use it for day to day tasks and have the ability and desire to learn something new. After all, it will be for your own reward, and your programming tasks will become more efficient. Our approach will be to take a few popular languages, show you how to configure emacs for the perfect development environment, then move on to the next language. Emacs configuration is made in a Lisp dialect called Elisp, but don't worry if you don't know it yet, we'll tell you what you need.

The languages

First, some background. This article is about emacs, not about any derivative like mg or jed that might or might not offer the desired functionality. That's because many derivatives were born from the need of creating a smaller emacs, since the original is pretty big, admittedly. So in the process of removing functionality there might just be just some functionality that's getting removed and we probably will need here. In short, emacs-only. Second, the files. In our examples, besides customizing the ~/.emacs file, we will create a directory named ~/.emacs.d/ where we will place our modes. Just as emacs knows what kind of syntax highlighting, indentation, etc. to use for several types of text, like HTML, TeX, C source code, and others, via modes, we can add/modify modes to our liking, to this is what we'll do. Practically speaking, a mode is a file with a .el extension (from Elisp) that will be dropped in ~/.emacs.d, then ~/.emacs will be altered for the editor to "know" about the new extension. You'll see in a few moments, depending on how fast you read.

C/C++

It seems obvious that we will start with C, because it's the lingua franca of Unix and Linux. Emacs is also written (partially) in C, so there's another reason for you. Although emacs knows how to detect the language you're writing in, based on file extension, you may need further customization. In our article we said that when in C mode, emacs already conforms to the GNU coding guidelines regarding tabs, indentation and so forth. But if you want to contribute code to the Linux kernel, the situation changes. Obviously you already have the kernel try downloaded somewhere, but if you don't, here's an excerpt from Documentation/CodingStyle (but if you're working with the kernel guys, this file must be under your pillow):

Tabs are 8 characters, and thus indentations are also 8 characters.
There are heretic movements that try to make indentations 4 (or even 2!)
characters deep, and that is akin to trying to define the value of PI to
be 3.
[...]
The other issue that always comes up in C styling is the placement of
braces.  Unlike the indent size, there are few technical reasons to
choose one placement strategy over the other, but the preferred way, as
shown to us by the prophets Kernighan and Ritchie, is to put the opening
brace last on the line, and put the closing brace first, thusly:

        if (x is true) {
                we do y
        }

Read more: Customizing emacs for development