http://www.mysite.com/users/images/user123/profile.jpg

Now, on changing this image, sometimes browsers don’t clear cache and thus show the last image. In Order to solve this, use a random number at the end of the image location and you will be able to see that image. as browsers will not cache it.

http://www.mysite.com/users/images/user123/profile.jpg?123

One method that I used is:

$stamp = time();
$imageLocation  = 'http://www.mysite.com/users/images/user123/profile.jpg?'.$stamp
echo $imageLocation

This should work.

Consider this script [parentProcess.php]

$backgroundCall = 'php childProcess.php > /dev/null &';
 
shell_exec($backgroundCall );
 
echo "Hello world ! My background job is fired";
 
// other code , redirects etc

When you call a background file with the ‘amersand’ in the end, it sure becomes a background file.
But it doesnot mean that the user will get control back of the browser. If you are using sessions,
the webpage will be busy until the background process finishes execution.

Why is that ?

[There is bug described here and it is still not solved even in the php 5.0+ versions.]

What happens is that the child process which is called by the system() ,exec() or shell_exec() command
holds the file lock on the parent process, thus locking the session file.
Thus, the session file being locked session_start(); function cannot access the file.
This is due to to open file descriptors locking the session file.Your entire session will be locked, so none of the pages of the website can be opened ( which are under session control )
unless you change the browser or delete cookies.

Solution:

There are places on the internet which provide C code and perl code to do that. But PHP has its own
function , which sadly is ignored session_write_close(); This function can be used to trigger the close of session. Thus instructing the session
that there is nothing to write until the next session start. Triggering this before firing background process will not hold locks on
the session file.

$backgroundCall = 'php childProcess.php > /dev/null &';
 
session_write_close();
 
shell_exec($backgroundCall );
 
echo "Hello world ! My background job is fired";
 
// other code , redirects etc

Ping is a Unix utility that sends ICMP ECHO to the server ( destination machine ) from the client ( originating machine ). It takes its name from a submarine sonar search – you send a short sound burst and listen for an echo – a ping – coming back. There is a myth that Ping is actually an acronym for the words ‘Packet INternet Groper’, but there is no proven evidence to support the statement.

For years, system admins, managers, developers, and the-next-door-jack have been using this utility, primarily to check if the ‘target system is alive’. Ping, is thus undercredited by deeming it as an utility which has only one function. “Type in ping <target_machine> , if it gives response the machine is UP, if it doesn’t either machine is down OR network is down. “NOT NECESSARILY TRUE!

What (other) Information can a ‘ping’ give you ?

1. If the <target_machine> is alive [ there are cases where this doesn't work ]
2. How long each packet exchange took
3. Interframe Gap
4. Reports other ICMP messages that might otherwise get buried in the system software
5. Exponential Moving Average
6. How occupied is the target system including the network that routes from host to the target.

Now, how ‘you’ use this information to draw conclusions and metrics, is upon you. I’ll show some examples in the later part of this article.

Lets see a sample Ping Output

-bash-3.00$ ping www.google.com
PING www.l.google.com (64.233.169.104) 56(84) bytes of data.
64 bytes from yo-in-f104.google.com (64.233.169.104): icmp_seq=0 ttl=246 time=7.97 ms
64 bytes from yo-in-f104.google.com (64.233.169.104): icmp_seq=1 ttl=246 time=13.7 ms
64 bytes from yo-in-f104.google.com (64.233.169.104): icmp_seq=2 ttl=246 time=9.69 ms
64 bytes from yo-in-f104.google.com (64.233.169.104): icmp_seq=3 ttl=246 time=8.15 ms
 
--- www.l.google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss
rtt min/avg/max/mdev = 7.971/9.884/13.716/2.313 ms

pretty normal eh ! Whats so new in that ?
Now, I can bet 50% of people who read this may not know the significance of all the digits or abbreviations in the output. It is a good idea to have a detailed knowledge of it in order to understand what is actually happening.
This will also help you to change the data in order to detect certain problems.

1. When you ping www.google.com it replies www.l.google.com which is actually a CNAME in the DNS database.
2. The second line has the IP address of the host that is found. This IP address is found from the DNS query to the DNS database. ( I will be writing an article about how the DNS query actually works ! Watch out for it )
3. Line two has two numbers (56 and 84 ) as bytes of data while all the reply has another (64) bytes of data. Now what are these ?

PING is an ICMP Packet with 20 bytes of IP header + 8 bytes ICMP header + XX byes of data ( or call it the payload )

IN default case, the whole frame is set to 64 bytes That is 20 IP header + 8 ICMP header + 56 bytes of payload. That is how you get all these numbers .. never thought of it isn’t it ?

image reference : www.caida.org

4. The next four lines show the successful reply from the IP address and their reverse DNS host names. See how the host names actually differ in all the first three lines of the output ? That is the magic of DNS and load balancing, which I will be discussing in a n separate article.

5. These also have 3 major things: (a) icmp_seq (b)ttl and (c)time in miliseconds

(a) icmp_seq number is the sequence number of packets that are transmitted and received back by the client. If a sequence number is missing OR if there is a gap in the sequence number that means that the client is sending more than what the server can take. It is called source quench. As ICMP is a lower level protocol it can only detect errors not correct it ( as IP or TCP does ).

how to simulate source quench ?

Image Courtesy: www.cartoonstock.com

You can use the command -i0 :this switch will set interval to 0 , this means that the client will continuously keep on sending packets without waiting for a response from the server. This will result in server getting busy and thus dropping packets.
You can also use the switch -s500 : this switch will set the payload along with ICMP header size to 500 and the server has to accept and process more data from the server ( A ping with packet larger than 64 is called The Ping of Death )
You can use both at the same time to get good results

ping -i0 -s500 www.google.com

Lets see an output:

64 bytes from host232.hostmonster.com (74.220.215.232): icmp_seq=24 ttl=50 time=114 ms
64 bytes from host232.hostmonster.com (74.220.215.232): icmp_seq=25 ttl=50 time=123 ms
64 bytes from host232.hostmonster.com (74.220.215.232): icmp_seq=27 ttl=50 time=112 ms

Here we see that the icmp sequence number 26 is lost. This indicates that the server is busy and thus cannot accept data at these throughputs. Now, in production environment something like this can be used to compare past ‘good’ results to a problem scenario. If all of a sudden you find that there is a large packet drop on the ping with the normal 64 byte 0 second interval packets, you can judge that there could be something wrong with the either devices OR the network pipe OR the internet connection itself could be slow. Comparing these figures on a different server / different network / different subnet would help you determine the problem area, and in some cases you can actually pinpoint the culprit.

(b) TTL

Any IP packet that gets sent out will have a TTL field which normally is set to a relatively high number (in the case of ping the default TTL is 255). As the packet traverses the network, the TTL field gets decreased by one by each HOP ( or router ) it goes through; when the TTL drops to 0, the packet is discarded by the router. The IP specification says that the TTL should be set to 60 (though it’s 255 for ping packets). The main purpose of this is so that a packet doesn’t live forever on the network and will eventually die when it is deemed “lost.” But for ping purposes, it provides additional information. The TTL can be used to determine approximately how many router hops the packet has gone through. If the TTL field varies in successive pings, it could indicate that the successive reply packets are going via different routes, which isn’t a great thing.

So a TTL of 245 for gogole.com means that it took (255-245 = ) 10 hops to reach google.com server.
On a production environment, in a ping output a low TTL means that it took the packet higher number of hops to reach to the destination. Time to check your network routing table ?

(c) time

The time is the time in milliseconds (ms) that it took to reach to the source and back. Thus it is also called RTT or the Round Trip Time. In a production environment a HIGH RTT suggests that there is a congestion in the network.

6. After the ping terminates ( normally ctrl+c OR with a switch -c<#> where <#> is the count before termination ) you will see number of packets transmitted, number failed and the percentage failed. In this example output its a 100% success but In the above source quench example we can see the number of packets failed and the percentage non zero.

7. The min/avg/max/mdev is the minimum / average / maximum / standard deviation of the round trip times. To have a good number and analysis, send at least 50 to 100 packets.

What else ?

Lets take a look at the end of the output:

64 bytes from yo-in-f99.google.com (64.233.169.99): icmp_seq=29 ttl=245 (truncated)
--- www.l.google.com ping statistics ---
30 packets transmitted, 30 received, 0% packet loss, time 540ms
rtt min/avg/max/mdev = 7.560/7.996/8.609/0.257 ms,  ipg/ewma 18.632/7.945 ms

Focus on the line: ipg/ewma . That is: inter-packet gap / exponential moving average.

InterPacket Gap (measured in seconds)

The Inter-packet gap (or the inter-frame gap) is an idle time period appended to the end of every frame by the ethernet adapter. This idle time gives the network media a chance to stabilize, and other network components time to process the frame. On specifying the i0 or the -f switch in ping we can get the output resulting ping statistics which gives the current ipg of the system.
The minimum interframe gap is 96 bit times (the time it takes to transmit 96 bits of raw data on the medium), which is
9.6 μs for 10 Mbit/s Ethernet,
960 ns for 100 Mbit/s (fast) Ethernet,
96 ns for 1 Gbit/s (gigabit) Ethernet, and
9.6 ns for 10 Gbit/s (10 gigabit) Ethernet.
This is the minimum gap as specified in Ethernet protocol and required for a non-colliding transmission. There are ways to reduce it for a faster UDP transmissions, but can cause heavy collisions if other devices ( client and server ) aren’t able to handle the high rate of transmission.

In ideal situations for a 10 Mbps line and 9.6 μs of IFG the loss is 14.28%

Now, once you have the ifg and your Ethernet pipe speed. You can easily determine the network efficiency.

Exponential Weighted Moving Average (measured in seconds)

Estimated packet rate is used to identify abnormal activities and attacks. The ethernet adapter estimates the arrival of the next packet based on the information of previous packet. If the packet time is more it will go to sleep (saving power).
Although I want to I cannot talk a lot about EWMA as it is beyond the scope of this article, but on a production system A quick look at the rtt and ewma will tell you if something is wrong. rtt ~ ewma for regular case.

During operations, the effective idletime is measured using an exponential weighted moving average (EWMA), which considers recent packets to be exponentially more important than past ones. The Unix loadaverage is calculated in the same way.
The calculated idle time is subtracted from the EWMA measured one, the resulting number is called ‘avgidle’. A perfectly loaded link has an avgidle of zero: packets arrive exactly at the calculated interval.

Take a look what Rasmus is talking about AJAX

He follows the rule of simplicity as he described in his framework approach.

The very basic of ajax structure -

Javascript

function createRequestObject() {

var ro;

var browser = navigator.appName;

if(browser == “Microsoft Internet Explorer”){

ro = new ActiveXObject(”Microsoft.XMLHTTP”);

}else{

ro = new XMLHttpRequest();

}

return ro;

}

var http = createRequestObject();

function sndReq(action) {

http.open(’get’, ‘rpc.php?action=’+action);

http.onreadystatechange = handleResponse;

http.send(null);

}

function handleResponse() {

if(http.readyState == 4){

var response = http.responseText;

var update = new Array();

if(response.indexOf(’|’ != -1)) {

update = response.split(’|');

document.getElementById(update[0]).innerHTML = update[1];

}

}

}

HTML

<a href=”javascript:sndReq(’foo’)”>[foo]</a>

<div id=”foo” />

server side script(rpc.php) -

switch($_REQUEST['action']) {

case ‘foo’:

/* do something */

echo “foo|foo done”;

break;

…

}

Thats all. Everything else is just building on top of this.

Source : http://news.php.net/php.general/219164

1.Make Fewer HTTP Requests

2.Use a Content Delivery Network

3.Add an Expires Header

4.Gzip Components

5.Put CSS at the Top

6.Move Scripts to the Bottom

7.Avoid CSS Expressions

8.Make JavaScript and CSS External

9.Reduce DNS Lookups

10.Minify JavaScript

11.Avoid Redirects

12.Remove Duplicate Scripts

13.Configure ETags

14.Make Ajax Cacheable

15.Flush the Buffer Early

16.Use GET for AJAX Requests

17.Post-load Components

18.Preload Components

19.Reduce the Number of DOM Elements

20.Split Components Across Domains

22.Minimize the Number of iframes

23.No 404s

24.Reduce Cookie Size

25.Use Cookie-free Domains for Components

26.Minimize DOM Access

27.Develop Smart Event Handlers

28.Choose <link> over @import

29.Avoid Filters

30.Optimize Images

31.Optimize CSS Sprites

32.Don’t Scale Images in HTML

33.Make favicon.ico Small and Cacheable

34.Keep Components under 25K

35.Pack Components into a Multipart Document

Source : Yahoo Developer Newtwrok

Click for detailed info

2. copy php-mode.el to emacs/lisp/

3.byte compile php-mode.el file to get php-mode.elc file

command:  M-x byte-compile-file

4.add following lines to you .emacs file

(require ‘php-mode)

(add-to-list ‘auto-mode-alist ‘(”\\.module$” . php-mode))

(add-to-list ‘auto-mode-alist ‘(”\\.inc$” . php-mode))

(add-to-list ‘auto-mode-alist ‘(”\\.install$” . php-mode))

(add-to-list ‘auto-mode-alist ‘(”\\.engine$” . php-mode))

done! emacs is ready to support php.

try C-c C-f on built in php function

Here is  a list of most used / most discussed / most important  *nix commands. Mastering this will surely get your confidence level high in the black background and white characters’ world !!

My experience says that for starters in UNiX / LiNuX , these commands are enough to get you in smoothly !  

clear Clears the terminal window

usage: [root@test~]$ clear

ls List contents of a directory

usage: [root@test~]$ ls
usage: [root@test~]$ ls  -l

cat Displays the contents of a file in the terminal

usage: [root@test~]$ cat filename.extension

rm Removes a file

usage: [root@test~]$ rm filename.extension
usage: [root@test~]$ rm -i filename.extension  [prompts for confirmation before removing a file]
usage: [root@test~]$ rm -f filename.extension  [force removal of the file regardless of it bieng write-protected or open]
usage: [root@test~]$ rm -r directory		   [remove files recursively in 'directory']

cp Copies a file

usage: [root@test~]$ cp file1.ext file2.ext
usage: [root@test~]$ cp file1.ext /some/other/location/file2.ext
usage: [root@test~]$ cp file1.ext user@some.other.server:~/some/other/location/file2.ext

passwd Changes password

usage: [root@test~]$ passwd
usage: [root@test~]$ passwd SomeNewUser [root access only]

motd Message of the Day

usage: [root@test~]$ motd

finger User information lookup program

usage: [root@test~]$ finger   -p
Login     Name       Tty      Idle  Login Time   Office     Office Phone
xuser              pts/0          Oct 22 22:04 (ool-4578671d.dyn.optonline.net)

startx Starts an X Window System server

usage: [root@test~]$  startx -dpi 300 -depth 32

less Displays the contents of a file in the terminal one page at a time

usage: [root@test~]$ less filename.ext
usage: [root@test~]$ less +G filename.ext [start from end]

info Displays information and documentation on shells, utilities and programs

usage: [root@test~]$ info
usage: [root@test~]$ info less

lpr Sends file to printer

usage: [root@test~]$ lpr filename.ext
usage: [root@test~]$ lpr -# 3 filename.ext [3 copies]

grep looks through files for strings

usage: [root@test~]$ grep "needle" haystack.ext
usage: [root@test~]$ grep -i "neEdlE" haystack.ext
usage: [root@test~]$ grep -i -r "neEdlE" directory/

head Displays first 10 lines of file

usage: [root@test~]$ head filename.ext

tail Displays last 10 lines of file

usage: [root@test~]$ tail filename.log
usage: [root@test~]$ tail -f filename.log [don't exit // realtime]

mv Moves or renames file

usage: [root@test~]$ mv filename.ext /some/other/directory/
usage: [root@test~]$ mv filename.ext newname.ext [rename]

file Displays information about file contents

usage: [root@test~]$ file program.php
program.php: PHP script text
 
usage: [root@test~]$ file archive.tar.gz
archive.tar.gz: gzip compressed data, from Unix

echo Copies string to terminal

usage: [root@test~]$  echo Hello Think-Lamp
usage: [root@test~]$  echo * | wc

date Displays current date and time

usage: [root@test~]$ date
usage: [root@test~]$ date -R

cal Displays calendar

usage: [root@test~]$ cal
usage: [root@test~]$ cal -y 2009

gzip Compresses a file

usage: [root@test~]$ gzip filename.etx > archive.gz
usage: [root@test~]$ gzip -f -q filename.etx  > archive.gz[force and supress errors]
usage: [root@test~]$ gzip -f -q -r directory/ > archive.gz

gunzip Decompresses a compressed file

usage: [root@test~]$ gunzip archive.gz

which Displays path to command

usage: [root@test~]$ which echo
usage: [root@test~]$ which less

whereis Displays paths to locations of commands

usage: [root@test~]$ whereis echo
usage: [root@test~]$ whereis less

who Lists currently logged on users

usage: [root@test~]$ who
usage: [root@test~]$ who -b

w Lists currently logged on users with processing usage

usage: [root@test~]$ w

mesg Sets options for letting other users write you messages

usage: [root@test~]$ mesg

write Sends message to other users

usage: [root@test~]$ write xuser
hello! This is test message
 
usage: [xuser@test~]$
Message from [root@test~]$ on pts/1 at 13:25 ...
hello! This is test message

talk Allows two way chat to other users

usage: [root@test~]$ talk xuser@test
 
usage: [xuser@test~]$
Message from root@test
talk: connection requested by root@test.
talk: respond with: talk xuser@test

chmod Changes file access permissions

usage: [root@test~]$ chmod 775 file.ext
usage: [root@test~]$ chmod -r 777 directory/
 
CHMOD can also to attributed by using Numeric Permissions:
 
400 read by owner
040 read by group
004 read by anybody (other)
200 write by owner
020 write by group
002 write by anybody
100 execute by owner
010 execute by group
001 execute by anybody
____
777  read/write/execute by everybody !

mkdir Makes a directory

usage: [root@test~]$ mkdir newdirectory

rmdir Removes an empty directory

usage: [root@test~]$ rmdir newdirectory

ln Creates link to an existing file [hard, soft]

usage: (hard link) [root@test~]$ ln targetfile.ext linkname.ext
usage: (soft link) [root@test~]$ ln -s targetfile.ext linkname.ext

df Displays all mounted filesystems

usage: [root@test~]$ df
usage: [root@test~]$ df -h

top Displays updating list of currently running processes

usage: [root@test~] top
# Once in  TOP screen, use shift+F to select 'sort by' field.
# Once in Top screen, use Shift+R to 'sort' ascending OR descending

tty Displays the name of the terminal in which the command was issued

usage: [root@test~] tty

kill Aborts a process by PID (Process Identification Number) or job number

usage: [root@test~] kill -p 2345

jobs Displays a list of current jobs

usage: [root@test~] jobs
usage: [root@test~] jobs -l

netstat Displays network connections

usage: [root@test~] netstat -a -e

I noticed that Erik Wendelin wrote an article “awk is a beautiful tool.” In this article he said that it was best to introduce Awk with practical examples. I totally agree with Erik.

Eric Pement’s Awk one-liner collection consists of five sections:

• 1. File spacing,
• 2. Numbering and calculations,
• 3. Text conversion and substitution,
• 4. Selective printing of certain lines,
• 5. Selective deleting of certain lines.

The first part of the article will explain the first two sections: “File spacing” and “Numbering and calculations.” The second part will explain “Text conversion and substitution”, and the last part “Selective printing/deleting of certain lines.”

These one-liners work with all versions of awk, such as nawk (AT&T’s new awk), gawk (GNU’s awk), mawk (Michael Brennan’s awk) and oawk (old awk).

Let’s start!

1. Line Spacing

1. Double-space a file.

awk '1; { print "" }'   filname.ext

So how does it work? A one-liner is an Awk program and every Awk program consists of a sequence of pattern-action statements “pattern { action statements }“. In this case there are two statements “1″ and “{ print “” }”. In a pattern-action statement either the pattern or the action may be missing. If the pattern is missing, the action is applied to every single line of input. A missing action is equivalent to ‘{ print }’. Thus, this one-liner translates to:

awk '1 { print } { print "" }'   filname.ext

An action is applied only if the pattern matches, i.e., pattern is true. Since ‘1′ is always true, this one-liner translates further into two print statements:

awk '{ print } { print "" }'   filname.ext

Every print statement in Awk is silently followed by an ORS – Output Record Separator variable, which is a newline by default. The first print statement with no arguments is equivalent to “print $0″, where $0 is a variable holding the entire line. The second print statement prints nothing, but knowing that each print statement is followed by ORS, it actually prints a newline. So there we have it, each line gets double-spaced.

2. Another way to double-space a file.

awk 'BEGIN { ORS="\n\n" }; 1'   filname.ext

BEGIN is a special kind of pattern which is not tested against the input. It is executed before any input is read. This one-liner double-spaces the file by setting the ORS variable to two newlines. As I mentioned previously, statement “1″ gets translated to “{ print }”, and every print statement gets terminated with the value of ORS variable.

3. Double-space a file so that no more than one blank line appears between lines of text.

awk 'NF { print $0 "\n" }'   filname.ext

The one-liner uses another special variable called NF – Number of Fields. It contains the number of fields the current line was split into. For example, a line “this is a test” splits in four pieces and NF gets set to 4. The empty line “” does not split into any pieces and NF gets set to 0. Using NF as a pattern can effectively filter out empty lines. This one liner says: “If there are any number of fields, print the whole line followed by newline.”

4. Triple-space a file.

awk '1; { print "\n" }'   filname.ext

This one-liner is very similar to previous ones. ‘1′ gets translated into ‘{ print }’ and the resulting Awk program is:

awk '{ print; print "\n" }'   filname.ext

It prints the line, then prints a newline followed by terminating ORS, which is newline by default.

2. Numbering and Calculations

5. Number lines in each file separately.

awk '{ print FNR "\t" $0 }'   filname.ext

This Awk program appends the FNR – File Line Number predefined variable and a tab (\t) before each line. FNR variable contains the current line for each file separately. For example, if this one-liner was called on two files, one containing 10 lines, and the other 12, it would number lines in the first file from 1 to 10, and then resume numbering from one for the second file and number lines in this file from 1 to 12. FNR gets reset from file to file.

6. Number lines for all files together.

awk '{ print NR "\t" $0 }'   filname.ext

This one works the same as #5 except that it uses NR – Line Number variable, which does not get reset from file to file. It counts the input lines seen so far. For example, if it was called on the same two files with 10 and 12 lines, it would number the lines from 1 to 22 (10 + 12).

7. Number lines in a fancy manner.

awk '{ printf("%5d : %s\n", NR, $0) }'   filname.ext

This one-liner uses printf() function to number lines in a custom format. It takes format parameter just like a regular printf() function. Note that ORS does not get appended at the end of printf(), so we have to print the newline (\n) character explicitly. This one right-aligns line numbers, followed by a space and a colon, and the line.

8. Number only non-blank lines in files.

awk 'NF { $0=++a " :" $0 }; { print }'   filname.ext

Awk variables are dynamic; they come into existence when they are first used. This one-liner pre-increments variable ‘a’ each time the line is non-empty, then it appends the value of this variable to the beginning of line and prints it out.

9. Count lines in files (emulates wc -l).

awk 'END { print NR }'   filname.ext

END is another special kind of pattern which is not tested against the input. It is executed when all the input has been exhausted. This one-liner outputs the value of NR special variable after all the input has been consumed. NR contains total number of lines seen (= number of lines in the file).

10. Print the sum of fields in every line.

awk '{ s = 0; for (i = 1; i <= NF; i++) s = s+$i; print s }'   filname.ext

Awk has some features of C language, like the for (;;) { … } loop. This one-liner loops over all fields in a line (there are NF fields in a line), and adds the result in variable ’s’. Then it prints the result out and proceeds to the next line.

11. Print the sum of fields in all lines.

awk '{ for (i = 1; i <= NF; i++) s = s+$i }; END { print s }'   filname.ext

This one-liner is basically the same as #10, except that it prints the sum of all fields. Notice how it did not initialize variable ’s’ to 0. It was not necessary as variables come into existence dynamically.

12. Replace every field by its absolute value.

awk '{ for (i = 1; i <= NF; i++) if ($i < 0) $i = -$i; print }'   filname.ext

This one-liner uses two other features of C language, namely the if (…) { … } statement and omission of curly braces. It loops over all fields in a line and checks if any of the fields is less than 0. If any of the fields is less than 0, then it just negates the field to make it positive. Fields can be addresses indirectly by a variable. For example, i = 5; $i = ‘hello’, sets field number 5 to string ‘hello’.

Here is the same one-liner rewritten with curly braces for clarity. The ‘print’ statement gets executed after all the fields in the line have been replaced by their absolute values.

awk '{
  for (i = 1; i <= NF; i++) {
    if ($i < 0) {
      $i = -$i;
    }
  }
  print
}'   filname.ext

13. Count the total number of fields (words) in a file.

awk '{ total = total + NF }; END { print total }'   filname.ext

This one-liner matches all the lines and keeps adding the number of fields in each line. The number of fields seen so far is kept in a variable named ‘total’. Once the input has been processed, special pattern ‘END { … }’ is executed, which prints the total number of fields.

14. Print the total number of lines containing word “Beth”.

awk '/Beth/ { n++ }; END { print n+0 }'   filname.ext

This one-liner has two pattern-action statements. The first one is ‘/Beth/ { n++ }’. A pattern between two slashes is a regular expression. It matches all lines containing pattern “Beth” (not necessarily the word “Beth”, it could as well be “Bethe” or “theBeth333″). When a line matches, variable ‘n’ gets incremented by one. The second pattern-action statement is ‘END { print n+0 }’. It is executed when the file has been processed. Note the ‘+0′ in ‘print n+0′ statement. It forces ‘0′ to be printed in case there were no matches (’n’ was undefined). Had we not put ‘+0′ there, an empty line would have been printed.

15. Find the line containing the largest (numeric) first field.

awk '$1 > max { max=$1; maxline=$0 }; END { print max, maxline }'   filname.ext

This one-liner keeps track of the largest number in the first field (in variable ‘max’) and the corresponding line (in variable ‘maxline’). Once it has looped over all lines, it prints them out.

16. Print the number of fields in each line, followed by the line.

awk '{ print NF ":" $0 } '   filname.ext

This one-liner just prints out the predefined variable NF – Number of Fields, which contains the number of fields in the line, followed by a colon and the line itself.

17. Print the last field of each line.

awk '{ print $NF }'   filname.ext

Fields in Awk need not be referenced by constants. For example, code like ‘f = 3; print $f’ would print out the 3rd field. This one-liner prints the field with the value of NF. $NF is last field in the line.

18. Print the last field of the last line.

awk '{ field = $NF }; END { print field }'   filname.ext

This one-liner keeps track of the last field in variable ‘field’. Once it has looped all the lines, variable ‘field’ contains the last field of the last line, and it just prints it out.

19. Print every line with more than 4 fields.

awk 'NF > 4'   filname.ext

This one-liner omits the action statement. As I noted in one-liner #1, a missing action statement is equivalent to ‘{ print }’.

20. Print every line where the value of the last field is greater than 4.

awk '$NF > 4'   filname.ext

This one-liner is similar to #17. It references the last field by NF variable. If it’s greater than 4, it prints it out.

Enjoy !!