Linux.com

Simple way is to use awk.

if all of those lines were in a file called test.txt this one line would print out the last column:

awk -F'|' '{print $NF}' test.txt

-F being the filed delimiter (in this case |)
$NF telling it to be the last column (you can make that $2 or any other number to tell it specifically what column you want)

:)
[Modified by: Khabi on May 12, 2008 07:13 PM]

No problem! Glad I could help out :)

Assuming the separator symbol is not part of any of the words, this also works:

cat file.lst | while read line ; do echo "${line##*|}" done

Quote Slarti:

The cut statement above only works if there are exactly three fields per line. I don't know awk to comment on that.

I think you're wrong. It will work fine as long as it's at least three fields. Otherwise it returns an empty string/line (which is still fine).

example:

$ echo '1232|23423|23432|2398746832|3242342|234324' | cut -f3 -d\|
23432
$ echo -e '1232|234324' | cut -f3 -d\|

$

I like the bash pattern substitution one :)

I must confess I love pipes but it's probably just me :)
For some reason I have a feeling that using dedicated binaries (eg. cut) is faster than awk especially if two sides of the pipe can run on a different CPUs. I often have to parse these neverending log files and it matters there. I might be completely wrong though, I mean, they can't run in parallel since one is connected to the other.
You definitely have a point with maintenance problems. These complicated pipes can get almost as bad as they were written in perl :)

Quote Anthony Robbins:

I must confess I love pipes but it's probably just me :)
For some reason I have a feeling that using dedicated binaries (eg. cut) is faster than awk especially if two sides of the pipe can run on a different CPUs. I often have to parse these neverending log files and it matters there. I might be completely wrong though, I mean, they can't run in parallel since one is connected to the other.
You definitely have a point with maintenance problems. These complicated pipes can get almost as bad as they were written in perl :)

You know, that would be an interesting test. If you get a chance next you parse whatever it is your are to try running the time test against it and possibly try it again with awk I would be interested to see what the results are. :)

I've run some tests a couple of times on a 14MB text file consisting of pwgen's output like

Yee2Aith Kaibo9ku oY2Ipe3y iPah9ieg zoo3oNg1 Pa3phae6 Caatho8a Ush5ohTe
thahth6S yeep3Phi ounee5Ei eopeik4S goi1Aiye quo8iZes Aewie0Xu AeZae0ia
phee5Ahw Too1uaR0 taem9ieP ShiY2eek sie8eeZ0 auNg7hie iich0Oom cohQu9Uu
...

# cut -f3,5 -d\ big2.txt > /dev/null
average: 0.13s

# awk '{print $3" "$5}' big2.txt>/dev/null
average: 0.27s (128% of cut|tr)

awk actually does more (it skips multiple field separators while cut does not)
so to achieve the exact same effect the correct command line would be
# tr -s \ < big2.txt | cut -f3,5 -d\ > /dev/null
average: 0.21s (100%)

since this is a semi-SMP system (3ghz p4 HT) I've run the last
two tests (awk vs. tr | cut) on a mono processor computer (2.4ghz p4) on the same file:

# awk '{print $3" "$5}' big2.txt > /dev/null
average: 0.38s (135%)

# tr -s \ < big2.txt | cut -f3,5 -d\ > /dev/null
average: 0.28s (100%)

The bottom line is that cut is way faster, cut | tr is still faster than awk
and it's interesting to see that HT-SMP did not help at all (tr | cut was relatively
slower on HT). All the computers were running the same kernel/os (2.6,debian etch)

edit:
I've managed to get my hands on a 8 CPU system and increased the file size to 10x

# awk '{print $3" "$5}' big3.txt>/dev/null
average: 2.35s (189%)

# tr -s \ < big3.txt | cut -f3,5 -d\ > /dev/null
average: 1.24s (100%)

looks like real SMP helps and filters with pipes in them scale a lot better :)

[Modified by: Anthony Robbins on June 25, 2008 09:16 PM]