Share this post

    

Do you have a stock exchange portfolio with shares that you need to monitor on daily basis? With command line and your shell nothing can be easier. In this article we will show you how to download all NYSE stock exchange ticker symbols in matter of minutes using few lines of bash script.
With some light modification this method can be applied to any stock exchange and any number of ticker symbols. This script can also be altered to behave as a monitoring tool and act upon your predefined instructions if combined with a website which displays intraday data. If you still have some questions after reading this article please try our new LinuxCareer Forum.

1. What you will need

Do not worry, there is no need to have an account with bloomberg.com! All you need at this point are ticker symbols of all shares you wish to monitor/download. This can be easily obtained freely from the internet.

2. NYSE stock exchange download script

#!/bin/bash

temp=`mktemp`
max=40

for ARG in $( cat $1); do   
    wget -qO- http://www.eoddata.com/stockquote/nyse/$ARG.htm | grep\
    "size:26px" | sed -n '/^$/!{s/]*>//g;p;}' | cut -d\; -f2 |cut \
    -d C -f1 | xargs echo $ARG: >> $temp &

    NPROC=$(($NPROC+1))
    if [ "$NPROC" -ge $max ]; then
        wait
        NPROC=0
    fi

done

wait
sort $temp > nyse.txt

3. Executing this script

Make the script executable and run it with a single argument and that is a file which contains all share ticker symbol you wish to download.

$ ls
NYSE.sh  tickers.txt
$ chmod +x NYSE.sh 
$ cat tickers.txt 
NCZ
CASC
DHI
HIS
AAR
$ ./NYSE.sh tickers.txt 
$ cat nyse.txt 
AAR: 21.66
CASC: 34.23
DHI: 9.780
HIS: 2.010
NCZ: 8.150

4. How this script works

This particular script is based on the information found at eoddata.com website. For each "for loop" iteration wget will fetch data a print it to STDOUT. In the next sequence of commands the script will extract shares prices for given ticker symbol by stripping out the wget's output of html tags and employing grep and cut commands to extract and format output data.

The max variable defines number of parallel commands used to fetch data. This vastly speeds up the whole process as we call 40 instances of wget at any given time. NPROC variable controls number of running processes and this number is checked with every iteration.

Since we cannot predict which wget's html request finishes first, we need to wait until last process is done and then we alphabetically sort retrieved data.

5. How this script can be improved

It needs to be said that the above script relies on a good will of eoddata.com. However, if you are unable to download data from eoddata.com simply modify your script to fetch data from any other website as stock exchange data are publicly available on Internet, so only what you need is some smart way to grab them.

Having said the above, this script is therefore highly unreliable. One can add more websites to the script to fetch data from, if first site fails. Another improvement could be to notify a user if any data retrieval fails.

Script can also be modified to continuously monitor stock exchange and notify user on any predefined events if combined with a website which displays intraday data .

6. Fetch 3234 NYSE tickers in 11 minutes

The script generally works well and we were able to download 3234 share prices in about 11 minutes as demonstrated below. We also used grep to search all lines which do not include "." as this are lines where the script failed to download data. As it turned out the script downloaded all 3234 NYSE tickers without fail. We also used tail to print sample data.

$ ls
NYSE.sh  tickers.txt
$ wc -l tickers.txt 
3234 tickers.txt
$ time ./NYSE.sh tickers.txt 

real    11m4.623s
user    0m53.991s
sys     0m29.098s
$ grep -v \. nyse.txt 
$ nl nyse.txt | tail -20 
  3215  YOKU: 17.41
  3216  YPF: 35.41
  3217  YSI: 10.08
  3218  YUM: 51.66
  3219  YZC: 22.55
  3220  ZA: 2.550
  3221  ZB-A: 16.53
  3222  ZB-B: 24.93
  3223  ZB-C: 25.35
  3224  ZB-E: 25.83
  3225  ZEP: 13.92
  3226  ZF: 2.930
  3227  ZLC: 3.060
  3228  ZMH: 53.52
  3229  ZNH: 24.69
  3230  ZQK: 3.350
  3231  ZTR: 3.030
  3232  ZX: 4.250
  3233  ZZ: 1.590
  3234  ZZC: 51.84

Can you browse this fast? This is yet another example of Linux command line power. Good alternative to this script could be a perl module: libfinance-quote-perl which is covered here.