Living rural provides certain challenges, especially if you rely on your internet connection. I do a lot of work from home, so it’s crucial that my Internet is running a peak performance. With all the rain we have had in our area, I am having to constantly check the DSL speeds, that means logging into the uVerse Modem, going into the Broadband tab where I get the current status.Now, I’m sure I’ve seen those historical data logs on this deficient the form of a long text table before. But finding it again becomes the issue. Forget it, that’s just too much of a hassle. Since the device given to me with my DSL service is so neutered, typical monitoring applications will not work here, the modem does not answer SNMP queries. The solution, scrub the information from the HTML provided by the NVG510 configuration Broadband / Status tabs, and present it in a graph.
The tools:
- uVerse Motorola NVG510 DSL Modem ( Software Version 9.0.6h2d45, although any version may do )
- Raspberry Pi 2
- Your favorite ARM based Linux Distro ( I'm using Retropie 3.0 / Debian Jesse release )
- MRTG/RRDTool (Installed both to ensure I got all of the perl libraries that I might need)
- Apache2
- CPAN & GD::Graph
- And a handful of scripts.
So, load up your Pi with any of the Linux Distros (I recommend the new RetroPi 3.0), The O/S you select should have a fairly recent of version of perl. ( perl 5, version 20 ). You will need to get access to the root account (may be as simple as “su passwd root” while logged in as pi), and enable the ssh server (see your specific OS Distro for these instructions). Using the package manager apt or yum (depending on your platform) “install libgd2, mrtg, rrdtool, apache2“. You will also need the Perl GD module, if it is not installed with the previous packages, you can always use “cpan install GD::Graph“.
The first hurdle to overcome was to strip out only the bits I needed, in this case kilobits, Yeah, I said kilobits, our connection is 1.5m down and 380k up. I used curl to pull down the status page HTML code, and store it in /tmp.
# Get latest connection stats from uVerse Motorola NVG510 DSL Modem
curl –silent “http://192.168.1.254/cgi-bin/dslstatistics.ha” > /tmp/dsl_speed_raw.htlm
I then used sed to grab just the HTML lines that I needed to parse the data again with sed in order to trim down the line HTML code to the piece of data I am looking for.
# Strip out the Downstream and Upstream speeds
sed -n 193p /tmp/dsl_speed_raw.htlm > /tmp/dsl_speed.dat
Down=$(sed -e ‘s/<tr>//’ -e ‘s/<td class=”col2″>//’ -e ‘s/^[ \t]*//’ -e ‘s/[ \t]*$//’ -e ‘s/<\/td>//’ /tmp/dsl_speed.dat)
sed -n 196p /tmp/dsl_speed_raw.htlm > /tmp/dsl_speed.dat
Up=$(sed -e ‘s/<tr>//’ -e ‘s/<td class=”col2″>//’ -e ‘s/^[ \t]*//’ -e ‘s/[ \t]*$//’ -e ‘s/<\/td>//’ /tmp/dsl_speed.dat)
Now that I trimmed everything down to just the values of the Downstream and Upstream speed, and assigned them to variables, I now have some useful data that I can use to generate a graph.
To create our graph, I used the RRDTool. The RRDTool (Round Robin Database Tool) creates a database, but not just any database. This database has the ability to cycle data through it, and age out stale data that eventually gets flushed out of the database. For more information see this site: RRDTool.org. For this to work, RRDTool uses a special timestamp, that we have to add to our script. The timestamp is the seconds since epoch (January 1, 1970, also known as POSIX time). So next I have to create some variables based on time.
# Create the time variables
ssepoch=$(date ‘+%s’)
defstep=’300′
COUNT=`expr $ssepoch – $defstep`
I set three variables here, ssepoch (seconds since January 1, 1970), defstep (300 seconds or 5 minutes) and then do a little subtraction of 300 second from epoch (used if we need to crate a new RRD file). In an RRD database, the default behavior is that is expects to be updated every five minutes (configurable).
We now have everything we need to create our database that we can generate graphs from. The script will create the database we need to graph our two variable, epoch time, downstream speed, and upstream speed using the rrdtool create command. However, if the file does exist, it will begin to add these values to our database using the rrdtool update command.
[ -f /var/www/mrtg/dslspeed.rrd ] && rrdtool update /var/www/mrtg/dslspeed.rrd ${ssepoch}:${Down}:${Up} || rrdtool create /var/www/mrtg/dslspeed.rrd –start ${COUNT} DS:Down:GAUGE:600:U:U DS:Up:GAUGE:600:U:U RRA:AVERAGE:0.5:1:600 RRA:AVERAGE:0.5:6:700 RRA:AVERAGE:0.5:24:775 RRA:AVERAGE:0.5:288:797 RRA:MAX:0.5:1:600 RRA:MAX:0.5:6:700 RRA:MAX:0.5:24:775 RRA:MAX:0.5:288:797
More information on this is available on Tobias Oetiker’s RRDTool page referenced earlier.
Now onto the creating the graphs. The rrdtool graph command handles this:
rrdtool graph /var/www/html/graphs/dslspeed_day.png -a PNG –width 600 –height 200 –title=”AT&T DSL Speeds” –color FONT#FFFFFF –color GRID#FFFF00 –color CANVAS#000000 –color BACK#000000 –vertical-label “(kbps)” –start -86400 DEF:down=/var/www/mrtg/dslspeed.rrd:Down:AVERAGE DEF:up=/var/www/mrtg/dslspeed.rrd:Up:AVERAGE CDEF:adjdown=down,1000,\* CDEF:adjup=up,1000,\* ‘AREA:adjdown#CC0000:Down Speed’ ‘AREA:adjup#FF9900:Up Speed’
rrdtool graph /var/www/html/graphs/dslspeed_week.png -a PNG –width 600 –height 200 –title=”AT&T DSL Speeds” –color FONT#FFFFFF –color GRID#FFFF00 –color CANVAS#000000 –color BACK#000000 –vertical-label “(kbps)” –start -604800 DEF:down=/var/www/mrtg/dslspeed.rrd:Down:AVERAGE DEF:up=/var/www/mrtg/dslspeed.rrd:Up:AVERAGE CDEF:adjdown=down,1000,\* CDEF:adjup=up,1000,\* ‘AREA:adjdown#CC0000:Down Speed’ ‘AREA:adjup#FF9900:Up Speed’
rrdtool graph /var/www/html/graphs/dslspeed_month.png -a PNG –width 600 –height 200 –title=”AT&T DSL Speeds” –color FONT#FFFFFF –color GRID#FFFF00 –color CANVAS#000000 –color BACK#000000 –vertical-label “(kbps)” –start -2592000 DEF:down=/var/www/mrtg/dslspeed.rrd:Down:AVERAGE DEF:up=/var/www/mrtg/dslspeed.rrd:Up:AVERAGE CDEF:adjdown=down,1000,\* CDEF:adjup=up,1000,\* ‘AREA:adjdown#CC0000:Down Speed’ ‘AREA:adjup#FF9900:Up Speed’
We now have the code to create three different graphs from our single data base that will track our DSL connection for 24 hours, 1 week, and one Month (30 Days). This is achieved by setting the –start flag in the graph configuration. Since there are 86,100 seconds in a day, this becomes a problem of simple arithmetic. The –start flag will tell the rrdtool to only use data collected for the graph between the current epoch time minus the –start flag variable and the current epoch time. In other words, the last 86,100 seconds (24 hours).
Now we have everything set up, we need to have this script run every five minutes. Sounds like a cron job to me. I could just put a sleep command in the script, and the have the script call itself. But that’s sloppy, and could causes issues if the process decides to “runaway”. So edit /etc/crontab and add this line:
*/5 * * * * root /root/scripts/get-dsl.sh
And that’s it, you are now collecting the data, and the graphs. But wait, now how are you going to look at them. Well, we installed the Apache web server earlier. Go to the doc root (usually /var/www/html directory) and create an index.html file and code in the <img> tag pointing at the graph that was created and you’re done. It takes several iterations of the script to run before you will see any meaningful data on the graph, so be patient. After an hour or so, you will be able to pull up graphs like these.
So now when the Raspberry Pi 2 is sitting idle, or when I have the time to actually play games on the RetroPie game console, it is still doing something useful. Hint, I used the metatag on my HTML page to refresh the page every 5 minutes.
<meta http-equiv=”refresh” content=”300″>
Putting it all together: if you are interested in trying this yourself, here is the whole script.
[snippet slug=get-dsl-sh lang=bsh]