| Cheshire Cat Computing http://mail.steveshipway.org/forum/ |
|
| trend.cgi wannabe http://mail.steveshipway.org/forum/viewtopic.php?f=2&t=430 |
Page 1 of 1 |
| Author: | cracraft [ Wed Dec 01, 2004 11:20 am ] |
| Post subject: | trend.cgi wannabe |
Folks, I've been trying to make a standalone version of trend.cgi and am quite close, I believe. But reading about the RRDs:functions on the man page hasn't gotten me to this point. When I run, the error "Could not read in rrd file" comes in. Please have a look at this code and let me know what you think. My goal is to use this to predict the x-axis value for a specific y-axis value and then determine if that x-axis value is in the nearterm and hence be predictive numerically as well as graphically, to drive other tools for the less volatile measurements. The problem below seems to be my lack of understanding of what constitutes the argument to the RRDs last function Thanks, Stuart #!/usr/local/bin/perl -w use strict; use CGI; use FileHandle; use Text::ParseWords; use File::Basename; use RRDs; # Variables our ($rrdfile ) = ''; our( $DECAY ) = 1; our( $device, $community, $targetwindow, $target, $file, $backurl ) = ( "","public","graph","","",""); our( $conffile ) = "/usr/local/apache/htdocs/mrtg/routers2.conf"; # overridden by passed parameters our( $routersurl ) = ''; our( $q ) = new CGI; our( %headeropts ) = (); our( $thishost ) = $q->url(); our( %config ) = (); our( $authuser ) = ''; our( $pathsep ) = '/'; our( %target ) = (); our( $tempfile ) = '/tmp/trend.foo'; our( $ds, $rrddata , $starttime, $endtime, $interval) = ('','','','',''); our( $trendstart, $trenddelta ); # array references our( $dwmy ) = 'y'; our( %interfaces ); our( $workdir ) = ''; our( $monthlylabel ) = '%W'; our( $dailylabel ) = '%H'; ####################################################################### # calculate weighted SD for given start/delta # weighted sd = square root of weighted average of squares of difference sub wsd($$$) { my( $idx, $start, $delta ) = @_; my( $c, $tot, $w, $row, $v ) = (0,0,0,0,0); my( $fx ); $fx = $start; $w = 1; foreach $row ( 0 .. $#$rrddata ) { $v = $rrddata->[$#$rrddata - $row]->[$idx]; if($v) { $tot += ($v-$fx)*($v-$fx) * $w; $c += $w; } $w *= $DECAY; # Next value is less significant $fx -= $delta; } if($c) { $tot = sqrt($tot / $c); } elsif( $tot ) { $tot = $tot * $tot; #XXXXXXX temp } return $tot; } ####################################################################### # This function will retrieve the RRD data, calculate the value and # slope of the trend line, then call the outputxml function and rrdtool # to create XML and the RRDTool database. sub do_trending { my( $rrdfile ) = @_; my( $timeframe ) = "1d"; my( $answer, $status, $e ); my( $n, $idx, $c, $tot, $w, $row, $v, $deltadelta ); my( $sd, $delta, $sda, $sdb, $lim ); print "Starting trending function - rrdfile = $rrdfile\n"; # set intervals (in seconds): 5min, 30min, 2hr, 1day foreach ( $dwmy ) { if( /w/ ) { $timeframe = "200h"; $interval = 1800; last; } if( /m/ ) { $timeframe = "800h"; $interval = 7200; last; } if( /y/ ) { $timeframe = "400d"; $interval = 3600*24; last; } $timeframe = "2000m"; $interval = 300; } # Read in the RRD file $endtime = RRDs::last($rrdfile); $e = RRDs::error; die "Could not read in rrd file\n" if ($e); ( $starttime, $interval, $ds, $rrddata ) = RRDs::fetch($rrdfile,"AVERAGE","-r",$interval, "-s",($endtime-(400*$interval)),"-e",$endtime); $e = RRDs::error; die "Could not fetch data from rrd file\n"; # Calculate trends # this results in (a) last update time, (b) avg value then, # (c) change in value per interval $trendstart = [ 0,0 ]; $trenddelta = [ 0,0 ]; foreach $idx ( 0, 1 ) { # calculate weighted average (start point for line) $w = 1; # weight of current sample $tot = 0; # total of values $c = 0; # total weight $n = 0; # count of samples processed foreach $row ( 0 .. $#$rrddata ) { $v = $rrddata->[$#$rrddata - $row]->[$idx]; if($v) { $tot += $v * $w; $c += $w; $n++; } $w *= $DECAY; # Next value is less significant } $trendstart->[$idx] = $tot / $c if($c); errlog("Weighted average for $idx is: ".$trendstart->[$idx]." = $tot / $c, $n valid samples"); # now calculate best angle of line. # vary delta as weighted standard deviation decreases, until # we dont get a gain by going up or down. $delta = $trenddelta->[$idx]; $deltadelta = $trendstart->[$idx]/100.0; $lim = $deltadelta/1000.0; $sd = wsd($idx,$trendstart->[$idx],$delta); $n = 0; while(($n < 100)and ($deltadelta>$lim)) { # put a cap on iterations $n++; errlog("Delta=$delta, Deviation=$sd : deltadelta=$deltadelta, limit $lim"); $sda = wsd($idx,$trendstart->[$idx],$delta+$deltadelta); $sdb = wsd($idx,$trendstart->[$idx],$delta-$deltadelta); errlog("up->$sda, down->$sdb"); if($sd<$sda and $sd<$sdb) { # we are in a trough $deltadelta /= 2; } elsif( $sda < $sdb ) { $delta+=$deltadelta; $sd = $sda; } else { $delta-=$deltadelta; # chage direction $deltadelta = -($deltadelta/2); $sd = $sdb; } } $trenddelta->[$idx] = $delta; } } $tempfile = "/tmp/trenddata"; $rrdfile = "/usr/local/apache/htdocs/mrtg/aqaba.idc.vzwcorp.com.disk.rrd"; $rrdfile = "aqaba.idc.vzwcorp.com.disk"; $rrdfile = "aqaba.idc.vzwcorp.com.disk.rrd"; do_trending($rrdfile,"$tempfile.rrd"); |
|
| Author: | stevesh [ Wed Dec 01, 2004 1:27 pm ] |
| Post subject: | |
If by 'standalone trend.cgi' you mean one that doesn't need the rrdtool executable and is pure perl, then the version currently shipping with routers.cgi v2.15beta matches this description. It works under windows and unix, too. I'm hoping to be able to add multiple do_trending functions to the trend.cgi so that you can pick which method to use... curently, I've only added a second method of calculating the basepoint for the best-fit line (either use weighted average, or last value) |
|
| Author: | cracraft [ Wed Dec 01, 2004 1:34 pm ] |
| Post subject: | success! |
No problem -- since posting the above, I was able to dig around and construct what I needed. I can pull the time for any given y-value, compare it to time(), see if that is nearterm, and then alarm on it. What I did was take the create_xml and made it a do_prediction function. Instead of creating an xml file it takes an y-value as parameter and returns the x-value at which the y-value has been surpassed. The caller then can compare the returned value with time(), see if it is greater but not by too much, and output its prediction of when the threshold will be reached. The point is to avoid people having to look at graphs too much for prediction if they don't want to. Also, this can link into our paging system or an emailed report system. I.e. the benefits of the trending capability of routers2.cgi can be parlayed to other areas for other benefits. For things that aren't too volatile naturally. Stuart |
|
| Page 1 of 1 | All times are UTC + 12 hours [ DST ] |
| Powered by phpBB® Forum Software © phpBB Group http://www.phpbb.com/ |
|