capturing the output of any program and inserting in db

Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like. The goal is to monitor cron jobs using something other than mail. Dave Cramer

Something like this? 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ >> /tmp/backup.log Mauro http://mauro.limeiratem.com - registered Linux User: 294521 Scripture is both history, and a love letter from God. 2014-10-29 10:40 GMT-02:00 Dave Cramer <davecramer@gmail.com>:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
The goal is to monitor cron jobs using something other than mail.
Dave Cramer
--- GTALUG Talk Mailing List - talk@gtalug.org http://gtalug.org/mailman/listinfo/talk

On Wed, Oct 29, 2014 at 8:40 AM, Dave Cramer <davecramer@gmail.com> wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
The goal is to monitor cron jobs using something other than mail.
Is the DB requirement necessary? I think Mauro's suggestion is fine if you don't need to put it into a DB as you can just have all the output redirected into a logging file. Just to improve on his suggestion though: 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ >> /tmp/backup.log 2>&1 That last little bit says to redirect the STDERR to STDOUT so you'll also get error messages logged. Otherwise you'll also get an email if there's anything on the STDERR. -Tim

Dave Cramer wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
You could use the sqlite. Running something like: #!/bin/sh sqlite3 log.db 'INSERT INFO log (message) VALUES ($MESSAGE);' in a table setup like this: CREATE TABLE log ( id INTEGER PRIMARY KEY, timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, message TEXT );

Myles, that is pretty much all I need, thx. Others, syslog is fine, but I want an easy way to view the results other than grep Dave Dave Cramer On 29 October 2014 12:13, Myles Braithwaite <me@mylesbraithwaite.com> wrote:
Dave Cramer wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
You could use the sqlite. Running something like:
#!/bin/sh sqlite3 log.db 'INSERT INFO log (message) VALUES ($MESSAGE);'
in a table setup like this:
CREATE TABLE log ( id INTEGER PRIMARY KEY, timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, message TEXT );
--- GTALUG Talk Mailing List - talk@gtalug.org http://gtalug.org/mailman/listinfo/talk

On 2014-10-29 12:13 PM, Myles Braithwaite wrote:
Dave Cramer wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
You could use the sqlite. Running something like:
#!/bin/sh sqlite3 log.db 'INSERT INFO log (message) VALUES ($MESSAGE);'
in a table setup like this:
CREATE TABLE log ( id INTEGER PRIMARY KEY, timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, message TEXT );
What if $MESSAGE == 'foo); DELETE FROM log;' Just saying.. :p

Dave Cramer On 29 October 2014 13:18, Jamon Camisso <jamon.camisso@utoronto.ca> wrote:
On 2014-10-29 12:13 PM, Myles Braithwaite wrote:
Dave Cramer wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
You could use the sqlite. Running something like:
#!/bin/sh sqlite3 log.db 'INSERT INFO log (message) VALUES ($MESSAGE);'
in a table setup like this:
CREATE TABLE log ( id INTEGER PRIMARY KEY, timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, message TEXT );
What if $MESSAGE == 'foo); DELETE FROM log;'
Just saying.. :p
good point, this is an internal tool, so probably not a real issue.
--- GTALUG Talk Mailing List - talk@gtalug.org http://gtalug.org/mailman/listinfo/talk

On 2014-10-29 1:37 PM, Myles Braithwaite wrote:
Jamon Camisso wrote:
What if $MESSAGE == 'foo); DELETE FROM log;'
Fixed:
#!/bin/bash CLEAN_MESSAGE="${MESSAGE//\;/}" sqlite3 log.db "INSERT INTO log (message) VALUES ('$CLEAN_MESSAGE');"
Better, but what about: MESSAGE="foo\u003B DELETE FROM log\u003B"

On 2014-10-29 1:50 PM, Myles Braithwaite wrote:
Jamon Camisso wrote:
Better, but what about:
MESSAGE="foo\u003B DELETE FROM log\u003B"
Does bash support unicode characters now? Cause it seems to be working fine:
3|2014-10-29 17:44:41|foo\u003B DELETE FROM log\u003B
Is there a Python like `u` for bash?
Not sure: if I echo $CLEAN_MESSAGE though I see the ; with \u003B or \x3B characters. When you run the insert in sqlite does it interpret the characters as literal strings again?

On 29 October 2014 12:13, Myles Braithwaite <me@mylesbraithwaite.com> wrote:
Dave Cramer wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
You could use the sqlite. Running something like:
#!/bin/sh sqlite3 log.db 'INSERT INFO log (message) VALUES ($MESSAGE);'
in a table setup like this:
CREATE TABLE log ( id INTEGER PRIMARY KEY, timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, message TEXT );
I'm finding I really like using command line tooling to write to syslog... The command "logger" <http://linux.die.net/man/1/logger> writes entries to syslog I populate shell scripts with wee "glog" functions to wrap this... glog () { local level=$1 local notice=$2 logger -i -p "${level}" -t "name-of-my-script.sh" "${notice}" echo "${level} ${notice}" } I then inject glog user.notice "Ooh, I did something!" and glog user.error "oops, something went wrong - rc=$?" and such. If I *really* wanted to, I could configure syslog to write some messages to a SQL database. <http://www.rsyslog.com/doc/rsyslog_pgsql.html> I did that for a while; found it to be more a pain than a help. (IPv6 issues blew up the logs on me, and I couldn't query the DB as fast as disk space ballooned away...) I'm finding that I [Heart] logger as a way of getting scripty apps to log activity. The OTHER thing I'd like is to have something materially better than cron, unfortunately, that's a surprisingly expensive problem. Improving on cron looks, historically, like it leads to the sort of all-encompassing framework expansion that is where (eek!) we're seeing controversies about SystemD. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?"

If you're putting event data in databases, reduce your usual logs first with antilog, so the borring regular stuff doesn't appear. Conversely, if certain things should appear, check for them and log an "expected event missing" entry. Think of the DB as a history of exceptions in a"management by exceptions" scheme. --dave [See also antilog, http://datacenterworks.com/stories/antilog.html and http://sourceforge.net/projects/antilog/] On 10/29/2014 08:40 AM, Dave Cramer wrote:
Is there anything out there that is written simply ? By this I mean I don't want to have to install a framework or a jvm or any large dependencies. Ideally this would be written in bash or the like.
The goal is to monitor cron jobs using something other than mail.
Dave Cramer
--- GTALUG Talk Mailing List - talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
-- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain
participants (8)
-
Christopher Browne
-
Dave Cramer
-
David Collier-Brown
-
Jamon Camisso
-
Mauro Souza
-
Myles Braithwaite
-
Stewart Russell
-
Tim Tisdall