Instruktaż/Tutorial Przekazywanie pełnych logów serwera na Gadu-Gadu

Silas Mariusz

rm -rf /
Help us, GOD!
5 Kwiecień 2008
9 926
38
3 445
153
40
Nowy Sącz
forum.qnap.net.pl
QNAP
TS-x77
Ethernet
1 GbE
(Uwaga! Program nie jest jeszcze ukończony!)


Przekazywanie w czasie rzeczywistym
monitów oraz komunikatów
z serwera QNAP NAS na Gadu-Gadu


Dla wszystkich urządzeń QNAP NAS
arm-x09 TS-109/209
arm-x19 TS-x12/x19/x20/x21/+
x86 TS-509/x39/x59/x69/x79/+


- przewodnik instalacji -
2013, Silas Mariusz, rithien

[Przykład]
Kod:
12:09:13 QNAP Logs Notifier
(WARN) (!) The system was not shut down properly last time.
(INFO) System started.

12:12:42 QNAP Logs Notifier
[SAMBA] admin/10.100.200.100 (silas-ibm-pc)
Logged in
Read: Video/MOViES/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS.nfo
Read: Video/MOViES/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS.mkv
Read: Video/MOViES/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS.srt

12:15:37 QNAP Logs Notifier
[SAMBA] admin/10.100.200.100 (silas-ibm-pc)
Read: Video/MOViES/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity.avi
Read: Video/MOViES/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity.nfo

12:17:51 QNAP Logs Notifier
[SAMBA] admin/10.100.200.100 (silas-ibm-pc)
Delete: Video/MOViES/Rejs.1970.PL.720p.HDTV.X264.AC3-TVM4iN/Rejs.1970.PL.720p.HDTV.X264.AC3-TVM4iN.mkv
Delete: Video/MOViES/Rejs.1970.PL.720p.HDTV.X264.AC3-TVM4iN

12:39:08 QNAP Logs Notifier
(WARN) [SAMBA] User2/193.93.122.60 (basesrv-old) (!) Login Fail

13:02:11 QNAP Logs Notifier
(WARN) admin/10.100.200.100 (!) [TCP/IP] Changed configuration of network interfaces [Trunking Group 1] from [balance-rr] to [balance-alb]
(INFO) Network connection resume.

13:05:47 QNAP Logs Notifier
(INFO) [ rtorrent:control ] Timezone Europe/Belgrade
(INFO) [ rtorrent:control ] Port 6009 (Web) is available.
(INFO) [ rtorrent:control ] Port 6008 (SecureWeb) is available.
(INFO) [ rtorrent:control ] Port 5000 (SCGI) is available.
(INFO) [ rtorrent:control ] Port 6881 (DHT) is available.
(INFO) [ rtorrent:control ] Port 42000 (Torrent) is available.
(INFO) [ rtorrent:control ] Redirecting ports (main): 5000, 6881, 42000
(INFO) [ rtorrent:control ] Iface IP address... 10.100.200.10
(INFO) [ rtorrent:control ] IP address... 8.8.8.8
(INFO) [ rtorrent:control ] Server hostname qcloudname
(INFO) [ rtorrent:control ] Server domain myqnapcloud.com
(INFO) [ rtorrent:control ] User mail address: qcloudname@myqnapcloud.com
(INFO) [ rtorrent:control ] Web server certificate found.
(INFO) [ rtorrent x: ] cmd: chmod 666 /dev/null...
(INFO) [ rtorrent x: ] cmd: chmod 666 /dev/ptmx...
(INFO) [ rtorrent:control ] screen-cleanup started.
(INFO) [ rtorrent:control ] rtorrent started.
(INFO) [ rtorrent:control ] lighttpd started.
(INFO) [ rtorrent:control ] Boot sequence done!


[Notka]
Przed przystąpieniem do instalacji można skorzystać z How-To ułatwiającego prace w Linuxie: QNAP od środka, czyli grzebiemy w linuchu - Jak sobie ułatwić?


[Instalacja]
  1. W swoim komunikatorze Gadu-Gadu, dodaj nowy kontakt numer 48280073 (qLog-IM).
  2. Będąc dostępnym, napisz do bota wiadomość o treści "ZAPISZ".
  3. Następnie za pomocą klienta SSH (np. PuTTY) zaloguj się na serwer i wykonaj poniższe:
    Bash:
    # Znajdz katalog w ktorym przechowywane sa paczki qpkg
    publicdir=`/sbin/getcfg Public path -f /etc/config/smb.conf`
    qpkgdir=`cd $publicdir && cd ../.qpkg/ && pwd`
    
    # Utwórz katalog w którym będzie znajdował się watcher
    mkdir -p $qpkgdir/qlogWatcher
    cd $qpkgdir/qlogWatcher
    
    # Pobierz program i rozpakuj 
    wget http://pool.qnapclub.pl/site/packages/qlogWatcher/qlogWatcher.tar.gz
    tar zxvf qlogWatcher.tar.gz
    rm -f qlogWatcher.tar.gz
  4. Otwórz w celu edycji plik z ustawieniami config.ini. Możesz do tego celu użyć WinSCP lub edytora w konsoli.
    Bash:
    mcedit config.ini        # edytor Midnight Commander
    vi config.ini            # klasyczny edytor viii
  5. W kluczu ARGS znajdującym się w sekcji Delegate otwartego pliku, podaj swój numer Gadu-Gadu poprzedzając prefixem number=gg.
    INI:
    [Delegate]
    ARGS=example=test&number=nr7662900
    URL=http://pool.qnapclub.pl/site/tools/utils/im.gadugadu/bot.php
    
    [General]
    numLast_event=20
    numLast_conn=0
    INTERVAL=1
    Zapisz zmodyfikowany plik z ustawieniami.
    (Aby móc pisać w edytorze vi, należy na klawiaturze nacisnąć przycisk 'i', natomiast aby zapisać plik i wyjść z edytora naciśnij kolejno: ESC, W, Q i potwierdź Enterem.)
  6. Teraz należy dodać plik qlogWatcher.sh do autostartu, aby program uruchamiał się przy każdym uruchomieniu serwera. W tym celu można posłużyć się najprostszą metodą wszczepienia się w inny skrypt startowy lub zapisać program jako paczka QPKG.
    Bash:
    # Skopiuj domyślne ikony
    # ... tu uzupelnic
    
    # Podlinkuj skrypty
    ln -sf $qpkgdir/qlogWatcher/qlogWatcher.sh /etc/init.d/qlogWatcher.sh
    ln -sf $qpkgdir/qlogWatcher/qlogWatcher.sh /etc/rcS.d/QS101qlogWatcher
    ln -sf $qpkgdir/qlogWatcher/qlogWatcher.sh /etc/rcK.d/QK101qlogWatcher
    
    # Dodaj jako program QPKG
    setcfg qlogWatcher Name qlogWatcher -f /etc/config/qpkg.conf -c
    setcfg qlogWatcher Version 1.0 -f /etc/config/qpkg.conf -c
    setcfg qlogWatcher Author "Silas Mariusz, rithien" -f /etc/config/qpkg.conf -c
    setcfg qlogWatcher QPKG_File qlogWatcher.qpkg -f /etc/config/qpkg.conf -c
    setcfg qlogWatcher Date "2013-08-11" -f /etc/config/qpkg.conf -c
    setcfg qlogWatcher Shell "${qpkgdir}/qlogWatcher/qlogWatcher.sh" -f /etc/config/qpkg.conf -c
  7. Przygotowaną paczkę uruchom:
    Bash:
    /etc/init.d/qlogWatcher.sh enable
    /etc/init.d/qlogWatcher.sh start
    To wszystko! Teraz będziesz otrzymywał logi serwera w czasie rzeczywistym na określony przez Ciebie numer Gadu-Gadu.


[Kod źródłowy]
qlogWatcher.sh:
Bash:
#!/usr/bin/env sh
##
##   QNAP Log Watcher
##   part of QNAP qLog-IM System Log Messages Parser Tool
##
##   Copyright © 2013, Silas Mariusz <silas@qnap.com.antispam>
##             © 2013, rithien <magik@post.pl.antispam>
##
##                     QNAP Club Polska
##
##   This program is free software; you can redistribute it and/or modify
##   it under the terms of the GNU General Public License as published by
##   the Free Software Foundation; either version 2, or (at your option)
##   any later version.
##
##   This program is distributed in the hope that it will be useful,
##   but WITHOUT ANY WARRANTY; without even the implied warranty of
##   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
##   GNU General Public License for more details.
##
##   You should have received a copy of the GNU General Public License
##   along with this program; if not, write to the Free Software Foundation,
##   Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
##
##   The latest version of this software can be obtained here:
##
##   https://forum.qnap.net.pl/
##
##
##   FILE: qlogWatcher.sh
##
##   RELEASE: 1.0
##   DATE: 2013-08-11


_scriptname="qLog-watcher"


## Set Basics
## ###########################################################################
[ "x$DEBUG" != "x" ] && echo "Hello kitty" > /tmp/logParser.$$
: ${AWK=awk}
export PATH="$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin:/opt/bin:/opt/sbin"


## find my name and real current directory
## ###########################################################################
PROGNAME=`type $0 | awk '{print $3}'`  # search for executable on path

PROGDIR=`dirname $PROGNAME`            # extract directory of program
PROGNAME=`basename $PROGNAME`          # base name of program

[ "$PROGDIR" == "." ] && PROGDIR=`cd $PROGDIR && pwd || echo $PROGDIR`

# Fully qualify directory path (remove relative components and symlinks)
# WARNING: The "bash" "pwd" builtin does not resolve symbolic links
# "sh" (as a link to "bash") also does not,   but "csh" does handle it.
# Symbolic link removal however is not critical.
ORIGDIR=`pwd`                                 # original directory (builtin)
PROGDIR=`cd $PROGDIR && pwd || echo $PROGDIR` # program directory

# Results...
#    $ORIGDIR    -- where the users was when called
#    $PROGDIR    -- script directory location (and now current directory)
#    $PROGNAME   -- This scripts executable name


## ###########################################################################
## get Web share real name
WebShare=`/sbin/getcfg SHARE_DEF defWeb -d Qweb -f /etc/config/def_share.info`
[ $WebShare = "Qweb" ] && QMOUNTS=TRUE || QMOUNTS=FALSE


## ###########################################################################
## Read config
numLast_event=$(/sbin/getcfg General numLast_event -f "$PROGDIR/config.ini" -d 10)
numLast_conn=$(/sbin/getcfg General numLast_conn -f "$PROGDIR/config.ini" -d 0)

## INTERVAL between next read log cycle (in seconds)
INTERVAL=$(/sbin/getcfg General INTERVAL -f "$PROGDIR/config.ini" -d 60)

## Delegate Messages...
URL=$(/sbin/getcfg Delegate URL -f "$PROGDIR/config.ini" -d unknown)
if [ "$URL" == "unknown" ]; then
    echo " [!] Delegate not configured. Check config.ini..."
    exit 1
fi
ARGS=$(/sbin/getcfg Delegate ARGS -f "$PROGDIR/config.ini" -d unknown)

## Date expression for yesterday
DATE="-1 day"

## Locale for printf number formatting (e.g. "10000" => "10,000")
LANG=en_US

# Settings
agent="Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0"
timeout=30
retries=1


## ###########################################################################
## log DB file (used to watch log changes - file size diff)
logDB_event="/etc/logs/event.log"
logDB_conn="/etc/logs/conn.log"


## ###########################################################################
## wget with retries
function wget
{
    local i
    for ((i=0; i<3; i++)); do
        if command wget -O- --quiet --no-check-certificate -T $timeout -t $retries "$@"; then
            return 0
        fi
    done
    return 1
}


## ###########################################################################
## UrlEncode function
EncodeEOL=yes
LANG=C  export LANG
function UrlEncode() {
echo "$@" | $AWK '
    BEGIN {
        # We assume an awk implementation that is just plain dumb.
        # We will convert an character to its ASCII value with the
        # table ord[], and produce two-digit hexadecimal output
        # without the printf("%02X") feature.

        EOL = "%0A"             # "end of line" string (encoded)
        split ("1 2 3 4 5 6 7 8 9 A B C D E F", hextab, " ")
        hextab [0] = 0
        for ( i=1; i<=255; ++i ) ord [ sprintf ("%c", i) "" ] = i + 0
        if ("'"$EncodeEOL"'" == "yes") EncodeEOL = 1; else EncodeEOL = 0
    }
    {
        encoded = ""
        for ( i=1; i<=length ($0); ++i ) {
            c = substr ($0, i, 1)
            if ( c ~ /[a-zA-Z0-9.-]/ ) {
                encoded = encoded c             # safe character
            } else if ( c == " " ) {
                encoded = encoded "+"   # special handling
            } else {
                # unsafe character, encode it as a two-digit hex-number
                lo = ord [c] % 16
                hi = int (ord [c] / 16);
                encoded = encoded "%" hextab [hi] hextab [lo]
            }
        }
        if ( EncodeEOL ) {
            printf ("%s", encoded EOL)
        } else {
            print encoded
        }
    }
    END {
        #if ( EncodeEOL ) print ""
    }
'
}


## ###########################################################################
## Send log container
sendLog() {
    local _message=$(UrlEncode $@)
    wget "${URL}?${ARGS}&message=$_message"
}

err() {
    echo -e "$@"
    sendLog "$@"
    exit 1
}

## ###########################################################################
## Returns true if $1 is Integer
## usage: is_int ${var} && echo "that's an int"
isInt() {
    if test ${1} -eq ${1} 2>/dev/null; then
        return 0
    fi
    return 1
}

## ###########################################################################
## Returns true if $1 is one of $2..$n
arrayIsPresent() {
    local val="$1"
    shift
    for v in "$@"; do
        [ "$v" = "$val" ] && return 0
    done
    return 1
}

## ###########################################################################
# make sure log database is present
[ ! -f "$logDB_conn" ] && \
    err " Log Database file not exist $logDB_conn ... \n\n Please reinitialize database: \n    #  /sbin/conn_log_tool -c \n\n"
 
[ ! -f "$logDB_event" ] && \
    err " Log Database file not exist $logDB_event ... \n\n Please reinitialize database: \n    #  /sbin/log_tool -c \n\n"

## ###########################################################################
## clean output
cleanLog () {
    while read data
    do
        echo "$data" | grep -v "Query database" | \
                grep -v "Done. Return value is" | \
                grep -v "conn_id,conn_type,conn_date,conn_time,conn_user,conn_ip,conn_comp,conn_res,conn_serv,conn_action"
    done
}

parsedConnLine=
parsedConnMsg=
## ###########################################################################
## parseLogConn
parseConnLog () {
    parsedConnLine=
    local _parsed= ; local _tmp=
    local i=0 ; local j=0
    local inputLine=$@
    local _trash=$(echo -e $inputLine | cut -d\, -f8-)      # <trash>
    local _temp_1=$(echo -e $inputLine | cut -d\, -f-7)     # first part
    local _temp_2=$(echo -e $_trash | cut -d\, -f2-)        # last part
    parsedConnMsg==$(echo -e $_trash | tr -d "$_temp_2")        # log msg

    i=0 ; j=0
    for (( i = 0; i < 7; i++ )); do
        j=$(expr $i + 1)
        _tmp=$(echo $_temp_1 | cut -d\, -f${j})

        if [ "$j" == "2" ]; then
            if isInt $j ; then
                case $j in
                    0)  _tmp="INFO" ;;
                    1)  _tmp="WARN" ;;
                    2)  _tmp="ERR!" ;;
                    *)  _tmp="UNKNOWN" ;;
                esac
            else
                _tmp="UNKNOWN"
            fi
        fi

        _parsed[$i]=$_tmp
        #echo ${_parsed[$i]}
    done

    i=0 ; j=0
    for (( i = 7; i < 9; i++ )); do
        j=$(expr $i - 6)
        _tmp=$(echo $_temp_2 | cut -d\, -f${j})

        if [ "$j" == "1" ]; then
            if isInt $j ; then
                case $j in
                    1)  _tmp="SAMBA" ;;
                    2)  _tmp="FTP" ;;
                    3)  _tmp="HTTP" ;;
                    4)  _tmp="NFS" ;;
                    5)  _tmp="AFP" ;;
                    6)  _tmp="TELNET" ;;
                    7)  _tmp="SSH" ;;
                    8)  _tmp="iSCSI" ;;
                    9)  _tmp="RADIUS" ;;
                    10) _tmp="VPN" ;;
                    *)  _tmp="unknown" ;;
                esac
            else
                _tmp="Unknown"
            fi
        fi

        [ "$j" == "2" ] && \
            if isInt $j ; then
                case $j in
                    1)  _tmp="Delete" ;;
                    2)  _tmp="Read" ;;
                    3)  _tmp="Write" ;;
                    4)  _tmp="Open" ;;
                    5)  _tmp="MakeDir" ;;
                    6)  _tmp="Mounted";;
                    7)  _tmp="Mount Failed" ;;
                    8)  _tmp="Rename" ;;
                    9)  _tmp="Login Fail" ;;
                    10) _tmp="Logged in" ;;
                    11) _tmp="Logout" ;;
                    12) _tmp="Unmount" ;;
                    13) _tmp="Copy" ;;
                    14) _tmp="Move" ;;
                    15) _tmp="Add" ;;
                    16) _tmp="Auth Fail" ;;
                    17) _tmp="Authorized" ;;
                    *)  _tmp="unknown" ;;
                esac
            else
                _tmp=Unknown
            fi

        _parsed[$i]=$_tmp
        #echo ${_parsed[$i]}
    done


    parsedConnLine=(${_parsed[@]})
}

## ###########################################################################
## get logs last entry num
line=$(/sbin/conn_log_tool --query --sort 1 --upper 1 --verbose | cleanLog | tail -n 1)
parseConnLog $line
lastID_conn=${parsedConnLine[0]}

_a=$(ls -l $logDB_conn)
while true
do
    _b=$(ls -l $logDB_conn)

    if test "$_a" != "$_b"
    then
        line=$(conn_log_tool --query --sort 1 --upper 1 --verbose | cleanLog | tail -n 1)
        parseConnLog $line
        curID_conn=${parsedConnLine[0]}
        content=$(conn_log_tool --query --sort 0 --lower ${lastID_conn} --upper ${curID_conn} --verbose | cleanLog)

        ## -----------------------------------------------------
        ## TODO: Read $content lines and parse
        ## -----------------------------------------------------
        ## TODO: Remove duplicate entries
        ## -----------------------------------------------------
        ## TODO: Format log messages and pass to delegate script
        ## -----------------------------------------------------

        echo $content

        ## -----------------------------------------------------
        ## TODO: Do the same for events log
        ## -----------------------------------------------------

        lastID_conn=$curID_conn
        _a=$_b
    fi
 
    /bin/sleep $INTERVAL
done
 
Ostatnio edytowane: