Instruktaż/Tutorial Przekazywanie pełnych logów serwera na Gadu-Gadu

Silas Mariusz · 10 Sierpień 2013

(Uwaga! Program nie jest jeszcze ukończony!)

Przekazywanie w czasie rzeczywistym
monitów oraz komunikatów
z serwera QNAP NAS na Gadu-Gadu

Dla wszystkich urządzeń QNAP NAS
arm-x09 TS-109/209
arm-x19 TS-x12/x19/x20/x21/+
x86 TS-509/x39/x59/x69/x79/+

- przewodnik instalacji -
2013, Silas Mariusz, rithien

[Przykład]

Kod:

12:09:13 QNAP Logs Notifier
(WARN) (!) The system was not shut down properly last time.
(INFO) System started.

12:12:42 QNAP Logs Notifier
[SAMBA] admin/10.100.200.100 (silas-ibm-pc)
Logged in
Read: Video/MOViES/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS.nfo
Read: Video/MOViES/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS.mkv
Read: Video/MOViES/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS/Dark.City.1998.Directors.Cut.720p.BluRay.x264-SiNNERS.srt

12:15:37 QNAP Logs Notifier
[SAMBA] admin/10.100.200.100 (silas-ibm-pc)
Read: Video/MOViES/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity.avi
Read: Video/MOViES/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity/Coffee.And.Cigarettes.2003.PL.SUB.720p.BRRip.XviD.AC3-inTGrity.nfo

12:17:51 QNAP Logs Notifier
[SAMBA] admin/10.100.200.100 (silas-ibm-pc)
Delete: Video/MOViES/Rejs.1970.PL.720p.HDTV.X264.AC3-TVM4iN/Rejs.1970.PL.720p.HDTV.X264.AC3-TVM4iN.mkv
Delete: Video/MOViES/Rejs.1970.PL.720p.HDTV.X264.AC3-TVM4iN

12:39:08 QNAP Logs Notifier
(WARN) [SAMBA] User2/193.93.122.60 (basesrv-old) (!) Login Fail

13:02:11 QNAP Logs Notifier
(WARN) admin/10.100.200.100 (!) [TCP/IP] Changed configuration of network interfaces [Trunking Group 1] from [balance-rr] to [balance-alb]
(INFO) Network connection resume.

13:05:47 QNAP Logs Notifier
(INFO) [ rtorrent:control ] Timezone Europe/Belgrade
(INFO) [ rtorrent:control ] Port 6009 (Web) is available.
(INFO) [ rtorrent:control ] Port 6008 (SecureWeb) is available.
(INFO) [ rtorrent:control ] Port 5000 (SCGI) is available.
(INFO) [ rtorrent:control ] Port 6881 (DHT) is available.
(INFO) [ rtorrent:control ] Port 42000 (Torrent) is available.
(INFO) [ rtorrent:control ] Redirecting ports (main): 5000, 6881, 42000
(INFO) [ rtorrent:control ] Iface IP address... 10.100.200.10
(INFO) [ rtorrent:control ] IP address... 8.8.8.8
(INFO) [ rtorrent:control ] Server hostname qcloudname
(INFO) [ rtorrent:control ] Server domain myqnapcloud.com
(INFO) [ rtorrent:control ] User mail address: qcloudname@myqnapcloud.com
(INFO) [ rtorrent:control ] Web server certificate found.
(INFO) [ rtorrent x: ] cmd: chmod 666 /dev/null...
(INFO) [ rtorrent x: ] cmd: chmod 666 /dev/ptmx...
(INFO) [ rtorrent:control ] screen-cleanup started.
(INFO) [ rtorrent:control ] rtorrent started.
(INFO) [ rtorrent:control ] lighttpd started.
(INFO) [ rtorrent:control ] Boot sequence done!

[Notka]
Przed przystąpieniem do instalacji można skorzystać z How-To ułatwiającego prace w Linuxie: QNAP od środka, czyli grzebiemy w linuchu - Jak sobie ułatwić?

[Instalacja]

W swoim komunikatorze Gadu-Gadu, dodaj nowy kontakt numer 48280073 (qLog-IM).
Będąc dostępnym, napisz do bota wiadomość o treści "ZAPISZ".

Następnie za pomocą klienta SSH (np. PuTTY) zaloguj się na serwer i wykonaj poniższe:

Bash:

# Znajdz katalog w ktorym przechowywane sa paczki qpkg
publicdir=`/sbin/getcfg Public path -f /etc/config/smb.conf`
qpkgdir=`cd $publicdir && cd ../.qpkg/ && pwd`

# Utwórz katalog w którym będzie znajdował się watcher
mkdir -p $qpkgdir/qlogWatcher
cd $qpkgdir/qlogWatcher

# Pobierz program i rozpakuj 
wget http://pool.qnapclub.pl/site/packages/qlogWatcher/qlogWatcher.tar.gz
tar zxvf qlogWatcher.tar.gz
rm -f qlogWatcher.tar.gz

Otwórz w celu edycji plik z ustawieniami config.ini. Możesz do tego celu użyć WinSCP lub edytora w konsoli.
Bash:
```
mcedit config.ini        # edytor Midnight Commander
vi config.ini            # klasyczny edytor viii
```
W kluczu ARGS znajdującym się w sekcji Delegate otwartego pliku, podaj swój numer Gadu-Gadu poprzedzając prefixem number=gg.
INI:
```
[Delegate]
ARGS=example=test&number=nr7662900
URL=http://pool.qnapclub.pl/site/tools/utils/im.gadugadu/bot.php

[General]
numLast_event=20
numLast_conn=0
INTERVAL=1
```
Zapisz zmodyfikowany plik z ustawieniami.
(Aby móc pisać w edytorze vi, należy na klawiaturze nacisnąć przycisk 'i', natomiast aby zapisać plik i wyjść z edytora naciśnij kolejno: ESC, W, Q i potwierdź Enterem.)

Teraz należy dodać plik qlogWatcher.sh do autostartu, aby program uruchamiał się przy każdym uruchomieniu serwera. W tym celu można posłużyć się najprostszą metodą wszczepienia się w inny skrypt startowy lub zapisać program jako paczka QPKG.

Bash:

# Skopiuj domyślne ikony
# ... tu uzupelnic

# Podlinkuj skrypty
ln -sf $qpkgdir/qlogWatcher/qlogWatcher.sh /etc/init.d/qlogWatcher.sh
ln -sf $qpkgdir/qlogWatcher/qlogWatcher.sh /etc/rcS.d/QS101qlogWatcher
ln -sf $qpkgdir/qlogWatcher/qlogWatcher.sh /etc/rcK.d/QK101qlogWatcher

# Dodaj jako program QPKG
setcfg qlogWatcher Name qlogWatcher -f /etc/config/qpkg.conf -c
setcfg qlogWatcher Version 1.0 -f /etc/config/qpkg.conf -c
setcfg qlogWatcher Author "Silas Mariusz, rithien" -f /etc/config/qpkg.conf -c
setcfg qlogWatcher QPKG_File qlogWatcher.qpkg -f /etc/config/qpkg.conf -c
setcfg qlogWatcher Date "2013-08-11" -f /etc/config/qpkg.conf -c
setcfg qlogWatcher Shell "${qpkgdir}/qlogWatcher/qlogWatcher.sh" -f /etc/config/qpkg.conf -c

Przygotowaną paczkę uruchom:
Bash:
```
/etc/init.d/qlogWatcher.sh enable
/etc/init.d/qlogWatcher.sh start
```
To wszystko! Teraz będziesz otrzymywał logi serwera w czasie rzeczywistym na określony przez Ciebie numer Gadu-Gadu.

[Kod źródłowy]
qlogWatcher.sh:

Bash:

#!/usr/bin/env sh
##
##   QNAP Log Watcher
##   part of QNAP qLog-IM System Log Messages Parser Tool
##
##   Copyright © 2013, Silas Mariusz <silas@qnap.com.antispam>
##             © 2013, rithien <magik@post.pl.antispam>
##
##                     QNAP Club Polska
##
##   This program is free software; you can redistribute it and/or modify
##   it under the terms of the GNU General Public License as published by
##   the Free Software Foundation; either version 2, or (at your option)
##   any later version.
##
##   This program is distributed in the hope that it will be useful,
##   but WITHOUT ANY WARRANTY; without even the implied warranty of
##   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
##   GNU General Public License for more details.
##
##   You should have received a copy of the GNU General Public License
##   along with this program; if not, write to the Free Software Foundation,
##   Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
##
##   The latest version of this software can be obtained here:
##
##   https://forum.qnap.net.pl/
##
##
##   FILE: qlogWatcher.sh
##
##   RELEASE: 1.0
##   DATE: 2013-08-11


_scriptname="qLog-watcher"


## Set Basics
## ###########################################################################
[ "x$DEBUG" != "x" ] && echo "Hello kitty" > /tmp/logParser.$$
: ${AWK=awk}
export PATH="$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin:/opt/bin:/opt/sbin"


## find my name and real current directory
## ###########################################################################
PROGNAME=`type $0 | awk '{print $3}'`  # search for executable on path

PROGDIR=`dirname $PROGNAME`            # extract directory of program
PROGNAME=`basename $PROGNAME`          # base name of program

[ "$PROGDIR" == "." ] && PROGDIR=`cd $PROGDIR && pwd || echo $PROGDIR`

# Fully qualify directory path (remove relative components and symlinks)
# WARNING: The "bash" "pwd" builtin does not resolve symbolic links
# "sh" (as a link to "bash") also does not,   but "csh" does handle it.
# Symbolic link removal however is not critical.
ORIGDIR=`pwd`                                 # original directory (builtin)
PROGDIR=`cd $PROGDIR && pwd || echo $PROGDIR` # program directory

# Results...
#    $ORIGDIR    -- where the users was when called
#    $PROGDIR    -- script directory location (and now current directory)
#    $PROGNAME   -- This scripts executable name


## ###########################################################################
## get Web share real name
WebShare=`/sbin/getcfg SHARE_DEF defWeb -d Qweb -f /etc/config/def_share.info`
[ $WebShare = "Qweb" ] && QMOUNTS=TRUE || QMOUNTS=FALSE


## ###########################################################################
## Read config
numLast_event=$(/sbin/getcfg General numLast_event -f "$PROGDIR/config.ini" -d 10)
numLast_conn=$(/sbin/getcfg General numLast_conn -f "$PROGDIR/config.ini" -d 0)

## INTERVAL between next read log cycle (in seconds)
INTERVAL=$(/sbin/getcfg General INTERVAL -f "$PROGDIR/config.ini" -d 60)

## Delegate Messages...
URL=$(/sbin/getcfg Delegate URL -f "$PROGDIR/config.ini" -d unknown)
if [ "$URL" == "unknown" ]; then
    echo " [!] Delegate not configured. Check config.ini..."
    exit 1
fi
ARGS=$(/sbin/getcfg Delegate ARGS -f "$PROGDIR/config.ini" -d unknown)

## Date expression for yesterday
DATE="-1 day"

## Locale for printf number formatting (e.g. "10000" => "10,000")
LANG=en_US

# Settings
agent="Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0"
timeout=30
retries=1


## ###########################################################################
## log DB file (used to watch log changes - file size diff)
logDB_event="/etc/logs/event.log"
logDB_conn="/etc/logs/conn.log"


## ###########################################################################
## wget with retries
function wget
{
    local i
    for ((i=0; i<3; i++)); do
        if command wget -O- --quiet --no-check-certificate -T $timeout -t $retries "$@"; then
            return 0
        fi
    done
    return 1
}


## ###########################################################################
## UrlEncode function
EncodeEOL=yes
LANG=C  export LANG
function UrlEncode() {
echo "$@" | $AWK '
    BEGIN {
        # We assume an awk implementation that is just plain dumb.
        # We will convert an character to its ASCII value with the
        # table ord[], and produce two-digit hexadecimal output
        # without the printf("%02X") feature.

        EOL = "%0A"             # "end of line" string (encoded)
        split ("1 2 3 4 5 6 7 8 9 A B C D E F", hextab, " ")
        hextab [0] = 0
        for ( i=1; i<=255; ++i ) ord [ sprintf ("%c", i) "" ] = i + 0
        if ("'"$EncodeEOL"'" == "yes") EncodeEOL = 1; else EncodeEOL = 0
    }
    {
        encoded = ""
        for ( i=1; i<=length ($0); ++i ) {
            c = substr ($0, i, 1)
            if ( c ~ /[a-zA-Z0-9.-]/ ) {
                encoded = encoded c             # safe character
            } else if ( c == " " ) {
                encoded = encoded "+"   # special handling
            } else {
                # unsafe character, encode it as a two-digit hex-number
                lo = ord [c] % 16
                hi = int (ord [c] / 16);
                encoded = encoded "%" hextab [hi] hextab [lo]
            }
        }
        if ( EncodeEOL ) {
            printf ("%s", encoded EOL)
        } else {
            print encoded
        }
    }
    END {
        #if ( EncodeEOL ) print ""
    }
'
}


## ###########################################################################
## Send log container
sendLog() {
    local _message=$(UrlEncode $@)
    wget "${URL}?${ARGS}&message=$_message"
}

err() {
    echo -e "$@"
    sendLog "$@"
    exit 1
}

## ###########################################################################
## Returns true if $1 is Integer
## usage: is_int ${var} && echo "that's an int"
isInt() {
    if test ${1} -eq ${1} 2>/dev/null; then
        return 0
    fi
    return 1
}

## ###########################################################################
## Returns true if $1 is one of $2..$n
arrayIsPresent() {
    local val="$1"
    shift
    for v in "$@"; do
        [ "$v" = "$val" ] && return 0
    done
    return 1
}

## ###########################################################################
# make sure log database is present
[ ! -f "$logDB_conn" ] && \
    err " Log Database file not exist $logDB_conn ... \n\n Please reinitialize database: \n    #  /sbin/conn_log_tool -c \n\n"
 
[ ! -f "$logDB_event" ] && \
    err " Log Database file not exist $logDB_event ... \n\n Please reinitialize database: \n    #  /sbin/log_tool -c \n\n"

## ###########################################################################
## clean output
cleanLog () {
    while read data
    do
        echo "$data" | grep -v "Query database" | \
                grep -v "Done. Return value is" | \
                grep -v "conn_id,conn_type,conn_date,conn_time,conn_user,conn_ip,conn_comp,conn_res,conn_serv,conn_action"
    done
}

parsedConnLine=
parsedConnMsg=
## ###########################################################################
## parseLogConn
parseConnLog () {
    parsedConnLine=
    local _parsed= ; local _tmp=
    local i=0 ; local j=0
    local inputLine=$@
    local _trash=$(echo -e $inputLine | cut -d\, -f8-)      # <trash>
    local _temp_1=$(echo -e $inputLine | cut -d\, -f-7)     # first part
    local _temp_2=$(echo -e $_trash | cut -d\, -f2-)        # last part
    parsedConnMsg==$(echo -e $_trash | tr -d "$_temp_2")        # log msg

    i=0 ; j=0
    for (( i = 0; i < 7; i++ )); do
        j=$(expr $i + 1)
        _tmp=$(echo $_temp_1 | cut -d\, -f${j})

        if [ "$j" == "2" ]; then
            if isInt $j ; then
                case $j in
                    0)  _tmp="INFO" ;;
                    1)  _tmp="WARN" ;;
                    2)  _tmp="ERR!" ;;
                    *)  _tmp="UNKNOWN" ;;
                esac
            else
                _tmp="UNKNOWN"
            fi
        fi

        _parsed[$i]=$_tmp
        #echo ${_parsed[$i]}
    done

    i=0 ; j=0
    for (( i = 7; i < 9; i++ )); do
        j=$(expr $i - 6)
        _tmp=$(echo $_temp_2 | cut -d\, -f${j})

        if [ "$j" == "1" ]; then
            if isInt $j ; then
                case $j in
                    1)  _tmp="SAMBA" ;;
                    2)  _tmp="FTP" ;;
                    3)  _tmp="HTTP" ;;
                    4)  _tmp="NFS" ;;
                    5)  _tmp="AFP" ;;
                    6)  _tmp="TELNET" ;;
                    7)  _tmp="SSH" ;;
                    8)  _tmp="iSCSI" ;;
                    9)  _tmp="RADIUS" ;;
                    10) _tmp="VPN" ;;
                    *)  _tmp="unknown" ;;
                esac
            else
                _tmp="Unknown"
            fi
        fi

        [ "$j" == "2" ] && \
            if isInt $j ; then
                case $j in
                    1)  _tmp="Delete" ;;
                    2)  _tmp="Read" ;;
                    3)  _tmp="Write" ;;
                    4)  _tmp="Open" ;;
                    5)  _tmp="MakeDir" ;;
                    6)  _tmp="Mounted";;
                    7)  _tmp="Mount Failed" ;;
                    8)  _tmp="Rename" ;;
                    9)  _tmp="Login Fail" ;;
                    10) _tmp="Logged in" ;;
                    11) _tmp="Logout" ;;
                    12) _tmp="Unmount" ;;
                    13) _tmp="Copy" ;;
                    14) _tmp="Move" ;;
                    15) _tmp="Add" ;;
                    16) _tmp="Auth Fail" ;;
                    17) _tmp="Authorized" ;;
                    *)  _tmp="unknown" ;;
                esac
            else
                _tmp=Unknown
            fi

        _parsed[$i]=$_tmp
        #echo ${_parsed[$i]}
    done


    parsedConnLine=(${_parsed[@]})
}

## ###########################################################################
## get logs last entry num
line=$(/sbin/conn_log_tool --query --sort 1 --upper 1 --verbose | cleanLog | tail -n 1)
parseConnLog $line
lastID_conn=${parsedConnLine[0]}

_a=$(ls -l $logDB_conn)
while true
do
    _b=$(ls -l $logDB_conn)

    if test "$_a" != "$_b"
    then
        line=$(conn_log_tool --query --sort 1 --upper 1 --verbose | cleanLog | tail -n 1)
        parseConnLog $line
        curID_conn=${parsedConnLine[0]}
        content=$(conn_log_tool --query --sort 0 --lower ${lastID_conn} --upper ${curID_conn} --verbose | cleanLog)

        ## -----------------------------------------------------
        ## TODO: Read $content lines and parse
        ## -----------------------------------------------------
        ## TODO: Remove duplicate entries
        ## -----------------------------------------------------
        ## TODO: Format log messages and pass to delegate script
        ## -----------------------------------------------------

        echo $content

        ## -----------------------------------------------------
        ## TODO: Do the same for events log
        ## -----------------------------------------------------

        lastID_conn=$curID_conn
        _a=$_b
    fi
 
    /bin/sleep $INTERVAL
done

dzwigar · 3 Listopad 2013

Przykro mi ale nie mam paczki qlogWatcher.tar.gz.

Nigdzie nie moge znalesc.
Prosze o pomoc.

Silas Mariusz · 27 Styczeń 2014

Silas Mariusz napisał:
(Uwaga! Program nie jest jeszcze ukończony!)

a program jest załączony jako kod źrodłowy~~Projekt można uważać za zamknięty, do roadmapu QNAP'a został włączony zasugerowany Report Publisher:~~

4] (important)
Report Publisher (exact copy of SMSC service) – dupe one, but dedicated to forward activities (system/connection logs) onto external central management and monitoring system.

It can be applied in a scenario when network administrator needs to use custom notification service other than one used in SMSC. Maybe you can think it’s nothing, but users are expecting instant view on activity and log.

On QNAPClub.pl we re hosting several gateways for local services.
Article on QNAP Club | 26001 gateway | 26003 gateway | IM (Gadu-Gadu) gateway | LogWatcher and Parser (dev)

Example scenarios:
1: locally hosted gateway written in PHP delivers system messages to foreign IM with user custom message filters
2: locally hosted RSS Aggregator written in PHP collects server activity log

(at least unlock std msg in SMSC - p.5)

5] (important)

Ability to deliver ALL type messages through SMSC service - including standard (information messages) (currently only warns and errors)

(In some cases it may be an alternative for p.4)

Szukaj

Instruktaż/Tutorial Przekazywanie pełnych logów serwera na Gadu-Gadu

Silas Mariusz

rm -rf /

dzwigar

Nowy użytkownik

Silas Mariusz

rm -rf /

Mogą Cię zainteresować