Observium nfsen configuration notes

Hacked my way through getting Observium to pick up the nfsen RRD so that I see the Netflow tab in the device in Observium.
2016-06-16_153005
Here’s what I did…

Install Prerequisite Software

apt-get install gcc flex librrd-dev make librrdp-perl librrds-perl libsocket6-perl libmailtools-perl mrtg rrdtool

Install nfdump

Download nfdump from SourceForge: https://sourceforge.net/projects/nfdump/files/stable/nfdump-1.6.13/

tar zxvf nfdump-1.6.13.tgz
cd nfdump-1.6.13/
./configure --enable-nfprofile --enable-nftrack
make && make install

Install nfsen

Download nfsen from SourceForge: https://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.7/

tar zxvf nfsen-1.3.7.tgz
cd nfsen-1.3.7
cp etc/nfsen.conf.dist etc/nfsen.conf

Make configuration changes to nfsen
Modify etc/nfsen.conf

$USER = www-data;
$WWWUSER = www-data;
$WWWGROUP = www-data;
%sources = (
'routername' => { 'port' => '9996', 'col' => '#0000ff', 'type' => 'netflow', 'IP' => '1.2.3.4' },
);
$MAIL_FROM = 'me@domain.com';
$SMTP_SERVER = 'mail.domain.com';

Save the file and then make a directory where nfsen will store data.

mkdir -p /var/nfsen
./install.pl etc/nfsen.conf

Start nfsen

cd /var/nfsen/bin
./nfsen start

Configure to start nfsen automatically at reboot.

ln -s /var/nfsen/bin/nfsen /etc/init.d/nfsen
update-rc.d nfsen defaults 20

Configure Apache2

Configure Apache2 so we can access nfsen while still using observium.
Make a directory to store nfsen HTML files

mkdir -p /var/www/html/nfsen

Edit /etc/apache2/conf-enabled/observium.conf and add the following line before the closing </VirtualHost>.

Alias /nfsen /var/www/html/nfsen

Restart Apache2

service apache2 restart

At this point you should be able to access http://yourip/nfsen/nfsen.php

Cannot create graph

If you see that error, check permissions of /var/nfsen and make sure it is accessible by www-data specified in /var/nfsen/etc/nfsen.conf.

Observium Configuration

Note: The %source in /var/nfsen/etc/nfsen.config must match the host you are using in Observium and it is case sensitive.
So I had a hard time with Observium configuration and decided to just hack it up.
I have Observium installed in /opt/observium, so substitute accordingly.
Add the following to /opt/observium/config.php.

$config['nfsen_enable'] = 1;
$config['nfsen_rrds'] = /var/nfsen/profiles-stat/live/;
$config['nfsen_split_char'] = ;
$config['nfsen_suffix'] = ;

Enjoy your graphs.
2016-06-16_155352

ISPConfig, Dovecot, Postfix and LetsEncrypt SSL

Please See: https://www.howtoforge.com/community/threads/lets-encrypt-working-with-ispconfig-interface-postfix-dovecot-tls-pure-ftpd-monit.75546/

ARCHIVED
I successfully configured Dovecot and Postfix to use my LetsEncrypt SSL certificate for my mail domain.

Generate SSL Certificate

When I installed ISPConfig 3.1b, I followed instructions on setting up LetsEncrypt which placed it in /opt/letsencrypt; If you have LetsEncrypt installed elsewhere, substitute the path below with the correct path.
I run in standalone mode so I need to stop Apache2.

service apache2 stop

Create the certificate.
Update: 12/2016 Install the certbot tool following this guide for Debian Jessie 8:  https://certbot.eff.org/all-instructions/#debian-8-jessie-apache

certbot certonly --standalone -d mail.techish.net

The certificate now lives in /etc/letsencrypt/live/mail.techish.net/

Configure Dovecot

I modified /etc/dovecot/conf.d/10-ssl.conf and added the following lines:

ssl = yes
ssl_cert = </etc/letsencrypt/live/mail.techish.net/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.techish.net/privkey.pem

Then I restarted Dovecot

service dovecot restart

That didn’t seem to work;  it was still publishing an invalid certificate, so I had a look around at more configuration files.
I then modified /etc/dovecot/dovecot.conf file.  In this file I saw the ssl_cert and ssl_key variables and I also noted the protocols. The bolded items are what I changed/added.

protocols = imap pop3 imaps pop3s
auth_mechanisms = plain login
disable_plaintext_auth = no
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_privileged_group = vmail
#ssl_cert = </etc/postfix/smtpd.cert
#ssl_key = </etc/postfix/smtpd.key
ssl_cert = </etc/letsencrypt/live/mail.techish.net/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.techish.net/privkey.pem

I then restarted Dovecot

service dovecot restart

Configure Postfix

I looked at /etc/postfix/main.cf and noted that the cert was pointed to /etc/postfix/ directory. I decided to backup the certs that existed and then create a symlink.

smtpd_tls_cert_file = /etc/postfix/smtpd.cert
smtpd_tls_key_file = /etc/postfix/smtpd.key

Backup and create symlinks.

cd /etc/postfix
mkdir ssl-backup
mv smtpd.* ssl-backup/
ln -s /etc/letsencrypt/live/mail.techish.net/fullchain.pem smtpd.cert
ln -s /etc/letsencrypt/live/mail.techish.net/privkey.pem smtpd.key

Restart Postfix

service postfix restart

Test Internet Speed on Linux Terminal

Instead of visiting sites like Speedtest.net through a browser, you can install speedtest-cli package on Linux and test bandwidth speeds from the command line.

$ apt-get install speedtest-cli

Here’s an example running the command.

root@zabbix:~# speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from Level 3 Communications (4.49.115.30)...
Selecting best server based on latency...
Hosted by Armstrong (Butler, PA) [44.33 km]: 13.106 ms
Testing download speed........................................
Download: 14.40 Mbits/s
Testing upload speed..................................................
Upload: 0.65 Mbits/s

Update 4/2022

root@kreinix:~# speedtest-cli
Retrieving speedtest.net configuration...
Testing from Lumen (4.49.115.30)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Magna5 (Boyers, PA) [58.69 km]: 17.63 ms
Testing download speed................................................................................
Download: 90.77 Mbit/s
Testing upload speed......................................................................................................
Upload: 86.52 Mbit/s

DNS Caching for Spamassassin RBLs

So I’m tweaking the mail filter server which is a Debian Linux server running Postfix, MailScanner and SpamAssassin.

I just wanted to share some of the performance improvements after installing pdns-recursor for local caching.

Install PowerDNS

root@mxfilter:~# apt-get install pdns-recursor

Obtain a sample spam email

root@mxfilter:~# wget http://people.apache.org/~wtogami/sample-spam.eml

First Test

root@mxfilter:~# cat sample-spam.eml | spamassassin -D 2>&1 | grep 'async: timing' | sed 's/^.*dbg: async: //'
timing: 0.740 . dns:A:45.135.176.118.iadb.isipp.com.
timing: 0.741 . dns:A:45.135.176.118.dnsbl.sorbs.net.
timing: 0.749 . dns:TXT:45.135.176.118.sa-accredit.habeas.com.
timing: 0.749 . dns:A:45.135.176.118.bb.barracudacentral.org.
timing: 0.750 . dns:TXT:45.135.176.118.bl.spamcop.net.
timing: 0.752 . dns:A:45.135.176.118.psbl.surriel.com.
timing: 0.753 . dns:A:45.135.176.118.list.dnswl.org.
timing: 0.756 . dns:A:45.135.176.118.zen.spamhaus.org.
timing: 0.758 . dns:A:45.135.176.118.bl.score.senderscore.com.
timing: 1.790 . dns:TXT:45.135.176.118.sa-trusted.bondedsender.org.

Second Test

timing: 0.002 . dns:A:45.135.176.118.iadb.isipp.com.
timing: 0.006 . dns:TXT:45.135.176.118.sa-accredit.habeas.com.
timing: 0.012 . dns:A:45.135.176.118.list.dnswl.org.
timing: 0.016 . dns:A:45.135.176.118.bl.score.senderscore.com.
timing: 0.206 . dns:A:45.135.176.118.psbl.surriel.com.
timing: 0.996 . dns:A:45.135.176.118.dnsbl.sorbs.net.
timing: 1.001 . dns:TXT:45.135.176.118.bl.spamcop.net.
timing: 1.003 . dns:A:45.135.176.118.bb.barracudacentral.org.
timing: 1.003 . dns:TXT:45.135.176.118.sa-trusted.bondedsender.org.
timing: 1.009 . dns:A:45.135.176.118.zen.spamhaus.org.

After running pdns-recursor for about 5 minutes here are some statistics.

root@mxfilter:~# rec_control get-all
all-outqueries  116
dlg-only-drops  0
dont-outqueries 0
outgoing-timeouts       0
tcp-outqueries  4
throttled-out   0
throttled-outqueries    0
unreachables    0
answers-slow    0
answers0-1      0
answers1-10     0
answers10-100   1
answers100-1000 24
case-mismatches 0
chain-resends   0
client-parse-errors     0
edns-ping-matches       0
edns-ping-mismatches    0
ipv6-outqueries 0
no-packet-error 0
noedns-outqueries       120
noerror-answers 15
noping-outqueries       0
nsset-invalidations     0
nxdomain-answers        18
over-capacity-drops     0
qa-latency      893
questions       33
resource-limits 0
server-parse-errors     0
servfail-answers        0
spoof-prevents  0
tcp-client-overflow     0
tcp-questions   0
unauthorized-tcp        0
unauthorized-udp        0
unexpected-packets      0
cache-entries   496
cache-hits      0
cache-misses    25
concurrent-queries      0
negcache-entries        10
nsspeeds-entries        369
packetcache-entries     24
packetcache-hits        8
packetcache-misses      25
sys-msec        36
tcp-clients     0
throttle-entries        0
uptime  462
user-msec       48

ldapsearch to get all SMTP email addresses

Cooked this up in a few minutes.  Should return all proxyAddress that are SMTP and all SMTP default addresses for all users not disabled in a specified Organizational Unit.

ldapsearch -x -h ldapserver.example.com -b ou=Users,ou=example,dc=example,dc=com -D exampleldapadmin -w s3cr3t '(& (objectClass=User)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))' | grep -i ^mail|^proxyAddresses: SMTP: | grep -v mailNickname | sed 's/mail: //' | sed -e 's/proxyAddresses: SMTP://gI' | sed 's!$! OK!' | uniq >> $VALID