Gluster on Debian

Remote all installation and config of gluster
apt-get purge --auto-remove glusterfs-server

Install gluster on servers, (server1, server2)
apt-get install glusterfs-server

On server1
gluster peer probe server2

On server2
gluster peer probe server1

On server1
gluster volume create gv0 replica 2 server1:/path/to/folderserver1 server2:/path/to/folderserver2

gluster volume start gv0 

On server2
mount -t glusterfs server2:/gv0 /path/to/sharedir 

sharedir is represent for broadcast to 2 folderserver1 and folderserver2, data store in sharedir is replicated to folderserver1 and folderserver2

Add column to all tables same schema

CREATE OR REPLACE FUNCTION add_version_column_to_all_tables()
RETURNS VOID
AS $$
DECLARE
 my_row RECORD;
BEGIN
 FOR my_row IN
 SELECT *
 FROM information_schema.tables
 WHERE table_schema = 'public' AND table_type='BASE TABLE'
 LOOP
 IF NOT EXISTS
 (
 SELECT attname FROM pg_attribute WHERE attrelid =
 (SELECT oid FROM pg_class WHERE relname = my_row.table_name )
 AND attname = 'subtenant'
 )
 THEN
     IF EXISTS
     (
         SELECT attname FROM pg_attribute WHERE attrelid =
         (SELECT oid FROM pg_class WHERE relname = my_row.table_name )
         AND attname = 'tenantid'
     )
     THEN
         EXECUTE('ALTER TABLE ' || my_row.table_name || ' ADD COLUMN subtenant text;');
         EXECUTE('UPDATE ' || my_row.table_name || ' SET subtenant = tenantid;');
     END IF;
 END IF;
 END LOOP;
END
$$
LANGUAGE plpgsql;

SELECT add_version_column_to_all_tables();

Test Ping

https://drive.google.com/open?id=0B30EeVPAcvhveHF6RDNaYkJGZGM

Usage:

java -jar TestNetword.jar ping domain1 domain2 domain3

<replace ping by "ping -t" on windows>

java -jar TestNetwork.jar report domain1.log filtervalue

<replace filtervalue by "timeout" or "15:01:" time point>

Install iRedmail on Debian 8

1. Install iRedmail follow official guide onhttp://www.iredmail.org/docs/index.html#install-iredmail

2. Reinstall Clamav
apt-get remove clamav clamav-daemon
apt-get install clamav clamav-daemon
/etc/init.d/clamav-daemon restart
/etc/init.d/clamav-freshclam restart

 3. Register TLS cert on startssl.com

4. When STARTSSL.COM send email to webmaster@youmaildomain.com to verify domain, maybe it is listed in greylist and is denied by postfix, so you should add ip and account of starssl into whitelist (you can check using tail -f /var/log/mail.log for detect ip and email account of startssl)

cd /opt/iredapd/tools/
python wblist_admin.py --list --whitelist
python wblist_admin.py --add --whitelist 4.14.40.143 no-reply@startcomca.com
python wblist_admin.py --add --whitelist 4.14.40.142 no-reply@startcomca.com

5. When register TLS cert successfully, you will receive a zip (assum it is gbsofts.net.zip) file contains cert and other info, download it and put into /root/cert/
mkdir /root/cert
cd /root/cert
openssl req -newkey rsa:2048 -keyout gbsofts.net.key -out gbsofts.net.csr
openssl rsa -in gbsofts.net.key -out gbsofts.net.decrypt.key
wget --no-check-certificate https://www.startssl.com/certs/ca-bundle.pem -O startssl-ca-bundle.crt
unzip gbsofts.net.zip
unzip NginxServer.zip
cat 1_gbsofts.net_bundle.crt gbsofts.net.decrypt.key > gbsofts.net.pem

nano /etc/nginx/conf.d/00-default.conf

ssl_certificate /root/cert/1_gbsofts.net_bundle.crt;
ssl_certificate_key /root/cert/gbsofts.net.decrypt.key;

nano /etc/postfix/main.cf

smtpd_tls_key_file = /root/cert/gbsofts.net.decrypt.key
smtpd_tls_cert_file = /root/cert/1_gbsofts.net_bundle.crt
smtpd_tls_CAfile = /root/cert/startssl-ca-bundle.crt

nano /etc/dovecot/dovecot.conf

ssl_cert = </root/cert/1_gbsofts.net_bundle.crt
ssl_key = </root/cert/gbsofts.net.decrypt.key

service postfix restart
service dovecot restart
service nginx restart

6. Add TXT record to domain to set SPF
v=spf1 ip4:a.b.c.d ~all

for other domain, add TXT
v=spf1 include:otherdomain.com ip4:a.b.c.d ~all

Authen SSH using key

ssh-keygen -t rsa

ssh username@remoteserver mkdir -p .ssh

cat .ssh/id_rsa.pub | ssh username@remoteserver 'cat >> .ssh/authorized_keys' 

Build Postgres from source

Install libs for compile source postgres

apt-get install build-essential libreadline-dev zlib1g-dev llvm-7 clang-7

if use Debian 11 or Ubuntu 22.04, change llvm and clang to current version

apt-get install build-essential libreadline-dev zlib1g-dev llvm clang libicu-dev pkg-config
 

Extract postgres source to postgres-source-dir (change lllvm-7 to current version llvm path)

mkdir -p /home/userpath/postgres/pgsql
cd postgres-source-dir

./configure --with-llvm LLVM_CONFIG="/usr/lib/llvm-7/bin/llvm-config" --prefix=/home/userpath/postgres/pgsql

make

make install

if you want to install postgres-contrib extension, you can compile it

cd contrib
make
make install

cd \

mkdir postgres

cd postgres

mkdir database

mkdir pgsql

pgsql/bin/initdb --locale en_US.UTF-8 -D database

Now you can setup everything with new portable postgres include 2 folder: pgsql and database

Run portable postgres 9.5.3 on Debian 8 64bit

I built portable postgres 9.5.3 from source for Debian 8 64bit. You can download it and free to use wihout install any dependencies.

Get file from https://drive.google.com/open?id=0B30EeVPAcvhvZmNFbXFidFVYRTA

tar xvf postgresql_9.5.3_portable_debian8_64.tar.gz

cd postgres

sh start_postgres.sh

To stop postgres

sh stop_postgres.sh

To use other command of Postgres:

source config_postgres

then all commands of postgres should be work properly

Update link for Postgres 9.6.2: https://drive.google.com/open?id=0B30EeVPAcvhvRHotSDdxYTA2LTA

Install Gitlab on Debian 7

Install Gitlab

sudo apt-get install curl
curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo apt-get install gitlab-ce
sudo gitlab-ctl reconfigure

Open file /etc/gitlab/gitlab.rb
sudo nano /etc/gitlab/gitlab.rb

Change lines

external_url 'http://yougitdomain.com'

gitlab_rails['gitlab_email_from'] = 'notification@yourdomain.com'
gitlab_rails['gitlab_email_display_name'] = 'Notification'
gitlab_rails['gitlab_email_reply_to'] = 'notification@yourdomain.com'

gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.gmail.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "yourgmail@gmail.com"
gitlab_rails['smtp_password'] = "yourpassword"
gitlab_rails['smtp_domain'] = "smtp.gmail.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = false
gitlab_rails['smtp_openssl_verify_mode'] = 'peer'
gitlab_rails['smtp_ca_path'] = "/etc/ssl/certs"
gitlab_rails['smtp_ca_file'] = "/etc/ssl/certs/ca-certificates.crt"

nginx['listen_port'] = 5555 #replace by your port

Reconfig Gitlab
sudo gitlab-ctl reconfigure

Gitlab will run on port 5555 (as above config)

Troubleshoot: when restart OS, maybe Gitlab cannot start, you will install systemd to auto boot Gitlab

sudo apt-get install systemd

Compile openssl

Get latest openssl serouce, at this time is link https://www.openssl.org/source/openssl-1.0.2h.tar.gz
Extract and point to source directory path.

apt-get install gcc
./config
make
make install
openssl version
mv /usr/bin/openssl /usr/bin/openssl.bak
ln -s /usr/local/ssl/bin/openssl /usr/bin/openssl
openssl version

Resize partition size

root@server:/home/server# lsblk
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda    254:0    0  20G  0 disk
└─vda1 254:1    0  20G  0 part /

root@server:/home/server# df -ah
Filesystem      Size  Used Avail Use% Mounted on
rootfs          9.9G  3.7G  5.8G  39% /
sysfs              0     0     0    - /sys
proc               0     0     0    - /proc
udev             10M     0   10M   0% /dev
devpts             0     0     0    - /dev/pts
tmpfs           388M  136K  388M   1% /run
/dev/vda1       9.9G  3.7G  5.8G  39% /
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           775M     0  775M   0% /run/shm

Block device size = 20GB
Current partition size= 10GB

Auto resize partition will expand it same block device size

root@server:/home/server# resize2fs /dev/vda1
resize2fs 1.42.5 (29-Jul-2012)
Filesystem at /dev/vda1 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/vda1 to 5242624 (4k) blocks.
The filesystem on /dev/vda1 is now 5242624 blocks long.

root@serer:/home/server# df -ah
Filesystem      Size  Used Avail Use% Mounted on
rootfs           20G  3.7G   16G  20% /
sysfs              0     0     0    - /sys
proc               0     0     0    - /proc
udev             10M     0   10M   0% /dev
devpts             0     0     0    - /dev/pts
tmpfs           388M  136K  388M   1% /run
/dev/vda1        20G  3.7G   16G  20% /
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           775M     0  775M   0% /run/shm

Install Haproxy from source on Debian 7

apt-get install build-essential make g++ libssl-dev

tar xvf haproxy-1.6.5.tar.gz

cd haproxy-1.6.5

make TARGET=linux2628  USE_OPENSSL=yes USE_ZLIB=yes

make install

cp /usr/local/sbin/haproxy /usr/sbin/
cp examples/haproxy.init /etc/init.d/haproxy
chmod 755 /etc/init.d/haproxy
mkdir -p /etc/haproxy
mkdir -p /run/haproxy
mkdir -p /var/lib/haproxy
touch /var/lib/haproxy/stats
useradd -r haproxy

Create pem file for Haproxy

Decrypt private key
openssl rsa -in domain.com.key -out domain.com.decrypt.key

unzip nginx.zip from Starssl
cat 1_domain.com_bundle.crt domain.com.decrypt.key > domain.com.pem

Install SRILM

Download and install ubuntu minimal system from http://archive.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64/current/images/netboot/mini.iso

Download source srilm 1.7.1 from: https://googledrive.com/host/0B30EeVPAcvhvdG04eVE4Uml1ZGs/srilm-1.7.1.tar.gz

wget http://googledrive.com/host/0B30EeVPAcvhvdG04eVE4Uml1ZGs/srilm-1.7.1.tar.gz
mkdir srilm
mv srilm-1.7.1.tar.gz srilm
cd srilm
tar -xvf srilm-1.7.1.tar.gz
export SRILM=/home/gbsofts/srilm
sh sbin/machine-type
sudo -s
apt-get install gcc
apt-get install gawk
apt-get install tcl
apt-get install tcl-dev tk-dev
apt-get install gzip bzip2 p7zip
apt-get install make
cd ../
wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz
tar -xvzf libiconv-1.11.tar.gz
cd libiconv-1.11
./configure --prefix=/usr/local/libiconv
make
make install
cd ../srilm/
export SRILM=/home/gbsofts/srilm
export PATH=$PATH:$SRILM/bin:$SRILM/bin/$MACHINE_TYPE
#change SRILM = /home/gbsofts/srilm from Makefile
make MACHINE_TYPE=i686-m64 World

Config date time

Change timezone:
dpkg-reconfigure tzdata

If you do not have root privileges or want to set for yourself a different timezone than the one the system uses, you can set the environment variable TZ. Use the command tzselect to show what value to use for TZ. 
export TZ=UTC

GIN Index on Postgres

Install postgresql-contrib package 
apt-get install postgresql-contrib-9.x
(replace x with current version in used)


Create extension pg_trgm
log in postgres with superuser role,
CREATE EXTENSION IF NOT EXISTS pg_trgm;


Create GIN index
CREATE INDEX tbdatatest_idx_date  on tbdatatest  USING GIN ( playerid gin_trgm_ops);

JCache with Payara