GitLab CI
May 7, 2022 — 20:36

Author: silver  Category: dev linux  Comments: Off

There’s a lot of things to like about GitLab in my opinion such as it’s API’s, the MR workflow and Terraform integration to name a few. Of course, there’s things to dislike too ;-)

Below are a few tips and notes on working with CI and editing .gitlab-ci.yml.

For a proper quick start see: https://docs.gitlab.com/ee/ci/quick_start

.

Variables

Predefined vars:

  • Package Registry: $CI_REGISTRY (docker login)
  • Docker image: $CI_REGISTRY_IMAGE
  • Build dir: $CI_PROJECT_DIR (docker WORKDIR)
  • Debugging: $CI_DEBUG_TRACE: "true"

Full list: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html

Conditional var:

Example where if INSTALL_ALL is set to "false", a docker image tag is added:

variables:
  INSTALL_ALL = "true"
  DOCKER_IMAGE: "${CI_REGISTRY_IMAGE}/foo"

workflow:
  rules:
    - if: $INSTALL_ALL == "false"
      variables:
        DOCKER_IMAGE: "${CI_REGISTRY_IMAGE}/foo:slim"

.

Jobs

Besides using the debug var mentioned above, this kludge is also useful when debugging. To quickly disable a job add a dot in front of it’s name “my_build_job”:

stages:
  - test
  - build

test_job:
 stage: test
 < ... >

.my_build_job:
  stage: build
  < ... >

.

Scripts

This is one line:

script:
  - test -d dir &&
      echo "dir exists"

Multi line:

script:
  - |
    echo "One"
    echo "Two"

.

Tags

First add tag(s) to runner in GitLab GUI: ‘Settings > CI/CD > Runners’. Then use those tags in gitlab-ci.yml to select a specific runner for job execution.

E.g. add inside job:

tags:
   - docker
   - socket

.

Docker in Docker (dind)

Used to build containers. For own self-hosted Runners there’s 2 possible methods using the docker executer: "privileged" or "socket".

This requires changing config.toml, under [runners.docker]:

Add "/var/run/docker.sock:/var/run/docker.sock" to volumes

Or privileged = true

The socket method is more secure but can be a bit more difficult to work with. For example when you need access to $CI_PROJECT_DIR inside container. A solution is available here.

It’s also possible to use the shell executor (also allows using docker compose).

Other alternatives are using Podman/Buildah or Kaniko instead of Docker.

For details see https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#enable-docker-commands-in-your-cicd-jobs.

DevOps
May 5, 2022 — 18:00

Author: silver  Category: dev linux  Comments: Off

What is a Man DevOps?

It started out with a pretty clear definition: Development and Operations teams working together (history at Wikipedia). Meaning Dev’s no longer throw code over the fence for Ops to take care of. And no more long waits for Ops to deliver new environments. Instead, leverage Infrastructure as Code (IaC) and create CI/CD pipelines to build, test and deploy code automatically.

Then it became a hype and now many companies use the name “DevOps Engineer” for all kinds of different roles such as Sysadmins or Platform/Infra Engineers. Since there is certainly overlap this can make matching knowledge, experience and skills to what is actually needed somewhat difficult.

Also, the term is currently often associated with Cloud, Microservices, Containers, Kubernetes, GitOps, services like AWS CF and Azure Pipelines and tooling such as GitLab CI, Docker, Ansible and Terraform…

and a lot more (via: CNCF, digital.ai)


DevOps Ad Absurdum

From source


DevOps ..without Devs? Wait, did anything actually change?

Before DevOps / After DevOps : r/ProgrammerHumor

From source


at least it’s easier to transfer a container to a cloud, than move a laptop to a DC ;)
IPMI
March 26, 2021 — 20:06

Author: silver  Category: hardware linux windows  Comments: Off

Intelligent Platform Management Interface

  • Linux kernel module: ipmi_devintf
  • Debian packages: apt install openipmi ipmitool

ipmitool

Get info:

ipmitool lan print
ipmitool user list
ipmitool channel getaccess 1

Add new user and permissions:

ipmitool user set name 3 myusername
ipmitool user set password 3 <password> 16
ipmitool channel setaccess 1 3 link=on ipmi=on callin=on privilege=4
ipmitool test 3 16 <password>

* where ‘3’ is user id and ’16’ is length of password

SuperMicro

  • IPMIView is a GUI app specific to SM BMC (Linux/Windows)
  • Since 2019 the default "ADMIN" password is no longer used
  • Every device now has a "Unique BMC Password" printed on sticker (pdf)
  • ADMIN user has id 2
Chroot SSH
February 5, 2021 — 0:19

Author: silver  Category: linux  Comments: Off

Example of a minimal config that allows "luser" to only execute "bash" and "ls" in his homedir, after logging in via ssh (Debian/Ubuntu).

Of course there are other ways to accomplish this like linux ns/cgroups, sandboxes such as bubblewrap, firejail or by using containers (gvisor) or microvm’s.

/etc/passwd:

luser:x:1003:1003::/app:/bin/bash

/etc/group:

prison:x:1004:luser

/etc/ssh/sshd_config:

# Optional: do not chroot source ip range 43.21.*
# Match User luser Address 43.21.*
#   ChrootDirectory none

Match User luser
  ChrootDirectory /home/luser/jail

AllowUsers luser@localhost

homedir:

drwxr-xr-x     root     root     /home/luser/
drwxr-x---     root   prison     /home/luser/jail

drwxr-xr-x     root     root     /home/luser/jail/etc
drwxr-xr-x     root     root     /home/luser/jail/bin
drwxr-xr-x     root     root     /home/luser/jail/dev
drwxr-xr-x     root     root     /home/luser/jail/lib
drwxr-xr-x     root     root     /home/luser/jail/lib64
drwxr-xr-x     root     root     /home/luser/jail/usr

drwxr-x---     luser    luser    /home/luser/jail/app

lrwxrwxrwx     root     root     /app -> /home/luser/jail/app

dev (mknod):

crw-r--r--   1 root     root       5,   0   /home/luser/jail/dev/tty
crw-r--r--   1 root     root       1,   8   /home/luser/jail/dev/random
crw-r--r--   1 root     root       1,   5   /home/luser/jail/dev/zero
crw-r--r--   1 root     root       1,   3   /home/luser/jail/dev/null

etc files:

-rw-r--r--   1 root     root    /home/luser/jail/etc/passwd (copy from /etc)
-rw-r--r--   1 root     root    /home/luser/jail/etc/group  (copy from /etc)

bin files:

drwxr-xr-x   2 root     root    /home/luser/jail/bin
-rwxr-xr-x   1 root     root    /home/luser/jail/bin/bash
-rwxr-xr-x   1 root     root    /home/luser/jail/bin/ls

lib files:

-rwxr-xr-x  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libpthread.so.0
-rw-r--r--  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libtinfo.so.6
-rw-r--r--  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libdl.so.2
-rw-r--r--  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libpcre.so.3
-rwxr-xr-x  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libc.so.6
-rw-r--r--  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libselinux.so.1
-rw-r--r--  root     root       /home/luser/jail/lib/x86_64-linux-gnu/libnss_files.so.2
drwxr-xr-x  root     root       /home/luser/jail/lib/terminfo

lib64 files:

-rwxr-xr-x   root     root       /home/luser/jail/lib64/ld-linux-x86-64.so.2

usr/bin files:

drwxr-xr-x   root     root       /home/luser/jail/usr/bin
-rwxr-xr-x   root     root       /home/luser/jail/usr/bin/dircolors
Partitions
January 22, 2021 — 17:21

Author: silver  Category: linux windows  Comments: Off

Linux

Create new partition with parted:

# mbr
parted /dev/sda mklabel msdos
parted /dev/sda mkpart primary ext4 0% 100%

# gpt
parted /dev/sda mklabel gpt
parted /dev/sda mkpart part-label ext4 0% 100%

Note that ‘part-label’ cannot be empty, to remove: fdisk /dev/sda x n <enter> r w

To resize: parted /dev/sdX then resizepart X

Clone MBR partition table:

Use sfdisk:

sfdisk -d /dev/sda > partitions.txt
sfdisk /dev/sdb < partitions.txt

sfdisk -d /dev/sda | sfdisk /dev/sdb
sfdisk -d /dev/sda | sfdisk --force /dev/sdb

Clone GPT partition table:

  • use sgdisk
  • or gdisk

Windows

Partitioning tools:

  • diskpart (cli)
  • diskmgmt.msc
  • Minitool Partition Wizard (3rd party)
  • EaseUS Partition Master (3rd party)

vnStat
January 22, 2021 — 16:58

Author: silver  Category: bsd linux  Comments: Off

You might have used vnstat to view monthly traffic statistics in CLI, e.g. vnstat -i eth0 -m. But if you’re like me you might also not have followed the project for a few years and know there’s now JSON output available (--json) and there is vnstati to output to PNG images.

Oneliner to output all possible stats:

inf="eth0"
for i in h d m t s hs vs; do
  vnstati -i $inf -${i} -o vnstati-$i.png
done
BATS
April 3, 2020 — 12:31

Author: silver  Category: dev linux  Comments: Off

Bash Automated Testing System

BATS is a framework for unit testing Bash scripts. The latest version can be found here: https://github.com/bats-core/bats-core

Testing will make sure changes to your script do not break stuff but you wont have to do this by hand every time, instead using BATS to automate it for you.

Bats test files have the ".bats" extension, and can be run like this: bats example.bats.

Libraries

There’s two extra repos you’ll also want to check out and load at the start of your tests:

Example

An example test case might look like this:

#!/usr/bin/env bats

load 'bats-support/load'
load 'bats-assert-1/load'

@test "$(date '+%H:%M:%S') test help" {
  run your_script.sh -h
  [ "$status" -eq 0 ]
  assert_output --partial "USAGE:"
}

@test "$(date '+%H:%M:%S') test invalid argument" {
  run your_script.sh -invalid
  [ "$status" -eq 1 ]
  assert_output --partial 'Error: invalid or unknown arg(s)'
}

We’ll first display the time and some text, then test the output "your_script.sh" by running it.

The first case will pass if your_script.sh -h outputs the text "USAGE:". There can also be other text output before and after since we assert a partial match.

The second case checks what the script would output on an invalid error and compares it to "Error: invalid or unknown arg(s)". If it’s the same, the test will pass.

More testing

If you need more complicated testing there’s also functions and variables. Two default functions are setup() and teardown() to set tests up.

A good way to run tests is to be able to call the functions inside your script directly, so you probably want to consider this beforehand.

Alternatively there’s also other frameworks available:

ClamAV
December 16, 2019 — 15:14

Author: silver  Category: linux  Comments: Off

ClamAV is a decent anti virus scanner for Linux. Unfortunately it does not run every well on low memory systems (<1GB).

Running it’s database update tool freshclam can cause OOM. You will notice this if getting daily cdiff’s keeps failing (see ‘dmesg’ and /var/log/clamdb). These are db differences only instead of full files. The problem is processing these to create whole cvd’s.

If there’s close to enough RAM you could try using cgroups (or systemd) – if thats available, or good old ulimit:

Edit /etc/cron.d/clamav-freshclam and replace whats there with:

29 */1 * * *    clamav [ -x /usr/bin/freshclam ] && { ulimit -Sm 512000; ulimit -Sv 512000; ulimit -Hm 1024000; ulimit -Hv 1024000; /usr/bin/freshclam --quiet; } > /dev/null

But what if you’re on an embedded system or small vps and there’s not even close to 1GB memory available?

Simple, just get the full cvd files instead:

29 */1 * * *     clamav { for i in bytecode.cvd daily.cvd main.cvd; do wget -N -q "http://db.local.clamav.net/$i" -O /var/lib/clamav/$i; done; }  > /dev/null

Password Managers
December 11, 2019 — 15:34

Author: silver  Category: encryption linux windows  Comments: Off

There are basically 3 different categories to choose from, depending on location of service and db: Local, "Cloud"/SaaS or selfhosted On-Premise.

For single user/home usage KeePass is fine or perhaps even the password manager included in web browsers. Using one of the SaaS options such as LastPass adds ease of access.

For company/enterprise usage sharing passwords in groups/teams should be supported and preferably an on-prem option.

nftables
December 11, 2019 — 14:32

Author: silver  Category: linux network  Comments: Off

nftables (nft) replaces iptables:

  • Debian (10 buster) links ‘iptables’ to ‘iptables-nft’ and ‘iptables-legacy’ is actually ‘iptables’
  • RH uses nft as as preferred firewall since RHEL8 and firewalld uses nft as backend

If you haven’t switched yet you might want to ‘translate’ your current iptables rules and make other programs use nft.

config

rules are located in:

  • Debian /etc/nftables.conf
  • RedHat /etc/sysconfig/nftables.conf

list

nft list ruleset

nft list chain ip filter INPUT

nft list tables nft list table ip filter

flush

nft flush ruleset

translate

iptables-restore-translate -f /etc/iptables/rules.v4 > /etc/iptables/ruleset.nft

ip6tables-restore-translate -f /etc/iptables/rules.v6 > /etc/iptables/ruleset6.nft

netfilter-persistent

Oddly enough the only place I could find a nft plugin was here

curl https://raw.githubusercontent.com/hardenedlinux/harbian-audit/master/docs/configurations/usr.share.netfilter-persistent.plugins.d.15-nft -o /usr/share/netfilter-persistent/plugins.d/15-nft

fail2ban

Make f2b use nft. From https://wiki.meurisse.org/wiki/Fail2Ban:

  • edit ‘/etc/fail2ban/jail.local.conf’: banaction = nftables-multiport

  • add to ‘/etc/nftables.conf’: include "/etc/fail2ban.conf"

  • create ‘/etc/fail2ban.conf’:

#!/usr/sbin/nft -f

# Use ip as fail2ban doesn't support ipv6 yet
table ip fail2ban {
        chain input {
                # Assign a high priority to reject as fast as possible and avoid more complex rule evaluation
                type filter hook input priority 100;
        }
}

mergerfs
September 6, 2019 — 18:36

Author: silver  Category: linux storage  Comments: Off

Union filesystem (FUSE) like unionfs, aufs and mhddfs. Merge multiple paths and mount them, similar to concatenating.

Get it here: https://github.com/trapexit/mergerfs or from OS package repository.

Compared to (older) alternatives mergerfs seems very stable over the past months I’ve been using it. It offers multiple options on how to spread the data over the used drives.

Optionally SnapRAID can be used to add parity disk(s) to protect against disk failures (https://www.snapraid.it).

Create/mount pool

Example using 5 devices /dev/sd[b-f]

Disks are already partitioned and have a fs

for i in {b..f}; do
  mkdir /mnt/sd${i}1
  mount /dev/sd${i}1 /mnt/sd${i}1 && \
  mkdir /mnt/sd${i}1/mfs
done && \
mkdir /mnt/mergerfs && \
mergerfs -o defaults,allow_other,use_ino /mnt/sd*/mfs /mnt/mergerfs

And here’s the result from ‘df’:

/dev/mapper/sdb1             3.6T  100M  3.5T  1% /mnt/sdb1
/dev/mapper/sdc1             3.6T  100M  3.5T  1% /mnt/sdc1
/dev/mapper/sdd1             3.6T  100M  3.5T  1% /mnt/sdd1
/dev/mapper/sde1             3.6T  100M  3.5T  1% /mnt/sde1
/dev/mapper/sdf1             3.6T  100M  3.5T  1% /mnt/sdf1
mergerfs                      18T  500M  8.5T  1% /mnt/mergerfs

Changing pool

remove old drive from mergerfs pool

xattr -w user.mergerfs.srcmounts -/mnt/data1 /mnt/pool/.mergerfs

add new drive

xattr -w user.mergerfs.srcmounts +/mnt/data4 /mnt/pool/.mergerfs

some other mount options (-o)

  • use_ino make mergerfs supply inodes
  • fsname=example-name name in df
  • no_splice_write fixes page errors in syslog

https://github.com/trapexit/mergerfs#mount-options

Pool info

xattr -l /mnt/mergerfs/.mergerfs

Tools

https://github.com/trapexit/mergerfs-tools

  • mergerfs.balance
  • mergerfs.consolidate
  • mergerfs.ctl
  • mergerfs.dedup
  • mergerfs.dup
  • mergerfs.fsck
  • mergerfs.mktrash

mergerfs.ctl

mergerfs.ctl -m /mnt/mergerfs info
mergerfs.ctl -m /mnt/mergerfs list values
mergerfs.ctl -m /mnt/mergerfs remove path /mnt/data1
mergerfs.ctl -m /mnt/mergerfs add path /mnt/data4

PowerShell
July 9, 2019 — 9:17

Author: silver  Category: dev linux windows  Comments: Off

I’ve been using PS for a while now and I don’t hate it anymore :) In fact I think it’s very usable for lots of tasks and automation.

Some Useful commands:

  • Get-Command *help* or Get-Command-Module PackageManagement
  • Get-Member to view properties e.g. Get-Disk | Get-Member
  • Get-Alias
  • Get-ExecutionPolicy -List
  • Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
  • piping to select, sort and where
  • Invoke-WebRequest $url

CSV, XML and JSON support is included:

  • Import-CSV Export-CSV
  • ConvertTo-XML
  • ConvertFrom-Json ConverTO-Json

And stuff like:

  • Logging sessions: Start-Transcript Stop-Transcript
  • Viewing Certificates: cd Cert:\ (now you can ‘dir’ etc)
  • Run as admin: powershell.exe -Command "Start-Process cmd -Verb RunAs"
  • PS Linting: https://github.com/PowerShell/PSScriptAnalyzer

Remote usage is also possible over WinRM (or OpenSSH):

  • Enter-PSSession -ComputerName <host>

Then there’s Loops, Params, Arrays and Hash Tables e.g. foreach, Param([string]$arg), @() and @{}

More info:

Cgroups and NS
May 30, 2019 — 21:18

Author: silver  Category: linux  Comments: Off

Linux Control Groups and Namespaces

Used for limiting and isolation

Docs

Utils

  • lsns
  • nsenter
  • cgroup-tools pkg (cgget, cgset, …)

Network

  • ip netns list
  • ip netns identify <pid>
  • ip netns exec <netns> ip
  • or: ip -n|-netns

Processes

  • ps axwww -o cgroup
  • ps axwww -o cgroup,user,pid,%cpu,%mem,vsz,rss,tname,stat,start,time,comm
  • ps axwww -o ipcns,mntns,netns,pidns,userns,utsns,pid,comm

Filesystem

  • /proc/<pid>/ns
  • /sys/fs/cgroup

Systemd

Vim linting
May 30, 2019 — 20:50

Author: silver  Category: dev linux  Comments: Off

Linting is basically making sure source code is correct.

For Vim there’s ALE: Asynchronous Lint Engine. It supports multiple tools like cpplint for C/C++, ShellCheck for shell scripts, phan for PHP etc etc.

Download

Get it here: https://github.com/w0rp/ale

Commands

  • ALELint
  • ALEEnable
  • ALEDisable
  • ALENext
  • ALEPrevious

.vimrc

To use Ctrl+j and Ctrl+k to moving between errors:

nmap <silent> <C-k> <Plug>(ale_previous_wrap)
nmap <silent> <C-j> <Plug>(ale_next_wrap)
Zonemaster
December 8, 2018 — 17:52

Author: silver  Category: linux  Comments: Off

Zonemaster is an Open source DNS validation tool

Source: https://github.com/zonemaster/zonemaster
Hosted: https://www.zonemaster.net/domain_check

Install Perl modules

Dependencies:

cpanm File::ShareDir cpanm File::Slurp Hash::Merge IO::Socket::INET6 List::MoreUtils Mail::RFC822::Address Module::Find Moose Net::IP Readonly::XS Text::CSV Devel::CheckLib

Zonemaster LDNS and Engine:

cpanm Zonemaster::LDNS
cpanm Zonemaster::Engine

Test

time perl -MZonemaster::Engine -e &#039;print map {&quot;$_\n&quot;} Zonemaster::Engine-&gt;test_module(&quot;BASIC&quot;, &quot;zonemaster.net&quot;)&#039;

Install Perl modules

Dependencies:

cpanm MooseX::Getopt Text::Reflow Module::Install

Zonemaster CLI:

cpanm Zonemaster::CLI

Examples

zonemaster-cli --test basic zonemaster.net
zonemaster-cli --no-ipv6 --show_level --show_module --progress --level INFO --test Syntax example.com
GNU find
March 30, 2018 — 14:51

Author: silver  Category: linux windows  Comments: Off

Just a few useful ‘find’ examples

Exclude:

find . -path ./foo -prune -o -name bar
find /home \( -path /usr/data -prune -o -path /usr/src \) -prune -o -name foo -print
find . -name Makefile -not -path foo
find . -type d ! -regex .*\/\(foo\|bar\).* \;

Permissions:

find . -perm -775
find . -perm /u+w,g+
find . -printf "%m:%f\n"
find . -printf "%m %h/%f\n"|grep -v '^\(644\|755\)'

Print date:

find -type f -printf '%TF %.8TT %p\n'

Windows:

find.exe . -name *.exe -exec certutil -hashfile {} SHA512 ; >c:\hash.txt

Updating CPU Microcode
March 28, 2018 — 12:50

Author: silver  Category: hardware linux windows  Comments: Off

BITS

Tool from Intel called “BIOS Implementation Test Suite”. Bootable (usb) image which can do several things including handling microcode:


Linux

load/update microcode using pkg:

Debian

  • Intel: apt install intel-microcode iucode-tool
  • AMD: apt install amd64-microcode
  • Doc: /usr/share/doc/{intel,amd64}-microcode/README.Debian.gz

CentOS/RH

load/update intel microcode manually:

  • get latest tgz from intel: see below
  • backup/copy files: /lib/firmware/intel-ucode
  • check kernel config: grep MICROCODE /boot/config-*
  • run iucode_tool:
    /usr/sbin/iucode_tool -tb -lS /lib/firmware/intel-ucode/*
  • update initramfs: update-initramfs -u -k all

reloading microcode:

  • echo 1 > /sys/devices/system/cpu/microcode/reload
  • or: rmmod cpuid; modprobe cpuid

show version:

  • dmesg | grep microcode
  • grep microcode /proc/cpuinfo

skip loading microcode on boot:

  • add to grub cmdline: dis_ucode_ldr


Windows

Microsoft includes microcode updates in Windows for certain CPU’s. For example: KB4090007, KB3064209, KB2970215.

load/update microcode:

show version:

  • get hwinfo64 and goto “Central Processor(s)” > “Microcode Update Revision”
  • or get “Read & Write Everything” (RWEverything) from http://rweverything.com


Get Microcode

Download the latest version from Intel:
https://downloadcenter.intel.com/download/27431/Linux-Processor-Microcode-Data-File?v=t.

Magic SysRq Key
November 25, 2017 — 22:19

Author: silver  Category: linux  Comments: Off

How to use SysRq (Print Screen key)

( “REISUB” )

enable:

echo 1 > proc/sys/kernel/sysrq

permanently:

/etc/sysctl.d/local.conf
kernel.sysrq=1

To BREAK: CTRL+PAUSE (Serial)

ALT+SysReq+KEY
When logged in using SSH the SysRq may be accessible by writing to /proc/sysrq-trigger
echo s > /proc/sysrq-trigger

useful options:

  • b: Immediately reboot the system, without unmounting or syncing filesystems
    echo b > proc/sysrq-trigger
  • e: Send the SIGTERM signal to all processes except init (PID 1)
  • f: Call oom_kill, which kills a process to alleviate an OOM condition:
  • s: Sync all mounted filesystems:
  • t: Output a list of current tasks and their information to the console:
  • u: Remount all mounted filesystems in read-only mode
  • w: Display list of blocked (D state) tasks
  • space: Print a summary of available magic SysRq keys

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/admin-guide/sysrq.rst
https://en.wikipedia.org/wiki/Magic_SysRq_key

GoAccess
November 25, 2017 — 18:04

Author: silver  Category: linux web  Comments: Off

GoAccess is a “real-time web log analyzer” which can output in CLI or HTML (like webalizer, awstats and piwik etc). It works out of the box with Apache, for lighttpd you probably need to specify the log format. Examples below are for lighttpd. Run “goaccess /var/log/httpd/access.log” without any other arguments and it will ask for the log format and drop you into the Dashboard (text based gui).

CLI

no conf, just arguments:

goaccess /var/log/lighttpd/access.log \
--date-format=%d/%b/%Y \
--time-format='%T %z' \
--log-format='%h %v %e [%d:%t] "%r" %s %b "%R" "%u"'

-or-

change /etc/goaccess.conf:

date-format %d/%b/%Y:%T %z
log-format %h %v %e [%d] "%r" %s %b "%R" "%u"

HTML

Output to “static” html file.

current log:

goaccess /var/log/lighttpd/access.log \
  --date-format=%d/%b/%Y \
  --time-format='%T %z' \
  --log-format='%h %v %e [%d:%t] "%r" %s %b "%R" "%u"' \
  --output=/var/www/html/goaccess.html

use all logs:

zcat -f /var/log/lighttpd/access.log*gz | goaccess \
  --date-format=%d/%b/%Y \
  --time-format='%T %z' \
  --log-format='%h %v %e [%d:%t] "%r" %s %b "%R" "%u"' \
  --ignore-crawlers \
  --with-output-resolver \
  -e 127.0.0.1 -e ::1 -e exclude.example.com
  --output=/var/www/html/goaccess.html

Server

The last option is to run it as Server using WebSocket. This allows it to:

  • output realtime HTML: --real-time-html
  • run as daemon: --daemonize
  • use FIFO: --fifo-in= --fifo-out=
  • use HTTPS: --ssl-cert= --ssl-key= --ws-url=wss://url

live log:

goaccess /var/log/lighttpd/access.log \
 --date-format=%d/%b/%Y \
 --time-format='%T %z' \
 --log-format='%h %v %e [%d:%t] "%r" %s %b "%R" "%u"' \
 --output=/var/www/html/goaccess.html \
 --real-time-html \
 --ssl-cert=//etc/ssl/certs/cert.pem \
 --ssl-key=/etc/ssl/private/privkey.pem --ws-url=wss://example.com:7890

Now https://example.com/goaccess.html should should a live Dashboard (tcp port 7890 needs to be open for client).

perf
August 23, 2017 — 16:45

Author: silver  Category: linux  Comments: Off

perf – performance analysis tools for Linux

Start with:

perf top
perf bench all

Example:

To find out why “kworker” process (kernel per-cpu threads) has high CPU usage:

  • record 10 seconds of backtraces on all CPUs to perf.data:
    perf record -g -a sleep 10
  • analyse recording:
    perf report

More info:
https://www.brendangregg.com/perf.html
https://askubuntu.com/questions/33640/kworker-what-is-it-and-why-is-it-hogging-so-much-cpu

Debian
August 23, 2017 — 15:37

Author: silver  Category: linux  Comments: Off

Remote upgrade using aptitude:

  1. echo “defscrollback 10000” >>/root/.screenrc
  2. screen
  3. /etc/sysctl.conf:
    # on kernel panic reboot after 60s
    kernel.panic = 600
    # enable magic sysrq key
    kernel.sysrq=1</pre>
  4. In /etc/apt/sources.list: change old to new dist (or “stable” etc)
    ( if needed: apt-get install debian-archive-keyring )
  5. aptitude update
  6. aptitude safe-upgrade
    ( optionally/if needed: full-upgrade, dist-upgrade )

Change default editor:

sudo update-alternatives --config editor

Install build tools:

apt-get install build-essential

pkg install dates:

for file_list in `ls -rt /var/lib/dpkg/info/*.list`; do \
  stat_result=$(stat --format=%y &quot;$file_list&quot;); \
  printf &quot;%-50s %s\n&quot; $(basename $file_list .list) &quot;$stat_result&quot;; \
done

backports:

apt-get -t stretch-backports install “package”
aptitude -t stretch-backports install “package”

/etc/apt/preferences:

Package: *
Pin: release a=stable
Pin-Priority: 900

Package: *
Pin: release o=Debian
Pin-Priority: -10

“testing” packages:

install a pkg from testing:
sudo apt-get -t testing install tmux
show all testing pkgs:
aptitude search -F "%p %V %v" '?narrow(~i, ~Atesting)
( stable, unstable, oldstable, etc )

apt-get install package=version

ping
March 4, 2017 — 16:01

Author: silver  Category: linux  Comments: Off

When trying to ping as non root user you might get the following error:

ping: icmp open socket: Operation not permitted

There are several ways to fix this:

reinstall pkg (debian):

$ sudo apt-get install --reinstall iputils-ping

(sets cap)

manually set cap:

$ sudo setcap cap_net_raw+ep /bin/ping
$ sudo setcap cap_net_raw+ep /bin/ping6
$ sudo getcap /bin/ping
$ sudo getcap /bin/ping6

needs kernel config:
CONFIG_EXT4_FS_SECURITY=y


dont use SOCK_RAW:

socket(PF_INET, SOCK_DGRAM, PROT_ICMP)

$ cat /proc/sys/net/ipv4/ping_group_range
$ sysctl net.ipv4.ping_group_range
  • “1 0” default, nobody except root
  • “100 100” single group
  • “0 2147483647” everyone (max gid)
$ sysctl net.ipv4.ping_group_range = "0 2147483647"


/etc/sysctl.d/local.conf
net.ipv4.ping_group_range=0 2147483647


suid:

chmod +s /usr/ping
chmod +s /usr/ping6
Linux Audit
March 4, 2017 — 15:43

Author: silver  Category: linux  Comments: Off

First make sure “auditd” is started

add rules:

auditctl -a always,exit -S all -F path=/etc/passwd -F key=config1
auditctl -w /etc/passwd -p rwa -k config2

del rules:

auditctl -d always,exit -S all -F path=/etc/passwd -F key=config1
auditctl -W /etc/passwd -p rwa -k config2

(or restart auditd)

make permanent:

add rules to /etc/audit/rules.d/audit.rules

show results:

ausearch -ts today -k config1
aureport -k

disable audit logs:

systemctl mask systemd-journald-audit.socket
HP ProLiant
December 9, 2016 — 21:54

Author: silver  Category: hardware linux  Comments: Off

Boot:

  • BIOS: F10
  • HP SSA Smart Storage Administrator / ACU Array Configuration Utility: F5
  • ORCA / Options ROM for Configuring Arrays: Press any key…, F8
  • HP IP: F10
  • Boot Menu: F11

Install HP software:

Repository:

wget http://downloads.linux.hp.com/add_repo.sh
sh add_repo.sh spp -d redhat -r 6.7 -n
sh add_repo.sh spp -d redhat -r 6.7
sh add_repo.sh spp -d redhat -r 5.10 -n
sh add_repo.sh spp -d redhat -r 5.10
sed -i 's/gpgcheck=0/gpgcheck=1/' /etc/yum.repos.d/HP-spp.repo
rpm --import http://downloads.linux.hp.com/SDR/hpPublicKey1024.pub
rpm --import http://downloads.linux.hp.com/SDR/hpPublicKey2048.pub
rpm --import http://downloads.linux.hp.com/SDR/hpPublicKey2048_key1.pub
for i in $( rpm -qa gpg-pubkey* ); do rpm -qi $i |grep -B 8 Hewlett; done
yum install hpacucli
yum install hponcfg

HP Server Management Application and Agents Command Line Interface

# hpasmcli -s "clear iml"

HP Lights-Out Online Configuration Utility for Linux

hponcfg -f Clear_EventLog.xml -i

Clear_EventLog.xml:

<RIBCL VERSION="2.0">
 <LOGIN USER_LOGIN="Administrator" PASSWORD="xxx">
 <RIB_INFO MODE="write">
 <CLEAR_EVENTLOG/>
 </RIB_INFO>
 </LOGIN>
</RIBCL>

Clear_IML.xml:

<RIBCL VERSION="2.0">
  <LOGIN USER_LOGIN="Administrator" PASSWORD="xxx">
  <SERVER_INFO MODE="write">
    <CLEAR_IML/>
  </SERVER_INFO>
  </LOGIN>
</RIBCL>

Administrator_reset_pw.xml:

<ribcl VERSION="2.0">
 <login USER_LOGIN="Administrator" PASSWORD="boguspassword">
  <user_INFO MODE="write">
   <mod_USER USER_LOGIN="Administrator">
    <password value="NewPass123"/>
   </mod_USER>
  </user_INFO>
 </login>
</ribcl>
gnome-keyring
November 26, 2016 — 17:52

Author: silver  Category: linux  Comments: Off

Quickfix issues

Restart:

gnome-keyring-daemon -r -d

If that doesn’t suffice, this extra steps might help:

pgrep -f gnome-keyring-daemon
rm -rf ~/.cache/keyring-*
setsid /usr/bin/gnome-keyring-daemon /dev/null 2>&1
ln -s ~/.cache/keyring-* $GNOME_KEYRING_CONTROLA
/usr/bin/gnome-keyring-daemon --start --components=pkcs11
/usr/bin/gnome-keyring-daemon --start --components=gpg
/usr/bin/gnome-keyring-daemon --start --components=ssh
find ~/.cache/ -maxdepth 1 -type l -name 'keyring-*' -delete







We use Matomo free and open source web analytics
We also use Jetpack WordPress.com Stats which honors DNT