BATS
April 3, 2020 — 12:31

Author: silver  Category: dev linux  Comments: Off

Bash Automated Testing System

BATS is a framework for unit testing Bash scripts. The latest version can be found here: https://github.com/bats-core/bats-core

Testing will make sure changes to your script do not break stuff but you wont have to do this by hand every time, instead using BATS to automate it for you.

Bats test files have the ".bats" extension, and can be run like this: bats example.bats.

Libraries

There’s two extra repos you’ll also want to check out and load at the start of your tests:

Example

An example test case might look like this:

#!/usr/bin/env bats

load 'bats-support/load'
load 'bats-assert-1/load'

@test "$(date '+%H:%M:%S') test help" {
  run your_script.sh -h
  [ "$status" -eq 0 ]
  assert_output --partial "USAGE:"
}

@test "$(date '+%H:%M:%S') test invalid argument" {
  run your_script.sh -invalid
  [ "$status" -eq 1 ]
  assert_output --partial 'Error: invalid or unknown arg(s)'
}

We’ll first display the time and some text, then test the output "your_script.sh" by running it.

The first case will pass if your_script.sh -h outputs the text "USAGE:". There can also be other text output before and after since we assert a partial match.

The second case checks what the script would output on an invalid error and compares is it to "Error: invalid or unknown arg(s)". If it’s the same, the test pass will pass.

More testing

If you need more complicated testing there’s also functions and variables. Two default functions are setup() and teardown() to set tests up.

A good way to run tests is to be able to call the functions inside your script directly, so you probably want to consider this beforehand.

Alternatively there’s also other frameworks available:

Scoop
February 14, 2020 — 14:49

Author: silver  Category: dev windows  Comments: Off

Besides Chocolatey there’s also https://scoop.sh. A more "dev" oriented package manager for Windows PowerShell.

It installs "UNIXy" packages such as Git, cURL and vim. Languages such as Perl, PHP Python and Go are available just like Apache, Nginx and MySQL. Packages are installed into your homedir/profile, no UAC needed or changing PATH env var.

PowerShell
July 9, 2019 — 9:17

Author: silver  Category: dev linux windows  Comments: Off

I’ve been using PS for a while now and I don’t hate it anymore :) In fact I think it’s very usable for lots of tasks and automation.

Some Useful commands:

  • Get-Command *help* or Get-Command-Module PackageManagement
  • Get-Member to view properties e.g. Get-Disk | Get-Member
  • Get-Alias
  • Get-ExecutionPolicy -List
  • Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
  • piping to select, sort and where
  • Invoke-WebRequest $url

CSV, XML and JSON support is included:

  • Import-CSV Export-CSV
  • ConvertTo-XML
  • ConvertFrom-Json ConverTO-Json

And stuff like:

  • Logging sessions: Start-Transcript Stop-Transcript
  • Viewing Certificates: cd Cert:\ (now you can ‘dir’ etc)
  • Run as admin: powershell.exe -Command "Start-Process cmd -Verb RunAs"
  • PS Linting: https://github.com/PowerShell/PSScriptAnalyzer

Remote usage is also possible over WinRM (or OpenSSH):

  • Enter-PSSession -ComputerName <host>

Then there’s Loops, Params, Arrays and Hash Tables e.g. foreach, Param([string]$arg), @() and @{}

More info:

Vim linting
May 30, 2019 — 20:50

Author: silver  Category: dev linux  Comments: Off

Linting is basically making sure source code is correct.

For Vim there’s ALE: Asynchronous Lint Engine. It supports multiple tools like cpplint for C/C++, ShellCheck for shell scripts, phan for PHP etc etc.

Download

Get it here: https://github.com/w0rp/ale

Commands

  • ALELint
  • ALEEnable
  • ALEDisable
  • ALENext
  • ALEPrevious

.vimrc

To use Ctrl+j and Ctrl+k to moving between errors:

nmap <silent> <C-k> <Plug>(ale_previous_wrap)
nmap <silent> <C-j> <Plug>(ale_next_wrap)
C
December 8, 2018 — 18:52

Author: silver  Category: dev  Comments: Off

Style:

1TBS of course ;)

Vi:

set: :set noai tabstop=8 shiftwidth=8 softtabstop=8 noexpandtab
modeline: /* vim: set noai tabstop=8 shiftwidth=8 softtabstop=8 noexpandtab: */
 

Strings:

char greeting[6] = {'H', 'e', 'l', 'l', 'o', '\0'};
char str[12] = "Hello World";
char str[0] = '\0';
char buf[];

copy: strcpy(str2, str1)
concat: strcat(str1, str2)
length: strlen(str)
duplicate: ptr = strdup(str)
substring: strstr(str1, str2)
extract: strsep(*strptr, delimstr)

strstr():

  • locate subscring
  • or: “check if ‘sent’ contains ‘word'”
    if(strstr(sent, word) != NULL) //..or skip NULL check

strsep():

  • extract token
  • or: “split substring using delim”
  • like deprecated strtok()
    while((found = strsep(&string,",")) != NULL)

NOTES:

  • set it before using it (initialize)
  • strings are char arrays terminated by \0
  • strings in double quotes are auto terminated by \0
  • strcpy : watch out for buffer overruns with long strings
  • use : checks, char buf[1024], \0 term, malloc(strlen(str) + xx)
  • buf[] has no size and no space allocated
  • convert int to string: itoa, sprintf, snprintf(buf, 10, "%d", num);

tutorialspoint.com/cprogramming/c_strings.htm

 

Print:

printf outputs to stdout stream
fprintf goes to a file handle FILE*
sprintf goes to a buffer you allocated (char*)

printf("This is a str: %s\n", str);
fprintf(fh, "log: %s", str);
sprintf(buf, "%s %s", str1, str2);

int: %d
long int: %ld
long long int: %lld
unsigned long long int: %llu

  • using *ptr is actually the first element (not its address),
  • “%s” format specifier for printf always expects a char* argument

 

Pointers:

// ptr is going to store address of integer value
int i = 20;
int *ptr;
                Operator        Operator Name           Purpose
*ptr            *               Value at Operator       gives Value stored at particular address
type* ptr;                      Same                    Same
&ptr            &               Address Operator        gives Address of Variable
**ptr                           Double Pointer          declares a Pointer to a Pointer
  • while int *var is the same as int* var, *var is better/more clear to use
  • initialize (NULL or valid address)

 

Extern:

  • tell compiler variable is declared elsewhere
  • use only in one place

example.h:

extern int global_variable;

example.c:

#include "example.h"
int global_variable = 1337;

example2.c:

#include "example.h"
void use_it(void)
{
    printf("Global variable: %d\n", global_variable++);
}

 

Compiler messages:

warning: implicit declaration of function 'function' [-Wimplicit-function-declaration]

You are using a function for which the compiler has not seen a declaration (“prototype”) yet.

int main()
{
    fun(2, "21"); /* The compiler has not seen the declaration. */
    return 0;
}

int fun(int x, char *p)
{
    /* ... */
}

You need to declare your function before main, like this, either directly or in a header:

int fun(int x, char *p);

From stackoverflow.com/a/8440833

 

Macro:

ifdef : “if defined” (quelle surprise;)

example.c:

#ifdef DEBUG
 printf("DEBUG: %s\n", dbgstuff);
#endif

gcc -DDEBUG -c

gcc.gnu.org/onlinedocs/cpp/Ifdef.html

 

Include guard:

ifndef : if NOT defined

#ifndef checks whether the given token has been #defined earlier in the file or in an included file; if not, it includes the code between it and the closing #else or, if no #else is present, #endif statement. #ifndef is often used to make header files idempotent by defining a token once the file has been included and checking that the token was not set at the top of that file.

example.h:

#ifndef _INCL_GUARD
#define _INCL_GUARD
#endif

cprogramming.com/reference/preprocessor/ifndef.html

Malloc

char *buf = (char *) malloc(bufferSize);

where bufferSize is the runtime result of some computation

Examples:

char *string = (char*)malloc(n+1 * sizeof(char));
size_t length = strlen(str1) + strlen(str2) + 1;
char *concat = malloc(sizeof(char) * length);
char* buffer = (char*)malloc(256);
rhost = (char *)malloc(sizeof(char)*128);
  • if allocatable buffer space is variable, use malloc instead of buf[1024]
  • when done use free: free(buf);
  • strdupe already does malloc, no malloc needed

Execute command:

system("ls -la")

get output: popen()

FILE *fp = popen("ls -la", "r");
if (fp) 
	fscanf(fp, "%100s", var);

-or-

	fgets(var, 100, fp);

Return:

return 0 or 1:

int function {
	if (success) {	return 0; }
	return 1;
}

return char* value:

char* function {
	char *buf;
	...
	return buf;
}

(use malloc or buf[1024] and free)

Examples:

str functions (example):

printf ("\nDEBUG: strncmp %s %i\n", flist_getfilename(ftmp), strncmp(flist_getfilename(ftmp), ".mp3", 4));
printf ("\nDEBUG: strcmp %s %i\n", flist_getfilename(ftmp), strcmp(flist_getfilename(ftmp), ".mp3"));
printf ("\nDEBUG: strcasecmp %s %d\n", flist_getfilename(ftmp), strcasecmp(flist_getfilename(ftmp), ".mp3"));

more strcmp:

if (!strncmp(flist_getfilename(ftmp), ".mp3", 4))
if (strncmp(flist_getfilename(ftmp), ".mp3", 4) != 4 )
Python 3
March 30, 2018 — 14:26

Author: silver  Category: dev  Comments: Off

Print modules:

python3 -c "import sys;print (sys.path)"
python3 -c "import sys;print (sys.modules)"
python3 -c "help('modules')"
python3 -c "help('modules package')"
pydoc3 modules

Packages:

  • Location in Debian: /usr/lib/python{3}/dist-packages/pkgname/{__init__.py}
  • Other distro’s use “site-packages”

pip3 install pkgname
pip3 uninstall pkgname
pip3 freeze
python3 -m pip install pkgname

python3
import pip
sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])

Virtualenv:

$ pip3 install virtualenv
$ mkdir workdir && cd workdir
$ virtualenv project
$ source project/bin/activate
$ pip3 install requests

(and/or git clone + python3 setup.py install / etc)

$ deactivate
$ rm -rf project

Notes:

  • make sure your script.py does not have the same name as package
  • print syntax changed from python2 to 3: print ("foo")
Perl
March 30, 2018 — 9:50

Author: silver  Category: dev  Comments: Off

List installed modules:

Pick one :)

  • perl -V
  • perldoc <module name>
  • perldoc perllocal
  • instmodsh
  • cpan -[l,a]

or:

perl -e 'use ExtUtils::Installed; foreach (ExtUtils::Installed->new()->modules()) {print $_ . "\n";}'

Install modules in homedir:

  • install “local::lib” module
  • print settings: perl -Mlocal::lib
  • add to .bashrc: eval "$(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)"

Then there’s also “Perlbrew“.








We use Matomo free and open source web analytics (opt-out)