05/19/22 Git hook to make commit messages conform to pattern

#!/bin/bash

MSG="$1"

if ! grep -qE "GIT-" "$MSG";then
    echo "Your commit message must contain a ticket number, ie 'GIT-XXXXX'"
    echo "If this is an emergency, or an experiment etc, you can bypass this check via git commit --no-verify"
    exit 1
fi

into .git/hooks/commit-msg. Can bypass with --no-verify flag.

05/10/22 Find a deleted file in git history

git log --all --full-history -- "**/thefile.*"

or:

githistory() {
    local FILE="$1"
    shift 1
    git log --all --full-history -- "**/$FILE.*"
}

04/08/22 Test single cucumber test suite

Add tag @local (or whatever), and add to command line --tags @local.

04/03/22 I keep forgetting this term for a data access object

DAO. Not an ORM. This is a raw sql query turned into a function to pull data directly out of the db.

03/25/22 Loop over a list of things, or some tuples

for i in a,b c,d e,f;
do IFS=","; set -- $i;
  doThings $1 $2; # $1 and $2 will give a and b, then c and d, etc.
done

03/24/22 Write output of command to file

command >> file.txt

01/19/22 Just some basic enuming in shell

THING=$1
case "${THING}" in
  apple) PORT=1234 ;;
  orange) PORT=1235 ;;
  pear) PORT=1236 ;;
  *) echo "Must pass a thing name: apple|orange|pear" 1>&2 && exit 1 # Puts stdout to stderr, not sure if `exit 1` is even really needed here
esac

01/18/22 Check for args in shell

myfunction() {
  if [ $# -lt 2 ]; then
    echo "Need 2 arguments"
    return
  fi
  # do thing
}

01/17/22 Ripgrep filenames

ripgrep can show all file names, so you need to pipe it back into rg to search it:

alias rgf="rg --files | rg"

01/16/22 Open html files in browser when using Ubuntu in Windows subsystem linux

# in your favourite rc file
alias see='explorer.exe'

# Then
see index.html

Note this is Windows Explorer, not Internet Explorer. If you set your default browser in your settings under Defautl Apps it will then open with the appropriate browser for you.

Also note this supercedes the /usr/bin/see command... but it's not like you were using that command anyway.

Things I tried instead and failed:

  • Installing firefox in wsl
  • Opening with firefox in wsl
  • Trying the open command in wsl

01/12/21 Find all CRLF files

While I was debating on if I should lint our repos to have CRLF or LF (LF seems to be the standard going forward), I found this handy line to file files that are CRLF:

find . -not -type d -exec file "{}" ";" | grep CRLF

If you want it to count the lines, can add | wc -l to the end of that.

01/06/22 How to presumably open a Java project correctly in IntelliJ

  • File > New > Project from Existing Sources...
  • Select repo folder
  • Select Import project from external model > Select Maven > Finish

10/21/21 Write global Gitignores

While I appreciate VSCode over most other editors for most things, I don't appreciate that opening a Java project it'll create bin folders, file .classpath, .project and .settings automatically in the workspace. Since I work with many workspaces, I found the need for a global .gitignore:

~/.gitignore

.DS_Store
.vscode
.idea
*/bin
*.classpath
*.project
*.settings

git config --global core.excludesfile ~/.gitignore

Will make that file ignore all these patterns.

5/20/21 Check for BOM in file using Vim

:set bomb? will say either "bomb" or "nobomb" (true/false). This is the byte order mark at the beginning of a file which can mess up binary reading of files.

How to remove bom: https://wincent.com/wiki/Remove_BOM_marker_from_file_with_Vim

5/3/21 Check if git stash has any changes at any point when cding

function cd() {
  builtin cd [email protected] || return
  if [[ -d .git ]]; then
    if [[ $(git stash list) ]]; then
      echo "Stashed content:"
      git stash list
    fi
  fi
}

beautiful crontab translation

https://crontab.guru

1/13/21 Find if port is actively being listened on

lsof -i :13411

12/10/20 Glob shell patterns

* is match one or more characters, ? is match exactly one character, so:

ls *.??

Will list any files that end with a two character file extension. There is also matching a single character out of multiple choices by putting the choices in brackets:

> ls [mv]*.??
monday.md  vacuums.md

11/22/20 Copy to clipboard

cat result.txt | pbcopy

09/18/20 Delete remote branch

git push --delete origin my-branch

Or all merged branches, something like:

git branch -r --merged | egrep -v "(^\*|master*)" | sed -e 's:origin/::' | xargs -n 1 git push --delete origin

08/19/20 Groovy star-dot operator

The following is a map function as far as I can tell.

// Functionally equivalent:
myArr.each { item -> item.sendReport() }
myArr*.sendReport() // Returns a new array

assert [1, 2] = myArr*.sendReport // Works
aseert [1, 2] = myArr.each { item -> item.sendReport() } // Doesn't work, does not return an array

Groovy/Java abstract class definition

myClass {
  abstract Action poop

  def afraid() {
    Action actPoop = poop()
    exclaimActions([actPoop]) // Not sure if this is proper syntax for a list
  }
}

poop here is not implemented, and this class is now an abstract class. It will be implemented by something that extends the class, and at that time, it'll make the other function where it gets used make sense of it.

08/14/20 Transitive

Transitive dependencies are where all sub-dependencies are also dependencies of the root package.

If A -> B, B is not -> A, B -> C, then A -> C

The < operator is a transitive operator, for example.

07/24/20 Get all the environment variables

set

Find something in it

set | grep something

06/30/20 Exit shell with non-zero code

Just run this:

~.

When pressing the dot, it'll close the connection.

You might wonder why I would want to ever do this. I was doing this to test what would happen when an ssh tunnel is broken, so I wanted to break it early. If you Ctrl + D, it will break gracefully (zero exit code).

06/30/20 Tell me when an ssh connection breaks

ssh myurl || say tunnel broke

Since say is a Mac command to say the next words, and only on a non-zero code, the next line will be interpreted due to the or statement.

06/30/20 Keep a session alive until a command exits

This ties in nicely with above:

caffeine -i command
#e.g.
caffeine -i ssh myurl || say tunnel broke

06/23/20 One-liner Shell for-loop

for i in `seq 1 2 3 5 7 11`; do sudo ping 127.0.0.$i; done;

06/12/20 Git log show names of changed files

git log --name-only

05/29/20 SSH Stuff

To reuse a local key from a bastion:

First add forwarding your key to the bastion:

In your .ssh/config, add

ForwardAgent yes

On the bastion:

ssh-copy-id remote-id-like-to-send-it-to
# e.g.
# ssh-copy-id 23.0.4.255
# ssh-copy-id dev.mysite.com

This may prompt for a password. Once entered, you have trusted your local key.

05/21/20 (Shell) - Time a command

time command
...
command  1.22s user 0.63s system 18% cpu 9.766 total

It's that simple.

05/20/20 (SSH) - extend config file to include another config file

This is especially useful if you pull in a team config file. Add the following to your .ssh/config file:

Include <full filepath>

e.g.

Include ~/.ssh/shared_config

04/27/20 (Git) - Check what commits affect a specific file (or path)

git log path/my-file

04/26/20 (Launchctl) - Starting things with system startup

You need a plist file to load.

launchctl load -w /Library/LaunchDaemons/pinger.plist

You may need to check it for errors:

plutil /Library/LaunchDaemons/pinger.plist

But if you want to just load it and put it into the background, that works too:

./ping.py &

But if you want to allow your shell to close and it still runs, use nohup:

nohup ./ping.py &

04/22/20 (Git) - Checking the diff between two items that do not have a common repo

git diff --no-index -- ./itemone.txt /var/itemtwo.txt

04/09/20 (Python) - Add to system path

Python lambda is a bit annoying how the imports need to be in the root of a project. Due to this, we need to add that path to the system path if we want to run a single file that is referencing anything at that root level.

To add to your path, a convention I think works well enough for now is:

# .venv/lib/python3.6/site-packages/root.pth

/my/absolute/path

You'll need to source the virtual environment. This isn't great, due to not being a part of version control, but... oh well.

04/05/20 (Awk) - Get ping from an endpoint

Quite simple.

ping google.com | awk '{print substr($7,6)}'

04/04/20 (Python) - How to make a basic package to distribute

https://medium.com/packagr/creating-and-sharing-private-python-packages-151a95e10735

03/27/20 (Python) - List comprehension: check deep equality between two lists

The following does work:

a = [{'a':[1,2,3]},{'b':'b'}]
b = [{'a':[1,2,3]},{'b':'b'}]
a == b

However some cases won't work (I'm failing to figure out what cases those are currently...)

The following works:

all([a == b for a, b in zip(actual, expected)])

03/26/20 (AWS Boto3) - Difference between client, resource, and session

Reference: https://stackoverflow.com/questions/42809096/difference-in-boto3-between-resource-client-and-session

Main takeaways:

Resource is a high level API giving lots of nice tools. Try to use it if it's available.

Client is generally a 1 to 1 mapping of the API, so low level and might not look as nice in the code.

Session boto3 generally creates this for you when needed, you don't need to worry about it. Allows you to create clients and resources using stored configuration information.

03/26/20 (Ripgrep) Only return filename, not the lines that were discovered

This is sorta useful if you've a crazy long output stream for where the term is discovered.

rg word -l

03/25/20 (Unix) Run a process as another user

This is only possible from root. Either run it with sudo or in a root shell.

runuser -l myuser -c 'mycommand'

03/20/20 (Unix - scp) Allow scp to work when bashrc contains commands

Add this to the top of the rc file of the server to disable it from doing anything in this file if in a non-interactive state, such as using scp.

# If not running interactively, don't do anything. This is needed to use scp
# as scp does not work well with any output generated by the bashrc.
[[ $- == *i* ]] || return

03/17/20 (Security) Share files securely between Windows and Mac systems

Some stuff is taken from this guide.

This uses the id_rsa file that you probably already generated as a dev.

This file is a private key. If you're a mac user, it's probably already in pem format.

The receiver:

  • Confirm you have an RSA private key as your id_rsa file. If you don't, turn it into that format. This is an example format:
-----BEGIN RSA PRIVATE KEY-----
MIIEpQ...(20 lines long)
-----END RSA PRIVATE KEY-----
  • Create a public key from this:
openssl rsa -in id_rsa -pubout -outform pem > id_rsa.pub

It should have the format of a one-liner:

ssh-rsa AAAAB3K9dfdf... /LrNbk9 my@email.com
  • Share this key.

The sender:

  • Create a random string of characters, encrypt it using the public key:

(Replace NAME and SECRET_FILE with the user's public key.)

openssl rand -base64 32 > key.bin
 openssl rsautl -encrypt -inkey NAME.pub -pubin -in key.bin -out key.bin.enc
openssl enc -aes-256-cbc -salt -in SECRET_FILE -out SECRET_FILE.enc -pass file:./key.bin
  • Send both the key.bin.enc and the SECRET_FILE.enc to the user.

The receiver:

  • Decrypt it:
openssl rsautl -decrypt -inkey id_rsa -in key.bin.enc -out key.bin
openssl enc -d -aes-256-cbc -in SECRET_FILE.enc -out SECRET_FILE -pass file:./key.bin 

Note: id_rsa needs to be in pem format, so you may have created it as id_rsa.pem.

03/17/20 (Git) Git stash with message

Rename a stash. Ref: https://stackoverflow.com/questions/25931026/how-can-i-rename-a-git-stash

Let's say you have a stash at position zero:

$ git stash list
[email protected]{0}: WIP on BF-3311-buyback-seo: a0e41571e SEO - Add alt tag to buyback book images
$ git stash drop
Dropped [email protected]{0} (9333f11868bf4c240999cb8b88804bs7052c42a7)
$ git stash store -m 'My message' 9333f11868bf4c240999cb8b88804bs7052c42a7

In summary:

git stash drop stash@{n}
git stash store -m 'Message' <hash>

03/15/20 (Python - tips) Basic python timer

if __name__ == '__main__':
    import time
    tic = time.time()
    print(minHours(rows, columns, grid))
    toc = time.time()
    elapsed = (toc - tic)* 1000
    print(f'{elapsed}ms')

03/05/20 (Git - custom settings) Git set upstream automatically

This updates your push command to infer that origin is the default place to push to:

git config --global push.default current

02/28/20 (Unix - custom tools) Ripgrep

This is a really handy file searcher that is faster than grep. User Guide

TODO: Add to a unix command page.

Basic command:

rg bestPrice This will find any files that contain "bestPrice" recursively in the current directory. Basically what I want to do all the time.

rg bestPrice site Same, except search in the site folder in the current directory. ./site works too, or /root/site if it's an absolute path.

02/26/20 (Git) Check git stashed changes

git stash show -u

If it is possible, it will show the diff without popping it.

(Git) Stage only part of a file:

https://stackoverflow.com/questions/1085162/commit-only-part-of-a-file-in-git

You can use git add --patch <filename> (or -p for short), and git will begin to break down your file into what it thinks are sensible "hunks" (portions of the file). It will then prompt you with this question:

Stage this hunk [y,n,q,a,d,/,j,J,g,s,e,?]?

Here is a description of each option:

  • y stage this hunk for the next
  • n do not stage this hunk for the next commit
  • q quit; do not stage this hunk or any of the remaining hunks
  • a stage this hunk and all later hunks in the file
  • d do not stage this hunk or any of the later hunks in the file
  • g select a hunk to go to
  • / search for a hunk matching the given regex
  • j leave this hunk undecided, see next undecided hunk
  • J leave this hunk undecided, see next hunk
  • k leave this hunk undecided, see previous undecided hunk
  • K leave this hunk undecided, see previous hunk
  • s split the current hunk into smaller hunks
  • e manually edit the current hunk
  • ? print hunk help

02/25/20 (Firefox tips) Firefox search shortcuts

The gist:

You can add "%s" into a bookmark and add a keyword term to search something.

e.g.:

jira.com/search=%s keyword: j

In the address bar enter j apples

It will redirect to jira.com/search=apples

To add a keyword

Open all bookmarks page Shift + cmd + B and you will be able to add it under this page.

02/15/20 (Programming terminology) React Refresher

Following the students, here's a brief reminder of what these are:

Imperative - written code that says what is done to generate something, such as loading items into the DOM. Think jQuery or JS.

Declarative - representative code that shows the structure of how something would be organized in the DOM. Think React components or HTML.

02/09/20 (Unix basics) File and folder ownership

I'm dealing with some permission issues that are mind boggling, so I'll note down stuff as I figure them out here:

Folder permissions

  • To be able to cd into a folder, you must be able to execute based on the folders permissions.
  • To be able to ls a folder, you must be able to read based on the folders permissions.

logrotate permissions

The file gets rotated (same permissions as before), and then the original file gets recreated. The permissions for the recreated file is based on what is defined by create, e.g.:

create 0640 ec2-user dev myfile.txt

01/19/20 (AWS - Cloud9 - tip/tutorial) Add ssh creds to Cloud9

Since this was a pain in the ass, I'm going to write down my steps in an attempt to help expedite the process next time:

  • Create the Cloud9 instance
  • Create the ssh folder, create an ssh key, start up ssh agent, add the key:
mkdir ~/.ssh
ssh-keygen -t rsa -b 4096 -C "your email"
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_rsa
  • Add the key to a ~/.ssh/config file:
Host *
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_rsa
  • Change the permissions on this file, since it needs to be strict permissions:
chmod 600 ~/.ssh/config
  • Read out the ssh key and add it to your github profile:
cat ~/.ssh/id_rsa.pub
  • Add git config details since you'll need to anyways:
git config --global user.name "Mario Viens"
git config --global user.email [email protected]
  • Test the ssh connection:
ssh -T git@github.com
  • If it works, you can now clone with ssh!

01/17/20 (Unix - Zip files) Quick unzip of .gz file

zcat file.gz > file

Quick write straight to output of .gz file

zcat file.gz

01/10/20 (Unix - SSH tips/tutorial) Double ssh hop!

 Host <first host>
   AddKeysToAgent yes
   User <username>
   IdentityFile ~/.ssh/yourpem.pem

 Host bf-*
   User <user>
   ProxyCommand ssh -q <first host> -W %h:%p
   IdentityFile ~/.ssh/yourpem.pem

 Host bf-dev
   Hostname <second host/ip>

or just

 Host <first host>
   AddKeysToAgent yes
   User <username>
   IdentityFile ~/.ssh/yourpem.pem

 Host bf-dev
   Hostname <second host/ip>
   User <user>
   ProxyCommand ssh -q <first host> -W %h:%p
   IdentityFile ~/.ssh/yourpem.pem

Now you can go to bf-dev with ssh bf-dev!

01/10/20 (Programming terminology) "No free lunch"

This term was a part of a discussion of performance trade-offs. I didn't understand it right away and reading into it, it means there is a trade-off for this benefit. The term comes from machine learning in the 90s speaking about when there is chance involved, that means for example if I were to flip a coin and guessed right, there is a 50% chance I would be in a universe where I guessed wrong (tails). This is a risk-based "no free lunch" scenario.

In my scenario, which isn't exactly the same, but if we were to use a different service, our server costs increase. Seems like we shouldn't do that, however there's a trade-off in security and performance monitoring. I suppose 'no free lunch' could mean there is a compromise in a choice, not just a win scenario.

01/09/19 (Unix - scp) Secure copy into an ssh session

Well, one way is copying and pasting into the terminal. But another way is to use scp which will send the file over.

e.g.

#scp <file name> <username>@<host>:<destination directory>
scp testfile.txt mario@10.0.2.100:~

If you need to add creds

scp -rp -i ~/.ssh/my.pem testfile.txt mario@10.0.2.100:~

01/08/20 (Javascript - tips) JavaScript one-liners

Average an array of numbers

const arrAvg = arr => arr.reduce((a,b) => a + b, 0) / arr.length;

01/07/20 (Unix - grep - tips) Grep something with a dash before it

This issue arises due to it considering it to be a flag for grep. Even with quotation marks.

grep -- -t

The double dash marks the end of the parameters/arguments.

01/06/19 (JS Framework - Svelte) Svelte - a framework that might be worth looking into over React, Vue, preact, etc.

I'm reading through Sveltes blog. It reads very nicely. Makes me want to design a website that is like that.

https://svelte.dev/blog/write-less-code

Their blog looks clean, also check out Sapper.

01/06/19 (Unix - tips/tutorials) Count how many words are in your clipboard

pbpaste | wc -c

01/05/20 (Apache - tips/tutorials) Check what modules are loaded by apache

This is pulled from an article on monitoring apache web server performance: https://www.datadoghq.com/blog/monitoring-apache-web-server-performance/

Run the Apache server like you normally would, but using -D DUMP_MODULES like:

httpd -f /custom/path/to/my/apache.conf -D DUMP_MODULES

01/05/20 (Linux - systemd - tips/tutorial)Run a systemd service as a user

AKA: Alter on-boot services that are created with systemd

The files for a service like collectd.service are in /lib/systemd/system.

These have a syntax like:

 [Unit]
 Description=Collectd statistics daemon
 Documentation=man:collectd(1) man:collectd.conf(5)
 After=local-fs.target network-online.target
 Requires=local-fs.target network-online.target

 [Service]
 ExecStart=/usr/sbin/collectd
 Restart=on-failure
 Type=notify

 [Install]
 WantedBy=multi-user.target

In here, we can update it to execute as a user by adding a User=myuser entry under [Service].

01/03/20 (Git - tips/tutorials) Check what branch multiple repos are on

Add this to your .zshrc or .bashrc to be able to use gits

 # Git check branch
 codepath=~/code
 function check_branch {
     if ! [[ `git rev-parse --abbrev-ref HEAD | tr -d '\n'` = master ]];     then
         echo ""
         pwd
         echo "Please Note: Current branch is not master, instead it is:"
         git rev-parse --abbrev-ref HEAD
         echo ""
     fi
 }
 alias gitinfo='check_branch; git status -s | sed -E "s/\?\?/?/g" | while    read mode file; do echo $mode "$(stat -c "%.16y % 6s" $file)" $file;        done|sort -k1,4'
 alias gits='cd $codepath/repo1; gitinfo; cd ~-; cd $codepath/repo2;   gitinfo; cd ~-; cd $codepath/repo3; gitinfo; cd ~-; cd $codepath/repo4; gitinfo; cd ~-;'

source ~/.bashrc when you're done editing so these commands load.

01/03/20 (Shell - tips/tutorials) Enter prompts automatically for a script

# Hit enter ten times for the ten prompts.
printf '\n\n\n\n\n\n\n\n\n\n' | sudo /myscript-setup.py

Depending on how you need to enter the prompts, this is a cheap and easy way to run this.

More examples: https://askubuntu.com/questions/338857/automatically-enter-input-in-command-line

Note: often there will be a non-interactive mode that you can enable if you want to skip prompts. In my case, checking the source of the setup file showed -I flag was what I would want.

01/02/20 (AWS - CLI - tips/tutorials) Using AWS cli without configuring it first, setting the region

If you use a command without performing aws configure, it can complain about no region being set.

aws cloudwatch put-metric-data --namespace "atest" --metric-data file://tmp/metric.json
You must specify a region. You can also configure your region by running "aws configure".

Adding the region to the end of the command with the appropriate region --region us-west-1 fixes this.

e.g.

aws cloudwatch put-metric-data --namespace "atest" --metric-data file://tmp/metric.json --region us-west-1

(Linux - systemctl) Systemctl commands - enable a service on startup and how to test for it

From this site which is honestly easier to read than my documentation: https://www.dynacont.net/documentation/linux/Useful_SystemD_commands/

Enables a service to be started on bootup:

systemctl enable something.service
systemctl disable something.service # to disable instead

Check to see if the service is enabled:

systemctl is-enabled foo.service

This is independent of 'start' and 'stop' command which determines that it starts running or stops now.

systemctl start something.service
systemctl stop something.service
systemctl restart something.service

12/31/19 (Unix - find - tips/tutorials) Unix find file by name command without "Permission denied"

find / -name myfile.txt  2>&1 | grep -v "Permission denied"

(Unix - processes - tips/tutorials) Find out what files are in use by a process

sudo ls -l /proc/12345/fd/

where 12345 is the pid of the process

12/27/19 (AWS - API Gateway - tt) AWS API Gateway setting new settings to a resource

This had me stumped. I was changing the API Key to be required on a resource, but it would not propagate to the stages.

What is needed to propagate the change is to Deploy (Resources > Actions > Deploy API) and select the stage. This makes it so when updating settings, you can roll out changes to your dev/beta environment and then your production environment later.

12/13/19 (Unix - processes - tt) Find how a process was run via command line

This didn't seem to work with nginx, it gave a description instead, but:

ps eaf | grep nginx

The following is an equivalent, as it's stored in a file under the pid:

cat /proc/12345/cmdline

can do ; echo "" to make it go onto its own line since there is no newline at the end. The whitespace also might not be interpreted.

12/10/19 (Firefox) Disable JS in Firefox

  • Go to about:config
  • Search for javascript
  • Double click javascript.enabled to disable it.

12/09/19 (AWS - Insight) Parse AWS Insight Logs

How to parse the timestamp like "2019-12-09T09:34:15.000-08:00", it's actually in UTC epoch ms, so will need to use math on it, as describes in datetime functions: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html

datefloor(@timestamp, 1d)

(Python) Combine two arrays in Python

a = [1, 2, 3]
b = [4, 5, 6]
a + b  # [1, 2, 3, 4, 5, 6]

(Python) Get the keys from a dictionary in Python

my_dict = {
  'a': 'apple',
  'b': 'banana',
}
list(my_dict.keys())  # ['a', 'b']
list(my_dict)  # ['a', 'b'] - even simpler

12/07/19 (Mac) Disable trackpad when mouse is connected

http://cdn.osxdaily.com/wp-content/uploads/2016/11/disable-trackpad-with-external-mouse-mac.jp size:small

12/06/19 (Unix) How to get a zip onto a remote system

curl https://website.com/zipfile.zip -o outputfilename.zip
unzip outputfilename.zip -o outputfolder

(Files) How to get image from mp3 file Mac OS

Install eyeD3:

pip install eyeD3

Use:

eyeD3 --write-images=DIR mp3_file

12/04/19 (Git) Git commands

How to reason with git cherry

What is in current branch that isn't in 1

git cherry 1

What is in 2 that isn't in 1

git cherry 1 2

...And get the commit message with -v

git cherry -v 1 2

(Git) How to deal with master updating while working on feature branch

git checkout feature
git rebase master

This is a great tutorial on it: https://www.atlassian.com/git/tutorials/merging-vs-rebasing

This effectively moves all the new commits in master to before the branch, so to make a very easy continuity for when merged back into master.

12/02/19 (Bash) Bash syntax - how to make an if statement on a boolean

Note 1: Keep any assignments cuddling:

myVar = true # wrong
myVar=true # right

This is the right way to approach boolean switches:

myVar=true

if [ "$myVar" = true ]; then
    # whatever
fi

Note that having quotes around the variable being defined is a convention recommended and explained here: http://tldp.org/LDP/abs/html/quotingvar.html

I'm not sure if it works without the quotations in this scenario.

if test "$myVar" = true; then

is functionally equivalent to:

if [ "$myVar" = true ]; then

Under the hood, it literally takes [ as an expression similarly to how test is, and requires the final argument is ']' which makes it imitate brackets.

Since the brackets are expressions, they must have the white space between it and the expression or else it won't execute correctly.

11/30/19 Set up a website with Github Pages

  1. Buy your domain.
  2. Create a Github organization with the name of your website.
  3. Create a repo within that organization that is self-name, e.g.: "organization.github.io"
  4. Add the file "CNAME" in the repo with the name of the website in this file e.g.: "organization.com"
  5. Add whatever static files you want to serve, you can use Jekyll to create something Github Pages will build and manage.
  6. In your domain provider add an A record with the four DNS IPs for GitHub Pages:
185.199.108.153
185.199.109.153
185.199.110.153
185.199.111.153
  1. (Optional) Add a CNAME record for www.mysite.com to mysite.com
  2. (Optional) Enable DNSSEC if it's available

How to redirect a website to another website

Go into the domain's website section and forward the domain (302 preferred).

11/29/19 Find largest 10 directories in system

Disk space usage - Show the 10 largest directories below "/"

sudo du -a / | sort -n -r | head -n 10

Check git config settings

git config -l
git config --global core.filemode false

This is default set to true.

Advanced monitoring of traffic on EC2 instances

To monitor the traffic going to and from network interface of your instances, use tcpdump command on both your search servers and public site server:

sudo yum install tcpdump -y              # installs tcpdump
sudo tcpdump -w foo.pcap -i eth0         # runs tcp dump on interface eth0

11/28/19 Grep Or Command - Search for multiple things

watch_logs | grep '404\|502'

Get cron logs

sudo tail -f /var/log/cron

11/19/19 build terminology

Version Set - Is the full code build for production without test or dev dependencies.

11/18/19 "vim multi cursor" - Change a set of occurences of a string in vim

Source: https://medium.com/@schtoeffel/you-don-t-need-more-than-one-cursor-in-vim-2c44117d51db

I wanted a way to finally do some of the hot code changes that I was able to do in my previous IDEs: effortlessly change multiple lines.

How do you change multiple occurences of something in Vim?

Prerequisite: knowledge of searching in Vim.

Search for the line you want. Take the following as an example:

setting project/path/Iwanttochange/settings.conf
settingsDefault project/path/Iwanttochange/settingsDefault.conf

I want to change this to:

setting project/conf/settings.conf
settingsDefault project/conf/settingsDefault.conf
  • Perform the search /path\/Iwanttochange
  • Enter
  • cgn (The word will now disappear and enter Insert mode)
  • Enter what you want to replace it with
  • Escape to be done and exit Intsert mode
  • . to replace next occurence, and so on.

11/14/19 merge master to branch without creating merge commit

(I am now making the title my google search entry)

From: https://shinglyu.com/web/2018/03/25/merge-pull-requests-without-merge-commits.html

Solution:

# (on branch)
git rebase master
a--b     ->  a--b
 \c--d           \c--d

This can even be done after a merge commit to nullify the merge commit.

   merge commit
       |
       v
a--b---e   ->  a--b
 \c--d/            \c--d

11/13/19 Shell commands: How to check if a file exists:

FILE=/etc/resolv.conf
if test -f "$FILE"; then
    echo "$FILE exist"
fi

11/06/19 Managing processed: how many files are open for a given process

https://stackoverflow.com/questions/21752067/counting-open-files-per-process

Apparently it's just ls /proc/$pid/fd/ | wc -l where $pid is the process id number.

11/04/19 Creating certs in AWS

This has been such a headache. Anyway, it's not too bad now that I've gone through the motions.

AWS has its own Certificate Authority (CA), called ACM. This is also where certs are imported if you didn't go with ACM to create your certs, and use the imported certs in CloudFront for your application.

One way to verify you are the owner of the sites you requested the CA for is to create a CNAME under your domain with the value they provide. They even provide a button to do this in Route 53 if you have your domain pointing to Route 53.

Tip: Make your cert encompass both www.mysite.com and mysite.com. Then ideally CNAME your www.mysite.com -> mysite.com.

10/31/19 Aliasing hosts you want to connect to

I can name my long ssh id something simple like mario-cloud by adding this to ~/.ssh/config:

Host mario-cloud
  Hostname dev-blah-mario-blahblahlongstring.us-west-2.hostcompany.com
  User myusername

10/30/19 Perl OOP

Sucks. But here is the official tutorial: https://perldoc.perl.org/perlootut.html

10/29/19 Setting up Perl locally because you have to

This is Mac intructions taken mostly from Derek Banas's perl tutorial.

curl -kL http://xrl.us/perlbrewinstall | bash

Then update your .bash_profile or your .zshrc with:

# Use perlbrew
source ~/perl5/perlbrew/etc/bashrc
perlbrew use perl-5.30.0

Then reopen your terminal and install the latest Perl (google it), and then source the file once more to have the 'use' command select the correct perl version.

perlbrew install perl-5.30.0
source ~/.zshrc

Cool, you have Perl. Now I know this pattern isn't ideal, but I think it's good enough and doesn't use/overwrite your OS's perl.

Style guide: https://github.com/chusqui/perl-style-guide I'm going to just hope this is good.

Basics tutorial: https://www.youtube.com/watch?v=WEghIXs8F6c

SEMICOLONS MATTER

10/25/19 I forgot to get a halloween costume, yet again

Working with awk and sed

So apparently awk and sed are great for messing with the output of commands. So it's great to learn. Here's some basic commands I figured out:

This is using load.txt as an example input (which is just uptime on two systems):

[10.0.3.24] out:  00:11:05 up 3 days,  1:31,  1 user,  load average: 1.13, 1.45, 1.30
[10.0.3.170] out:  00:11:05 up 3 days,  1:31,  1 user,  load average: 2.01, 1.29, 1.32

AWK

Print first column and then second column

cat load.txt | awk '{print $1 $2}'

or

awk '{print $1 $2}' load.txt

I like the first one better because it reads more linearly piping the output of cat to the input of awk.

Results:
[10.0.3.24]out:
[10.0.3.170]out:

Remove first character from a string

cat load.txt | awk '{print substr($1,2)}'

Results:
10.0.3.24]
10.0.3.170]

Remove last character from a string

cat load.txt | awk '{print substr($1, 1, length($2)-1)}'

Results:
[10.0.3.24
[10.0.3.170

Have a space separating different prints

cat load.txt | awk '{print $1, $2}'

Results:
[10.0.3.24] out:
[10.0.3.170] out:

SED

Delete any line that contains word

cat load.txt | sed '/24/d'

Results:
[10.0.3.170] out:  00:11:05 up 3 days,  1:31,  1 user,  load average: 2.01, 1.29, 1.32

Delete all lines except the last 10

Okay I don't know what this actually means right now...:

sed -e :a -e '$q;N;11,$D;ba'

This would return the same thing, but if you're dealing with >10 servers instead of 2, youll get only the bottom 10.

Sort, oops theres that too

Sort the rows by the second column, using ] as the separator (' ' as separator is default)

cat load.txt | awk '{print $1 $12}

Result:
[10.0.3.170]2.01,
[10.0.3.24]1.13,

cat load.txt | awk '{print $1 $12} | sort -t] -nk2

Result:
[10.0.3.24]1.13,
[10.0.3.170]2.01,

Pretty good bashrc sharing

# bashrc shortcuts
alias pullbashrc='cp ~/.bashrc ~/.bashrc.old; cp /(common location)/.bashrc ~/.bashrc' #pull master bashrc, backup your bashrc to .bashrc.old
alias pushbashrc='cp ~/.bashrc /(common location)/.bashrc' # push your bashrc to master
alias getdiffbashrc='vi -d ~/.bashrc /(common location)/.bashrc' # Gets a side by side color coded delta of bashrc, opens in vim.

diff --brief ~/.bashrc /(common location)/.bashrc > /dev/null # This checks for a difference between bashrc files. It sets $? to 1 if these files are different, which gets used below.

if [ $? -eq 1 ]; then
   echo "Bashrc is different than the one at '/(common location)/.bashrc'."
   echo "You can check the differences with ( getdiffbashrc )"
   echo "Please update your script ( pullbashrc ), or update their script ( pushbashrc ) and make a patch. :)"
fi

Color coded side by side diff in Vim!

vi -d file1 file2

So easy. :')

10/21/19 Fabric command host targetting

I had trouble wrapping my head around this. For context, fabric is a python library used to execute commands. It can be used to execute commands against different hosts, which is great to use when you've multiple servers deployed from one master server.

To target a host, you simply add -h with comma-separated values for what all the hosts are.

fab -H 10.0.1.2 check_thing

I think you might also be able to alter env.hosts to an array of hosts to also accomplish this if you're chaining fabric commands fab firstcommand secondcommand

10/20/19 Repeating history commands

history will show a list of past history commands, with a number on the side.

!10 will perform the one at line 10 again. Woo.

10/17/19 Volumes, drives, mounting...

df -h will list your mounted partitions.

ls -l dev will have some mountable partitions (not really clear to me yet)

sudo mount -t ext4 /dev/sdf /myfolder will mount a partition located at /dev/sdf with the filesystem ext4 to /myfolder.

Per: https://askubuntu.com/questions/154180/how-to-mount-a-new-drive-on-startup Adding to fstab to make this a process done on system start:

Add to /etc/fstab:

#device        mountpoint             fstype    options  dump   fsck

/dev/sdf    /product    ext4    defaults    0    1

Makes an associated drive mount on startup.

10/16/19 Tree of processes!

pstree

Amazing.

ps axjf

Oh nice.

With this combined with the fact that I can use a kill command to enter an if statement instead of exiting early, I am able to get the correct process to kill in the event that we've killed the wrong process:

if kill -SIGQUIT $PID > /dev/null 2>&1; then
    sleep 1
fi
echo "Checking for remaining uWSGI processes..."
if pgrep -x 'uwsgi' > /dev/null; then
    echo "number of uWSGI processes remaining:"
    ps -ef | grep -c "[u]wsgi.pid" # The bracket is a way to make the grep process be excluded from the list
    process=$(ps jx|grep "[u]wsgi"|awk '{print $2}'|head -1)
    echo "Parent process id is likely: $process"
fi

Executing python file without stating python!

Use a shebang at the top of a file:

#!/usr/bin/env python3

env lets you use the system $PATH for the next command. We need to use the full path for where env is to then find the user's python (and I want it to be python3 so I say python3 just to be sure).

10/15/19 Running shell commands from python!

import os
myCmd = 'ls -lah'
os.system(myCmd)

Very cool.

Docopt is also very cool!

"""

Usage:
    main.py [options]

Options:
    -f --folder=<name>    Folder to make torrents from.

"""

import os
from dotenv import load_dotenv
from docopt import docopt

if __name__ == "__main__":
    load_dotenv()
    arguments = docopt(__doc__)
    print(arguments)
    folder = arguments['--folder']
    ANNOUNCE_URL = os.getenv("ANNOUNCE_URL")

    if not ANNOUNCE_URL:
        raise "Need an announce url. Set this in a .env file."
    if not folder:
        raise "Need a folder"

    myCmd = 'mktorrent -l 18 -p -s OPS -a ANNOUNCE_URL ~/folder -o filename'
    #os.system(myCmd)
    os.system('pwd')

This isn't great, but you should get the point.

10/10/19 Python and lower level stuff

Some terms:

Python has GIL: "Global Interpreter Lock". This allows mutex to happen, as Python is synchronous in nature but allows for threading: https://opensource.com/article/17/4/grok-gil

Bytecode: "Bytecode is program code that has been compiled from source code into low-level code for a software interpreter. It may be executed by a virtual machine (such as a JVM) or further compiled into machine code, which is recognized by the processor."

Mutex (threading) and Semaphore (processes): http://www.linfo.org/kernel_space.html

These work in: user space (thread management) and kernel space (process management): http://www.linfo.org/kernel_space.html

10/3/19 It's October 3rd.

Python args and kwargs

I was spooked by the stars and stuff, but understand now:

def myGet(kind=False, *args, **kwargs):
    for a in args:
        print(f'args {a}')
    for a in kwargs:
        print(f'kwargs {a}')
    for key, value in kwargs.items():
        print(f'{key}: {value}')

myGet(1, 2, 3)

print('-------')

myGet(1, 4, banana=2, pear=3)
args 2
args 3
-------
args 4
kwargs banana
kwargs pear
banana: 2
pear: 3

But there are other places that have single and double star! I confuse!

It's destructuring an array or object, respectively.

E.g. (I think)

a = (1,2,3)
def b(first, second, third):
    ...

b(*a) # assigns 1, 2, 3 to the first second and third argument

Setting up python with Vim

I still use vanilla vim, so the Vundle use in the tutorial here will be skipped. All I'm going to do for now is add this to my .vimrc:

au BufNewFile,BufRead *.py
    \ set tabstop=4
    \ set softtabstop=4
    \ set shiftwidth=4
    \ set textwidth=79
    \ set expandtab
    \ set autoindent
    \ set fileformat=unix

10/2/19 Python commands and modules

Since I don't know how to debug like a normal person in Python yet, I'm doing exploratory inspection to find out things about our modules.

To see the version of a module

python
>>> import boto
>>> boto.__version__

To see the key/values of an object

>>> for property, value in vars(boto.config).iteritems():
     print property, ": ", value
(one last enter to execute)
(Output:)
_optcre :  <_sre.SRE_Pattern object at 0x7f58c030b030>
_defaults :  OrderedDict([('debug', '0'), ('working_dir', 'redact')])
_sections :  OrderedDict([('s3', OrderedDict([redact]))])
_dict :  <class 'collections.OrderedDict'>

So now I can see that boto.config._sections is something I can inspect for what I'm doing. I just guessed 'config' as an attribute.

But we can run into the issue of not knowing what key to originally look under. Also iteritems() doesnt seem to work in Python 3. How do we find the keys under a module?

Python's inspect library.

>>> import inspect
>>> inspect.getmembers(module)

This will give all those keys are under this object.

9/18/19 Vim customizations

I added these into ~/.vimrc as recommendation by a coworker.

set autoindent
set smartindent
set ignorecase
set smartcase

9/16/19 Amazon Product Advertising API - Making searches by ISBN

Amazons API Docs: https://webservices.amazon.com/paapi5/documentation/item-info.html

The docs dont have anything to do with ISBN searches unfortunately. Mostly their own internal IDing.

If you are using their paapi5_python_sdk (or probably any other PAAPI5 SDK), you'll need to send with your GetItemsRequest item_id_type="ISBN".

From there, I'm still trying to figure out the format it expects. Right now an array of ISBNs give:

('Status code:', 400)
('Errors :', '{"__type":"com.amazon.paapi5#ValidationException","Errors":[{"Code":"InvalidParameterValue","Message":"The value [9780073523910] provided in the request for ItemIds is invalid."},{"Code":"InvalidParameterValue","Message":"The value ISBN provided in the request for ItemIdType is invalid."}]}')
('Request ID:', '27c20cf5-f34b-41f3-b5ab-d1dd910bcf79')

And a single string breaks the entire thing:

('Status code:', 500)
('Errors :', '{"__type":"com.amazon.coral.service#InternalFailure","message":"The request processing has failed due to some unknown error, exception or failure. Please retry again."}')
('Request ID:', 'f7e80745-29f5-4b7c-8a05-545d10c0e7f1')

9/10/19 Perl

I'm reading through https://perl101.org/ and it's pretty straight forward. I'm using Perl 5. Perl 6 is the newest version, but everyone's migrating away from Perl.

Basics

my in Perl is var in JS.

$ is a 'sigil' for a 'scalar' value. In other words, a primitive value. E.g. my $name = "Mario"

A sigil is something you put before the variable name.

You can interpolate strings when using double quotes. Single quotes does not interpolate strings. E.g. my $fullname = "$name Viens"

@ is a sigil for an array. E.g. my @arr = qw( 1 2 3 )

qw() stands for "quote on whitespace into a list"

Accessing arrays is the same as JS @arr[0], @arr[-1]

Undefined in Perl is undef. @arr[3] # undef

Arrays flatten if combined

my @sandwich = ( 'PB', 'J' );
my @other_sandwich = ( 'B', 'L', 'T' );
my @ingredients = ( @other_sandwich, @sandwich );
# ( 'B', 'L', 'T', 'PB', 'J' )

References are like pointers in C. This section is great and succint: https://perl101.org/references.html

It in essence turns a non scalar (non primitive) into a scalar reference value so you can refer to a more complex value by its reference in the spot where a scalar value can be stored and then dereference it.

my $aref = \@array; # reference

my @other_array = @{$aref}; # new array

You can dereference and access a value within a non scalar with an arrow: my $stooge = $aref->[1];

8/30/19 SSH failure into rebuilt EC2 Instances

(fab_env_2.7) [vienmari@master2 master]$ ssh 10.0.2.214
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:M7wX6GGZrcPopNXo/CS3ZXMuFUADt62oE9CErLzWPIA.
Please contact your system administrator.
Add correct host key in /home/[name]/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/[name]/.ssh/known_hosts:50
ECDSA host key for 10.0.2.214 has changed and you have requested strict checking.
Host key verification failed.

This is due to logging into an ip with one key but when trying to log in again once keys have been rotated, it'll try to log in with the old key.

Moving (safer) or deleting the known_hosts file fixes this.

8/23/19 Shell Error handling

https://unix.stackexchange.com/questions/39623/trap-err-and-echoing-the-error-line

This looks neat.

err() {
    echo "Error occurred:"
    awk 'NR>L-4 && NR<L+4 { printf "%-5d%3s%s\n",NR,(NR==L?">>>":""),$0 }' L=$1 $0
}
trap 'err $LINENO' ERR

8/20/19 Python and AWS upgrades!

chown -R username filename - sets the ownership of a directory recursively to an individual.

sftp extension for vscode is great. Able to pull files down to your local machine.

Upgrading Amazon Linux to Amazon Linux 2

I had troubles with the previous scripts for instantiating a new AMI clone.

sudo yum update --security --releasever=latest

Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Could not retrieve mirrorlist http://amazonlinux.us-east-1.amazonaws.com/latest/core/latest/x86_64/mirror.list error was
14: HTTP Error 403 - Forbidden


 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=<repoid> ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot find a valid baseurl for repo: amzn2-core/latest/x86_64

This issue was at least in part due to Amazon Linux 2 not having a link to latest. Since without args, it pulls the latest, I changed sudo yum update -y --security --releasever=latest to sudo yum update -y

Thoughts on vim

Visual mode should really be called "highlight" or "select" mode. Once I think of it that way, it's made it easier to understand.

8/19/19 Messing around with python and virtualenvs

virtualenv is Python's rough equivalent of having a locked python package environment per repo or project, similar to npm.

General usage

Create a new environment

(Note .env or venv in the root of a repo is a common pattern)

virtualenv .env: creates .env folder as the new virtualenv.

virtualenv -p python .env: creates like above, but using the version of python defined.

source .env/bin/activate: starts virtualenv session

deactivate: stops virtualenv session

Creating a "what to install" file

pip3 freeze > requirements.txt

Installing locally

On a fresh Mac, there is no pip. There is python2.

Install python3 (needs brew):

brew install python

Install virtualenv:

pip3 install virtualenv

Now virtualenv will ONLY install python3 if we give it no args. This might be because we installed virtualenv with pip3:

➜  ~ virtualenv venv-no-args
Using base prefix '/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/vienmari/venv-no-args/bin/python3.7
Also creating executable in /Users/vienmari/venv-no-args/bin/python
Installing setuptools, pip, wheel...
done.
➜  ~ source venv-no-args/bin/activate
(venv-no-args) ➜  ~ python --version
Python 3.7.4

What we probably want is to tell it to install with python2, which globally is python:

➜  ~ virtualenv -p python venv2
Running virtualenv with interpreter /usr/bin/python
Already using interpreter /usr/bin/python
New python executable in /Users/vienmari/venv2/bin/python
Installing setuptools, pip, wheel...
done.
➜  ~ source venv2/bin/activate
(venv2) ➜  ~ python --version
Python 2.7.10
(venv2) ➜  ~ python3 --version
Python 3.7.4

More useful commands

cp -avr /home/blah ./blahblah

-a : Preserve the specified attributes such as directory an file mode, ownership, timestamps, if possible additional attributes: context, links, xattr, all. -v : Verbose output. -r : Copy directories recursively.

8/15/19 Check ur ec2-user mail

echo $MAIL
> /var/spool/mail/ec2-user
> You have mail in /var/spool/mail/ec2-user
vi $MAIL

Then that annoying You have mail in /var/spool/mail/ec2-user shuts up. Can search for the Subject line (/Subject, enter, n, n, etc.), or skip to bottom G or top gg of the file.

More Commands >:3

Find empty files: find . -size 0 Find empty files and delete them: find . -size 0 -delete

8/14/19 More helpful commands!

ls | wc -l: wc is word count, -l counts lines. We can use this to find out how many files are in the directory.

ls -a: includes hidden files

ls -a | wc -l: counts all files including hidden files.

8/13/19 Words, kill processes

Idempotent - means if the same operation was done again, the result will be the same. Not additive behavior (e.g. Put instead of Post).

How to check process names and kill them

ps -A: show all processes

pgrep -l [processname]: short form of greping ps. Very similar to ps -A | grep [processname].

sudo pkill -USR1 [processname]: kill all processes with this name!

8/9/19 Dev Ops

SSH credentials

Private vs public vs pem

So you can create a private key (no extension). This is paired with a public key (.pub).

Some things want the public key in a format of a pem file, such as the ec2 instances keys. This can be obtained by: ssh-keygen -f key.pub -e -m pem

logging into an ec2 instance uses the "Key Name" column

You can generate or import a new key by going into EC2 > Network and Security > Key Pair.

It'll use the public key.

Removing ssh-added keys

The easiest way imo is to remove all the keys:

ssh-add -l # shows all keys ssh-add -D # deletes all keys

Then just re-add the keys you want.

8/8/19 Dev Ops

AWS CloudWatch

this is where logs are passed along to from the different AWS EC2 instances. This makes it so you can make clusters and still pull their logs collectively.

Perl

There's a pattern of doing:

my $error = 0;
$error|=add_public_key('blah','blah');
$error|=add_public_key('blah1','blah');
$error|=add_public_key('blah2','blah');

...

return $error;

This is assigning an error message to $error if it isn't already occupied with an error message. As a result, the return value of a function is the error message if anything went wrong. Seems pretty straight forward.

7/31/19 Dev Ops

Updating a folder to recursively add write access for group:

chmod g+w -R foldername

Checking to see if a branch contains any commits not already on master:

git cherry -v master unfinished-code-1

6/10/19 React Native... more

To apply multiple styles to a line, you can use array notation:

<View style={[styles.base, styles.background]} />

6/7/19 React Native

I'm building out something in React Native and documenting setup here: https://wiki.mitchellburton.ca/react-native/

The main ways to build up a quick app with a cli is either expo-cli or react-native-cli. One thing to note about the react-native-cli boilerplate is that it provides a nice example of how to show something on iOS and something else on Android

const instructions = Platform.select({
  ios: 'Press Cmd+R to reload,\n' + 'Cmd+D or shake for dev menu',
  android: 'Double tap R on your keyboard to reload,\n' + 'Shake or press menu button for dev menu',
});

1/23/19 ...Copying mysql db

use commands from outside of mysql:

Some aliases first:

alias ssh-db='mysql --defaults-extra-file=[Your path to your login config]/config.cnf' alias mysqldump-loggedin='mysqldump --defaults-extra-file=[Your path to your login config]/config.cnf'

Commands

mysqldump-loggedin olddatabasename > db.sql // This copies the old db

Log into mysql, create a database.

ssh-db newdbname < db.sql // This replicates the copied db to the new db.

01/23/19

Anyways. Happy New Year.

Setting up Nanoleaf to be hacked

Looks like Nanoleaf is easy to hack with python. I'd prefer JS, but whatever.

https://github.com/software-2/nanoleaf

The instructions say to use pip, but I've pip3 so... pip install nanoleaf --upgrade

12/22/18

I'll preface this with thank you Vaugham Hong, who without a doubt I will not be able to do what I am today without him and his infinite patience and wisdom. Be it seeing his reaction to "a series of if statements" to an AI. Be it willing to share deep knowledge of his written and licensed code. I suppose in that, I recognize the need to license code. But not to limit others, but the opposite. To make sure it never does get limited by others.

App solution

Make there a way to bring your system back to restore to where it was before. Just as simple as resetting a computer to factory restore.

TOC: seven things:

  1. Application rebuilding
  2. Log into everything
  3. 4. 5. 6. 7.

  4. Application rebuilding This is where you install everything. Most people are back up to business as usual. Things you may want:

  • Spotify
  • Alfred
  • iTerm2
  • Visual Studio Code
  • Serato DJ Pro
  • Caprine
  • WhatsApp
  • zsh
  • oh my zsh
  • Slack
  • Add vscode as a terminal bin
  1. Log into everything
  • Set up your fucking ssh. If you know what 'ssh' even could mean, this applies to you
  • Log into everything
  1. Load back up file restores
  • Think: Synced from the cloud. It'll be different person to person.
  • Music
  • ~/.zshrc file
  • install ohmyzsh?
  1. Customize your environment further
  • Add hyperzoom option i.e. accessibility options
  • The rest is here https://wiki.mitchellburton.ca/mac/
  1. Sort the chaos
  2. Premature optimizations
  3. The death of the universe

(Try to avoid the last two steps)

Lateral thoughts

It wasn't until I was pulling cords out of a drawer full of more cords that I realized how garbage collection and refactoring code really could apply to life problems.

I mean, I always knew, but I didn't find the political responsibility to engage in these sorts of things.

It's good to be able to map unrelated things.

12/20/18

Design thoughts regarding sub-keys of objects that may not exist on first load

After dealing with the invalid state of objects that need to be async get before we can access its keys, we run into the issue of trying to access a key on something that is undefined.

If we only use these keys within a new component, then we can have filters, e.g.:

const MainPage = (props) => {
  const {organization} = props

  return (organization ? <OrganizationView {...organization} /> : null)
}

const OrganizationView = (organization) => {
  const {metricsUrl} = organization

  return (metricsUrl ? <div>metricsUrl</div> : null)
}

Something like that.

12/19/18

Deciphering TypeScript Error Messages, Part 3

Functions declaration with types:

function ImgMediaCardPanoramaUpload({classes, group: GroupKeys }: GroupDetailsProps) {

Functions with types:

uploadThumbnail: (param: {file: object, id: string}) => void

12/18/18

Deciphering TypeScript Error Messages, Part 2

This one is to do with a form:

private form: null

...

this.form.current.reset()
...

<Formsy
  ref={this.form}
>

gets

TS2531: Object is possibly 'null'.

Making the null part of it to check if it's null first doesn't seem to solve things though:

this.form!.current.reset()

gets

TS2339: Property 'reset' does not exist on type 'never'.

So it seems even though to my understanding it shouldn't be attempting to access reset on it if it's undefined or null according to the !, this doesn't seem to work either.

  private form: Formsy = React.createRef<Formsy>()

This fixes the error.

Onto the next error...

TS2339: Property 'blur' does not exist on type 'EventTarget'.

If from line:

event.target.blur()

This is due to the type of event.target not being an HTMLInputElement. Okay, lets set that:

;(<HTMLInputElement>event.target).blur()

gets:

Module parse failed: Unexpected token (33:16)
You may need an appropriate loader to handle this file type.
|                 (React.createElement(HTMLInputElement, null,
|                     "event.target).blur() // Fires off update } } render() ",
>                 ,
|                     " ",
|                     id,

It doesnt know that syntax at all. Okay, there's another way to do it, maybe I fucked up:

;(event.target as HTMLInputElement).blur()

This finally works.

Alternative:

const target = event.target as HTMLInputElement
target.blur()

12/17/18

Deciphering TypeScript Error Messages

TS2345: Argument of type 'typeof Account' is not assignable to parameter of type 'ComponentType<ConsistentWith<Props, { classes: Record<"root" | "select" | "formControl" | "heading", string>; innerRef?: ((instance: any) => void) | RefObject<any> | null | undefined; }>>'.
  Type 'typeof Account' is not assignable to type 'ComponentClass<ConsistentWith<Props, { classes: Record<"root" | "select" | "formControl" | "heading", string>; innerRef?: ((instance: any) => void) | RefObject<any> | null | undefined; }>, any>'.
    Types of property 'defaultProps' are incompatible.
      Type '{ user: { id: string; email: string; }; }' is not assignable to type 'Partial<ConsistentWith<Props, { classes: Record<"root" | "select" | "formControl" | "heading", string>; innerRef?: ((instance: any) => void) | RefObject<any> | null | undefined; }>>'.
        Types of property 'user' are incompatible.
          Type '{ id: string; email: string; }' is missing the following properties from type 'User': name, createdAt, organization

This means that the defaultProps set don't match the TS types set for this element (in my case, the React component 'Account').

TS2322: Type '{ children: Element; title: string; }' is not assignable to type 'Readonly<Props>'.
  Types of property 'children' are incompatible.
    Type 'Element' is missing the following properties from type 'Component<{}, {}, any>': context, setState, forceUpdate, render, and 2 more.

This means React.Component is not the type of the prop, it is actually React.ReactNode in this case. Not sure why.

Mario's Best Practices - Part 1

Lately I've been dealing with general ideas on code a lot, so I'm just putting my brain onto electronic paper about it.

  1. Declare state directly in the class.

...#5 of this guide (also linked below) has an opinion on this.

  1. (TS) No commas or semicolons after each item in an interface. It's just not needed.

Good:

interface Props {
  user: User
  getOrganizations(): null
  updateUser(user: UpdateUser): null
  changePassword(event: Event): null
  classes
}

Bad:

interface Props {
  user: User,
  getOrganizations(): null,
  updateUser(user: UpdateUser): null,
  changePassword(event: Event): null,
  classes,
}
  1. (TS) If the type of something is any, but you need to write the name of it, just write the name and any is implied.

Good:

interface Props {
  changePassword(event)
  classes
}

Bad (x3):

interface Props {
  changePassword(event: any): any
  classes: any
}
  1. Use the names State and Props when the file only has one component defined within it since it's in its own namespace.

I love how there's four 1's instead of the actual numbers I put above... so much for an itemized list.

Part 1.5

Read this: https://medium.com/@martin_hotell/10-typescript-pro-tips-patterns-with-or-without-react-5799488d6680

Important notes:

  • .#1 Don't use public keyword, functions are already public by default.
  • .#5 Don't use the constructor keyword.
  • .#7 Don't export props and state interfaces.
  • .#8 Always define the types on children since there will be cases where we don't want children for the component.
  • .#12 Don't use namespace.
  • .#14 Use kebab-case filenames. Sure?
  • .#15 Put types above the rest of the content. Separates run-time and compile-time code. Sure.
  • .#16 Don’t use method declaration within interface/type alias. This means use doThing: () => string instead of doThing(): string. Okay.

I don't 100% agree with #2. I don't really understand the benefit of #11 other than not needing to change the code as much between JS and TS.

12/14/18

The codebase isn’t complaining about a component with missing props where the props are defined as required in TS… I just happened upon it being used incorrectly, but no complaints.

I had to mull over why that’s happening for a bit, but I think I figured it out. It’s because the part of the codebase where it is used is a regular JS file. So even if I implement the bare minimum for all the other files for now, it seems it’ll extend the type-reach of the typed components.

12/13/18

This would be a nice small project that could turn into a website:

The conversion from babel presets to typescript presets (lib).

Babel:

presets: [
  '@babel/preset-react',
],
plugins: [
  // New post-ES6 syntax
  '@babel/plugin-syntax-object-rest-spread', // Spread syntax
  'babel-plugin-transform-class-properties', // Class arrow functions
  'react-hot-loader/babel', // Hot reload webpack
],

Typescript:

...
"lib": ["es5", "es6", "dom"],

es6 could be broken up into the spread, class arrow syntax, but I haven't taken the time to figure out which one is which.

12/6/18

React Router Render Redirect on click!

I want to click on a link and have it behave like an a tag without actually behaving like an a tag. Apparently that's not the easiest request.

This is what I was shoving into my codebase in-component. and then using this.setState({redirect: path}) to redirect on click:

import React from 'react'
import PropTypes from 'prop-types'
import { Redirect } from 'react-router-dom'

class Redirect extends React.Component {
  state = {
    redirect: false
  }

  redirectTo = route => {
    this.setState({ redirect: route })
  }

  render() {
    const {redirect} = this.state

    // Redirect handler
    return redirect ? <Redirect push to={redirect} /> : null
  }
}

Redirect.propTypes = {
  to: PropTypes.string,
}

export default Redirect

Where Redirect is basically the name of the component it's intertwined with.

Issues with this:

  • Extremely coupled. I tried to pull it out into its own component (the above code) and failed.
  • Have to create another item in state, add a function, and follow the same pattern for this Redirect to work.

Instead what works better (and a sledgehammer now that I got the hang of it): use a Render prop. Specifically, use Route without a path so that it always renders. Now it just gives you the history prop which you can use to push/redirect within the component it is wrapping:

<Route render={({history}) => (
  <TableRow
    hover
    key={id}
    className={classes.hover}
    onClick={() => { history.push('/organizations/' + id) }}
  >
    <TableCell>{name}</TableCell>
    <TableCell onClick={stopPropagation}>
      {metricsUrl && <CopyToClipboardField text={publicMetricsUrl} showMessage={showMessage}/>}
    </TableCell>
  </TableRow>
)} />

Something like that.

11/23/18

Render Props instead of HOCs

No more HOCs.

Previously:

import React from 'react';
import {store} from 'redux/store';
import {Redirect} from 'react-router-dom';

const home = '/';

export function authorize(Component) {
  return props => {
    const level = 'admin'; // Only User and Admin levels for now so we can hard-code the only option
    const userPermission = store.getState().auth.user.permissions;
    const isAdmin = level === userPermission;
    return isAdmin ? <Component {...props}/> : <Redirect to={home}/>;
  };
}
authorize(AdminPage)

Render Prop:

import React from 'react';
import {store} from 'redux/store';
import {Redirect} from 'react-router-dom';

const home = '/'

function AdminRoute(props) {
  const {component: Component, ...rest} = props
  const level = 'admin'
  const userPermission = store.getState().auth.user.permissions
  const isAdmin = level === userPermission
  return (
    <Route
      {...rest}
      render={props =>
        isAdmin
        ? <Component {...props}/>
        : <Redirect
            to={{
              pathname: home,
              state: {from: props.location},
            }}
          />
      }
    />
  )
}
<AdminRoute render={() => {
  <Route path="/adminpage" component={ AdminPage } />
}}/>

11/21/18

More Socket Troubles

VM42:1 POST http://staging.uforis.com/socket.io/?EIO=3&transport=polling&t=MSuM5vj net::ERR_CONTENT_DECODING_FAILED

On staging.

VM2702:1 POST http://localhost:3000/socket.io/?EIO=3&transport=polling&t=MSuMABS 404 (Not Found)

Locally.

09/26/18

Reading Redux docs thoroughly

Key notes:

  • Action creators are functions that create actions. The two terms can be easily conflated.

Action:

const run = {
  type: RUN
}

Action creator:

const running = speed => {
  return {
    type: RUNNING,
    payload: { speed }
  }
}
  • Here is a good example of what kind of pattern to follow for actions: https://github.com/redux-utilities/flux-standard-action

  • I didn't realize connect(bindActionCreators) in a container was adding the store.dispatch to the action that's being passed in to the component.

09/25/18

Promises Pt. 2

Going back to promises, I like them but I also don't. They're great but they're sometimes hard to wrap my head around. Anyway, I have to iterate over a bunch of async functions and wait for the end. So I'm going to try to use Promise.all.

results.forEach(project => {
  connection.filter('Group', {projectId: project.id}, 'projectId', true, 0, null, (err, results) => {
    if (err) {
      reject(err)
    }
    addProjectsGroups(payload, results);
  })
})
resolve(payload)
Promise.all(results.map(project => new Promise(resolve => {
  connection.filter('Group', {projectId: project.id}, 'projectId', true, 0, null, (err, results) => {
    if (err) {
      reject(err)
    }
    addProjectsGroups(payload, results);
    resolve();
  })
})))
.then(() => {
  resolve(payload)
})

This seems to work.

  1. Change forEach to map so that we get a list returned
  2. Promise.all this list
  3. add a new Promise into the function chain and resolve() once we're done with the async steps.

09/12/18

Webpack 4 Upgrade Notes

Todo: make this list easier to read, in a rush.

  • Add JS transpilers (babel). use @babel/name over babel-name repos since thats the new convention
  • Add CSS transpilers.
  • Add HtmlWebpackPlugin for html to move over to dist + inject with webpack js.
  • Add webpack.DefinePlugin for defining globals that are used in the app.

09/08/18

Websockets and HTTPS

Issue #1: upgraded to HTTPS, now the websocket connection isn't working.

VM845:1 POST http://localhost:3000/socket.io/?EIO=3&transport=polling&t=MMx89HA 404 (Not Found)

Switch from http to https. running the socket.io-client with just io() will set the address to be the same as the get request (which is http).

this.socket = io('https://localhost:3000')

Issue #2: VM845:1 GET https://localhost:3000/socket.io/?EIO=3&transport=polling&t=MMx8bWK net::ERR_CONNECTION_CLOSED

Switching from http to https, I needed to switch also to port 3001 since 3000 was for http.

this.socket = io('https://localhost:3001')

Issue #3: VM587:1 GET https://localhost:3001/socket.io/?EIO=3&transport=polling&t=MMx7ZJ7 net::ERR_CERT_AUTHORITY_INVALID

I need to trust the certificates we're using.

Turns out the way the server was being created was substantially different, so I'll need to wait for someone else to change their code. This isn't really solved.

08/15/18

React Middleware

Adding middleware is pretty simple but you need to follow a function signature that is kinda odd:

export default store => next => action => {
  // do things
  // maybe use action.type to check against the action being passed through the reducers

  next(action) // need this part to go to the next middleware
}

I completely forget what this was needed for.

07/31/18

CSS Modules Globals

To load CSS modules to apply to all pages, load your styles on the root component and use the :global decorator. Otherwise, they're all local by default and have to be referenced directly to be applied.

07/23/18

Simple React error messages:

Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: undefined. You likely forgot to export your component from the file it's defined in, or you might have mixed up default and named imports.

Issue:

import FuseMessage from '../../../client/src/@fuse';

Solution:

import {FuseMessage} from '../../../client/src/@fuse';

07/03/18

Webpack config and loaders/presets

Learning a lot of good things from this lecture: https://www.youtube.com/watch?v=h7nYAjY7O2M

Some takeaways:

Any presets for webpack will need to be installed separately as a peer dependency. e.g.:

If this is our config file:

const path = require('path');

module.exports = {
  module: {
    rules: [
      {
        test: /\.jsx?$/,
        loader: 'babel-loader',
        include: path.resolve(__dirname, '../../'),
        exclude: /node_modules/
      }
    ]
  }
};

babel-loader is our preset and we need to install it to our project to use it.

Good loaders:

JS/JSX: babel-loader-env and babel-loader-react. One transpiles to older versions of JS, the other js/jsx react code to js code.

SCSS/CSS: sass-loader loads sass files node-sass transpiles sass files css-loader adds additional functions to css files style-loader makes it able to load css styles into the style tags.

Files: file-loader maintained by webpack and moves files over.

Configuring babel:

You can create a .babelrc to load certain loaders, but if you only use it with webpack, you can define the loaders in the webpack config instead.

.babelrc example file for reference:

{
  "presets": ["env", "react"]
}

06/29/18

Cmd + Hover in VS Code for Lookup Definition

On hover of something like CardBody in

import CardBody from "components/Card/CardBody.jsx";

We can't currently go to the definition because component isn't a node_module. To fix this, you must:

  1. Add a jsconfig.json file to the root of your project with the path.
  2. Add the compatibility of searching for jsx files and what compiler it uses (I think).

1:

{
  "compilerOptions": {
    // This must be specified if "paths" is set
    "baseUrl": "./src",
    // Relative to "baseUrl"
    "paths": {
      "components": [
        "./components/*"
      ]
    }
  }
}

2:

{
  "compilerOptions": {
    "target": "esnext",
    "jsx": "react"
}

Final result:

{
  "compilerOptions": {
    "target": "esnext",
    "jsx": "react",
    // This must be specified if "paths" is set
    "baseUrl": "./src",
    // Relative to "baseUrl"
    "paths": {
      "components": [
        "./components/*"
      ]
    }
  }
}

You can reload the file by going to a file within the project (don't be in the jsconfig.json file for whatever reason) and Cmd+Shift+P -> Javascript: Reload Project

Command line tips - go to last directory

cd -

Music organization - Setting up mac with apollo-api

Prereqs:

FLAC, LAME, Mktorrent, Sox

brew install flac
brew install lame
git clone https://github.com/Rudde/mktorrent.git
cd mktorrent
make
make install

Download Sox, enter the folder and mv sox /usr/local/bin to put the binary in your $PATH.

Install

git clone https://github.com/6x68mx/apollo-cli.git
cd apollo-cli
pip3 install -r requirements.txt

Command:

python3 ~/code/personal/apollo-cli/apollobetter.py --search-dir ~/Torrents/Apollo/Music -o ~/Torrents/Apollo/Music --torrent-dir ~/Torrents/Apollo/Watch

05/10/18

Music organization - Moving Multiple Folders

Requires finding the folders using wildcards and then executing a shell command (move) on it afterwards.

find . -name "*FLAC*" -exec mv {} ../../FLACs\ \(172GB\) \;

05/16/18

Squarespace's parallax scrolling and optimal header size Part 2: The height

If you need to give perfect dimensions to the designer, you'll need to put in your mock 2500 width image, and inspect the parallax image's dimensions in return to math out the ideal height:

  • Options+CMD+i to open inpector.
  • Navigate to the div with Parallax-host-outer class > .Parallax-host > .Parallax-item > figure > img.
  • Check the image's width and height elements.

For example, mine is 2840.62px x 909px.

Mine is being upscales from 2500x800 to the height 909 and scales to 2840 width and blurs my image.

Mine needs a height and width of 2500px x 909px. Yours may be different based on the height of your parallax section.

05/09/18

Squarespace's parallax scrolling and optimal header size

I'm sure many people run into this same issue I had. I want to use an image that parallax scrolls that has a predictable size so I can be sure that part of the image isn't cut off.

Dealing with it for long enough, I've come to some conclusions:

They turn the image into a variety of formats. One of the key formats is 2500w. So 2500 across. If I format my image to be that size across, we should have minimal quality degradation.

Another thing, is that at full screen (atleast for me at 1080p), Parallax scrolling with smart crop turned on, we see that it gets fixed to 1991.69px across. That's a bit odd, and larger than 1920 across.

So now we know a large image 2500 across is getting scaled down to 1991px. If you start with a 1991px image and upload it, it will grow to 2500 and then scale back down to 1991px. If you are after pixel-perfect positioning, this should be the best solution.

Now for highest quality imagery, I'd output your image as 2500 across, upload it, and then just know that it will be scaled down to 1991px.

One caveat is that with different screen sizes, browser window sizes, and devices, this will never be 100% accurate to the positioning you want.

05/03/18

How to Get Used to Someone Elses CSS

I'll start with what to do, and then list the things not to do.

What to do: Take the code that you're trying to learn and try to move it around. Make sure your assumptions are correct about what parts of the code are doing what.

I find this fairly simple approach instead of trying to code right away helps you understand the code, and potentially be able to make changes so that the code is stronger. We can only assume that the code is written without perfect design, but can strive towards that it can have better design.

What not to do: Take other components and then inject it into your own area of implementation. Copy the styling and try injecting it into your own implementation.

The issue here is that unless you have a good understanding of what the styling is going to accomplish and how it interacts with the components around it, you are only guessing a what it should be doing.

Receiving no response back from your Domain, file.ico 0 ()

GET http://[publicip]:3000/vendor.b2e84ee1342f9dec42d1.js 0 ()
[publicip]:3000/favicon.ico:1 GET http://[publicip]:3000/favicon.ico 0 ()

This is something I've had happen on a few rare occasions. I believe this is from the HOST being missing when setting up my server.

05/02/18

Selecting the Right Date format can save you a lot of headache

Rule of thumb to take away from this: DATETIME should be used if you're going to ever do math on it like comparing the times. TIMESTAMP should be used if we care about displaying the time in relation to the person's timezone.

Our issue is around storing a datetime as a timestamp. It is being stored as UTC (datetime) which is 7 hours in the future. When it says what that value is in our timezone (timestamp), it'll say it is in 7 hours from now.

I believe our fix for this will be changing the mysql table to be datetime instead of timestamp.

04/30/18

Package-lock's Optional Flag

My package-lock.json mysteriously pops up with "optional": "true", peppered throughout it. How did this happen?

I checking my terminal history from before I left work the other day and discovered the only different was that I did a production install (npm i --production). While the NPM documentation doesn't mention why this appears, I'm confident enough in my assumption that this marks what packages are not needed in the production install and that I can safely commit this change.

04/25/18

Webpack Configurations To Keep

Webpack is sort of confusing. When I think of it as a thing, I imagine this obese monster that has characteristics of the things it wanted to incorporate into its being.

In trying to figure out what webpack modules we even have, I've narrows it down to: style-loader, css-loader, hmr (hot module reload), and cssnano.

cssnano is included in newer versions of css-loader, so that can completely go.

Also when referencing these loaders, we don't need the -loader suffix. I don't see where this is documented, but figured it out after reading enough code.

css

This allows us to require() and @import in css. I'm not sure the benefit, seems to allow us to do mixins?

loader

This allows us to require() external libraries into front end javascript. Usually can only do that with backend (server-side) javascript!

I don't think we're including anything, so this might be superfluous.

style

I think this allows us to include css via javascript. It will load it into the page. Handy with jsx! We use it on a per-component basis.

04/20/18

New keyboard shortcuts with VS Code

Cmd + click on a method to find its definition.

New: Ctrl + - to go back to original line.

Cleaning up React Lifecycle methods

Upgrading to React 16? This introduces the deprecation of componentWillMount, componentWillUpdate and componentWillReceiveProps.

This is where you'll want to move code within those methods to another method!

This is where the React Lifecycle comes in handy:

On initialize:
getDefaultProps() // Gets called on obtaining props (is only called once)

getDefaultState() // Gets called on obtaining state (is only called once)

componentWillMount() // TO REMOVE

render()

ComponentDidMount()
On State Update:
shouldComponentUpdate(nextProps, nextState) // We have this.props and this.state for current props, and the next props and state are being passed in

componentWillUpdate(nextProps, nextState) // We have this.props and this.state for current props, and the next props and state are being passed in

render()

componentDidUpdate(prevProps, prevState) // We have this.props and this.state for current props, and the previous props and state are being passed in since this is after we re-rendered with the new props/state.

So to handle the three changes, I have created some rules:

1. componentWillMount()

If it doesn't matter if we do these functions until after the render, then move to componentDidMount().

If we need values that are being used in the render function, we can still move to componentDidMount(). Create a conditional on rendering only once we have the values we were previously waiting for.

2. componentWillReceiveProps(nextProps)

Should be replaced with safer getDerivedStateFromProps: https://reactjs.org/docs/react-component.html#static-getderivedstatefromprops

(TBC)

04/16/18

Three.JS Helpers - Display Axes

This is actually really easy to implement:

var axesHelper = new THREE.AxesHelper( 5 );
scene.add( axesHelper );

Axes Helper

04/09/18

What You Never Need in a React Lifecycle

(If I make a title or sub-title a good summary, you'll read this more right?)

Argument against: componentWillMount use componentDidMount.

In my case, this was the better solution anyways of the two. However, because of future plans for React, componentWillMount is deprecated as of React 16. Can read more here for the three methods that are unsafe for async rendering in React and how to migrate.

Simple conditional React rendering

For some reason I didn't know how to make a component only load once the required props for it are successfully loaded. Here is how:

render() {
  if (this.props.panorama.tour) {
    return <Panorama {...this.props}/>
  }
  return null
}

04/05/18

VS Code Keyboard Shortcuts

https://code.visualstudio.com/shortcuts/keyboard-shortcuts-macos.pdf

I actually got a lot of use out of checking out the Welcome page to VS Code finally. The Interactive Playground on the right side of the page gave some good tips. Things I got out of it that I find immediately useful:

⌥+⌘+[ - Fold code. collapses the section.

⌥+⌘+] - Unfold code.

⌘+] - indent line of code.

⌘+[ - de-indent.

F2 - rename method or variable in entire codebase.

03/26/18

More code tips

We can create defaults for a function for example the contrived code below will set the default for north be 0 when it is not passed in:

function returnNorth(url, north = 0) {
  return north;
}

However, we can set a default when passing in values to a function, like in this case where north is undefined:

const url = '/';
const north;
returnNorth(url, north || 0);

This helps when there is a possibility of north being undefined, but we don't want to create a default on the entire function.

VS Code Shortcut - making a cursor above or below the line you're on

Added Alt+Cmd+up/down to my shortcuts. This lets me make another cursor above where my cursor is (or below).

03/23/18

In terms of bounding boxes, Three.JS Sprites appear to have a circular area. When a raycast is cast against the sprite, it won't contain the outer corners. This is too bad, and may force me to use something that has geometry to get accurate clicks on a sprite.

Also figured out how to use a mouse event's event.pageX and event.pageY to locate distance from the top of the view of a world. This was needed for a raycasting bug where hovering over an object in the scene was not accurate if part of the page was scrolled.

updateMouseCoords: (mouse, event, canvasPosition) => {
  // Subtract the boundary of the canvas
  const mouseX = event.pageX - canvasPosition.left
  const mouseY = event.pageY - canvasPosition.top
  // Not taking into account scroll height that is minused from event.clientY
  mouse.x = ( mouseX / canvasPosition.width ) * 2 - 1;
  mouse.y = - ( mouseY / canvasPosition.height ) * 2 + 1;
},

Creating documentation out of your code

I've started growing my arguments into a function. I realized that I should just take in one param argument and pull out the values from there. Example code:

this.loadPanorama = (url, north) => {
  const cubeMap = ThreeHelpers.getTexturesFromCubeMapFile(url, 6, 2, true)
  this.background = ThreeHelpers.getMeshFromCubeTextures(cubeMap, 100)
  this.background.name = url
  ThreeHelpers.rotateCube(this.background, north)
  this.scene.add(this.background);
}

I could use panorama.url and panorama.north, but how do I know as a new person reading this that those are the only values passed in?

If I use object destructuring, I don't have to change my values throughout my code, and this line of code const {url, north} = panorama shows that the two keys url and north are what I expect to be passed in.

I just change url, north into panorama in the arguments list on line one and added the destructuring line as the first line into the code:

this.loadPanorama = (panorama) => {
  const {url, north} = panorama
  const cubeMap = ThreeHelpers.getTexturesFromCubeMapFile(url, 6, 2, true)
  this.background = ThreeHelpers.getMeshFromCubeTextures(cubeMap, 100)
  this.background.name = url
  ThreeHelpers.rotateCube(this.background, north)
  this.scene.add(this.background);
}

Cool!

03/22/18

Still adding to Mouse events in Three.JS.

Figured out that for Raycasters, we can make a recursive check for raycaster.intersectObjects(scene.children) by setting the second argument to true, e.g. raycaster.intersectObjects(scene.children, true). This helped solve an issue with the scene attaching groups that have sprites attached to them. We were only checking the groups (which do not take up the 3D space) with the first line.

03/21/18

2D floating text in Three.JS

I'm building a texture out of text in Three.JS. This requires drawing text onto a 2D canvas and then copying it into the texture, placing it in the scene, potentially displacing it or changing its size.

So I made this: Creating floating text in Three.JS

Linting

I think I'm going to stop using semicolons. Seems like a bit of unnecessary overhead dotting these i's for no reason.

Click events on objects in Three.JS

I need a way to find out if I clicked on an object such as a button to go to another room.

Adding to Mouse events in Three.JS

03/20/18

VS Code Freezing continued

Disabling prettier doesn't affect the usage of prettifying the document. Uninstalling the package doesn't affect the usage. Is this a built in feature?

Anyway, I got rid of it. ¯\(ツ)

03/19/18

Building up more Three.JS things

Creating a hotspot. Using a Sprite class object to load since these always face the camera, no matter what angle you're facing.

03/16/18

VS Code Freezing

Reinstalled and it still had the same issue. I disabled my latest extension, Prettier, to see if it would stop it.

03/15/18

Accessing files from S3 in Three.JS (i.e. JavaScript)

Looks like we just need to enable CORS on AWS. The default rules are:

<CORSConfiguration>
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Which just needs the headers switched to accepting *, which is any header:

<CORSConfiguration>
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

In our production environment, we've a different S3 bucket, which I'll allow our domain in AllowedOrigin so nothing can access S3 other than that. If I wanted to be generous and allow access to localhost (and the world, but sort of hidden), I could leave the AllowedOrigin to be *.

03/13/18

Hidden bugs in Three.JS + React

Using the Three.JS methods in React seem to cause for silent errors which makes console.log debugging very tedious...

Something like this

componentDidMount() {
    this.initScene = () => {
      this.scene = new THREE.Scene()
    }
    this.initObjects = () => {
      const geometry = new THREE.BoxGeometry(1, 1, 1)
      const material = new THREE.MeshBasicMaterial({ color: '#433F81' })
      this.cube = new THREE.Mesh(geometry, material)
      this.scene.add(cube)
    }
    this.initScene()
    this.initObjects()
...
}

Will cause no errors to report, but the cube being added should be this.cube. Sentry doesn't catch it as a client-side error, the server doesn't catch it, it simply makes a black screen. I'll need to figure out how to listen for the errors if I don't want further troubleshooting and experimenting to be this painfully slow.

03/07/18

Created Three.js article.

Unsuccessfully adding a cubemap background from a single image, I figured to start from the basics and create this walkthrough.

03/06/18

Creating a Cubemap Background in Three.JS

I have a single image with a 3x2 grid of the images I want as the panorama of my scene.

The regular way to load an image in the background is by setting this.scene.background to the color, Texture, or CubeTexture. In this case, we want a CubeTexture.

03/06/18

Working with Three.JS. Using the docs to create a scene and Rachel Smith's tutorial, I'm trying to render just a basic scene.

I ran into issues using Rachel Smiths as she skipped over the render method, but she does include it in her final code so I binary searched and figured out what was different in our code. I would recommend sticking to the Three.JS docs basic tutorial which is basically the same steps, but if you want to get a deeper understanding for how the scene is built, read hers as it has a good breakdown of what each thing does.

03/01/18

After uncovering an issue with cyclical dependencies in my code, I'm trying to plot out my require statements as a graph.

Testing out dependency trees with the npm module madge and application graphviz.

02/27/18

Advanced replacement technique in VS Code

I want to replace "-1" with -1 to -n where n is the amount of errors we're sending back in a file that has -1 as the err response.

  • Cmd + F to pop up search

  • Fill in -1 as the value to search.

  • Press the arrow on the left of the search bar to access replace. Fill in -2 for replace.

  • Cmd + Shift + 1 to replace only the next occurrence with -2.

  • Replace -2 with the next number, -3 in the replace field. Hit Cmd + Shift 3. Repeat.

02/21/18

There's a cross-editor extension that keeps track of how many LOC you change on a certain type of language.

While I don't think this really an accurate representation of how good of a programmer you are, I think it'll be fun to see the stats.

I've installed it for VS Code right now to further persuade me to work in that editor.

https://codestats.net/users/tendermario

02/20/18

Added: http://wiki.mitchellburton.ca/vscode/

Added: http://wiki.mitchellburton.ca/mac/

Added: http://wiki.mitchellburton.ca/sublimetext/

Added: http://wiki.mitchellburton.ca/terminal/

Added: http://wiki.mitchellburton.ca/mysql/

02/16/18

Turning callbacks into promises

Most of our codebase uses callbacks in this fashion:

function callback = {//next set of instructions};
thing.build(params, callback);

Our newer features are being built with promises to take advantage of an easier to read flow:

thing.build(params)
  .then(() => {//next set of instructions});

You're able to turn the first one into the second one by creating a Promise wrapper. There are a few easy issues that you can run into if you've never done this before.

We basically want the promise to return information at the end of the function. How do we access the end of the function? The callback is executed at the end of the function and typically passed an error if there was any issue, and maybe results. We can wrap this.

Issue 1: turning the callback into a Promise

function promiseCallback = (err, results) = new 
Promise((resolve, reject) => {
  resolve(err, results); // this will return a promise to where the callback is executed. Since there could be a large amount of complexity and this callback is being run within something that isn't a promise, the result of 
});

thing.build(params, promiseCallback).then(() => {
  // then is not a function build, as build is not currently returned as a promise object
});```

We cannot chain this so we can execute something after the function.

There were probably more issues I ran into, but this was the main one that I had issues with.

Solution:

Create a function as a Promise wrapper that contains the callback, which in turn has access to the Promise wrapper's resolve/reject functions and can successfully return a promise to the surrounding code.

const promiseExecution = (params) => new Promise((resolve, reject) => {
  // Generic promise callback
  const callback = (err, results) => {
    if (err) {
      return reject(err);
    }
    return resolve(results || task);
  }
  thing.build(params, callback);
});
return promiseExecution(params);

We can now chain a promise after the promiseExecution function, or we can let that Promise return to its surrounding function and chain a .then() off of it.

02/09/18

throwing errors and have our Error Reporter see it even though something else is reporting the uncaught rejection.

Sequelize decides to swallow every error within its ecosystem and deliver it to the server as not a thrown error but as an uncaught promise rejection.

This is good if we depend on console.logs for our errors to be reported. Since we use Sentry.io to report errors in our API and client, this isn't useful.

In our API, specifically, we watch for uncaughtException (i.e. thrown errors that make the server crash) and errors that are elegantly passed through our framework as next(err). Since Sequelize reports internal errors in neither of these ways, they are quietly happening, only logging to some log file on the server the application is hosted on.

Solution:

Really hidden in Sentry's documentation, they have an options object with key/value captureUnhandledRejections: true to pass to the config() function:

Raven.config('https://ce215826869f43cd89774fe47a8842c9:[email protected]/245925', {
  captureUnhandledRejections: true
}).install();

Now it will find these rejections.

Alternative solution:

If you want your promise rejections to crash the server (why not, crashing the server on error isn't always a bad thing), then you can use the dangerous process.on('unhandleRejection') method, where you throw this error, and as a result, the thrown error goes to Raven, and your server also crashes.

process.on('unhandledRejection', function (err) {
   throw err;
});

02/08/18

'this' context in a prototype

Task.prototype.start = () => {
  return this.update(); // this returns {}
}

This failure may be blatantly obvious to some, but since there's a fat arrow function, the 'this' isn't referring to the immediate parent class (the instance of this class), but the one above it. Here it will be an empty object {}. What it's referring to? I'm not sure.

Switching it to a regular function expression fixes the this reference

Task.prototype.start = function () {
  return this.update(); // this returns instance of task
}

What is public static vs public?

Reading the docs for the MYSQL model ORM we use, Sequelize, it contains those large class definitions that are always confusing.

Inside, it defines .update as both a public static method and a public method. Which one am I using? They return different results with their promises.

Public Static does not require an instantiation of the class.

Public is of an instance of the class.

Since I'm using .update on the instantiation, it's the public class.

Task.findOne({where: {status: 'pending'}})
  .then(task => {
     task.update(); // This is the instance that we found when querying the class
  });

await / async is really constrained

I'm trying to pull the value out of a promise that executes like:

Task.start = () => {
  return Task.findOne({where: {status: 'pending'}})
    .then(task => {
      return task.update({status: 'started'}); // <- operation I want the return value from
    }, err => {
      throw err;
    });
};

To find out the return value of Task.start() since it's an async function, I thought I'd try to use async / await.

Rule 1: await must be used in an async function block

You can't use await anywhere you'd like. It seems as if the async function block wraps a section of code that obtains the ability to execute await.

Incorrect usage:
const value = Task.start();
console.log('value' + await value);

A very simple wrapper function basically executes exactly what I want and returns the value in the following .then().

Correct usage:
async function start() {
  return await Task.start();
}
start().then(value => { console.log('value' + value) });

Simplified since the value gets passed as an argument to the cb function defined in .then(cb);, we can do:

async function start() {
  return await Task.start();
}
start().then(console.log);

Rule 2: This only is useful when you don't have access to the original function being executed.

I have access to the original function right above it, and a .then() run on that function will do the same thing. So this was all a waste, since I could have simply done:

Task.start = () => {
  return Task.findOne({where: {status: 'pending'}})
    .then(task => {
      return task.update({status: 'started'});
    }, err => {
      throw err;
    })
    .then(console.log);
};

Task.start();

This does the same thing, returns the output I want.

Rule 2 redux: Actually you can do this even without invading the original function

Task.start = () => {
  return Task.findOne({where: {status: 'pending'}})
    .then(task => {
      if (task) {
        return task.update({status: 'started'});
      }
      return;
    }, err => {
      throw err;
    });
};

Task.start()
    .then(console.log);

This time, the .then() is tacked onto the executed function, so we can leave this test code even further decoupled from the original function. This seems like a good end point, and not even close to the original code I was attempting to create.

I basically wanted to use the async / await functionality where it actually isn't the right use case, so as a result, I didn't use it.

02/06/18

"button" elements in React give Grief

Issue:

I recently styled our website and added in some <button> elements to get with the times. Then our register button kept refreshing the page whenever I tried to register.

Solution:

Given the button MDN docs, the "type" of the button by default is submit. Since we're not needing to submit the form by a regular HTML way and instead using actions and reducers in Redux, we want to use type="button" which essentially disables any built-in submitting action.

</button>