Miscellaneous

Last updated: May 28, 2024
Tags: [  misc  tips  ]

A collection of stuffs that I am interested in and something I had to search more than once.

MacOS tips

GitHub

  1. Removing sensitive data from a repository
  2. Resetting remote to a certain commit

Virtual Machines

“Virtual machines are software computers that provide the same functionality as physical computers. Key files that make up a virtual machine include a log file, NVRAM setting file, virtual disk file, and configuration file.”

  1. VirtualBox
  2. VmWare
  3. Vagrant

Tips for professionalism

  1. Ten simple rules for writing and sharing computational analyses in Jupyter Notebooks
  2. Cookiecutter Data Science
  3. Principles for making things for the web
  4. Setup Keychain for SSH agent
  5. Jupyter Notebook Extensions
  6. Dealing with path in different OS
  7. PyAutoGUI: a cross-platform GUI automation Python module, used to programatically control the mouse and keyboard. This module can be applied to automate jobs with Python scripts.
  8. Auto format excel file from Python with XlsxWriter: no longer need to manually check and highlight the results in Excel. With xlsxwriter, we can format highlights (bold, font color, background color, etc. ) for the excel output.

  9. TOC generator in SublimeText

    • Enable TOC:
      • Press Shift + Command + P to open Command Palette.
      • Type TOC and choose Insert TOC
    • Modification of TOC:
      • levels="2,3" –> Level of headers to be display in TOC
      • autolink="true" –> Link with the part
      • style="ordered" –> Add the number before header
    • Update TOC: Open Command Palette and choose Update TOC

  10. Github Protips:

    Examples:

    svn ls https://github.com/clauswilke/dviz.supp
svn ls https://github.com/clauswilke/dviz.supp/branches/master/data/


  11. Set up dotfiles

Websites for Machine Learning

  1. Distil

    A powerful medium to share new ways of thinking.

  2. Tensorboad from a remote server

Command lines for an easy life

  1. Find files containing specific text 
    grep -Ril "text-to-find" <directory where to find>

    Learn more here.

  2. Crontab: Allow tasks to be automatically run in the background at regular intervals by the cron daemon. Example: To remove .DS_Store (a file that stores custom attributes of its containing folder in MacOS) every hour 
    @hourly find ~ -name ".DS_Store" -depth -exec rm {} \;

  3. Privacy: Change directory and file permissions.

  4. Youtube downloader

  5. Mount/Unmount: To mount: 
     # at your local machine
 mkdir <new_folder>
 sshfs <usrnam>@<hostname>:<path/to/folder> <path to new_folder>

    To unmount with umount, we need super user permission. As a normal user, you can use fusermount: fusermount -u <mountpoint>

    Issues:

  6. Convert .mov|.MP4 to .gif
    • Convert & Optimize: https://gist.github.com/dergachev/4627207
    • Endless replay: https://superuser.com/a/1017674/1186435
    • Optimizing memory: gifsicle -O3 animation.gif -o animation-optimized.gif (link)
  7. Check if port is in use: link 
    #List all processes listening from all ports
lsof -i -P -n | grep LISTEN 
netstat -tulpn | grep LISTEN
# List processes listeing to specific ports
lsof -i:<port>
    • Kill process listeing from port: 
      lsof -ti:<port> | xargs kill -9
  8. Check PID of process 
    ps ax | grep <something>

  9. Rename multiple files https://www.cyberciti.biz/tips/renaming-multiple-files-at-a-shell-prompt.html

    POISIX for shell loop:

     for i in *.bak; do mv -v -- "$i" "${i%.bak}.txt"; done

    The above command will change the extension of all bak files to txt files.

  10. Replace string

    Pattern: ${parameter/pattern/string}

    Example:

    mkdir test
for i in {0..5}; do touch test/test_file"$i".txt; done && ls test
for i in $(ls test); do echo ${i/.txt/_replaced.txt}; done # replace .txt by _replaced.txt
  11. Optimize the size of pdf files 
    gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -dPDFSETTINGS=/screen -dCompatibilityLevel=1.4 -sOutputFile=<NameOfOutputFile> <PATH/TO/INPUT>
  12. Update VSCode 
    sudo apt update
sudo apt install code

Latex

  1. Getting to know about directory structure of Latex: https://tex.stackexchange.com/a/184430/220467
  2. List of LaTeX mathematical symbols: https://oeis.org/wiki/List_of_LaTeX_mathematical_symbols

Visualization

  1. Graph gallery This page is a collection of beautiful charts with the R programming language. It also supplies many materials for visualizsation task.
  2. Animations in Jupyter Notebook: http://louistiao.me/posts/notebooks/embedding-matplotlib-animations-in-jupyter-as-interactive-javascript-widgets/
  3. Save Matplotlib Animations as GIFs: http://louistiao.me/posts/notebooks/save-matplotlib-animations-as-gifs/

Jokes

  1. Statistics Jokes: A gallery of statistics jokes by Dr. Ramseyer Yihui Xei also gave his comment about those jokes in a presentation: link

Data Science Workflow

  1. Dask

Cheat sheet

Vim

Tmux

  • Prefix: ^b

  • Enable mouse scrolling in tmux: press prefix + :, then:

    On linux

    • For tmux v < 2.1: setw -g mode-mouse on
    • For tmux v >= 2.1: setw -g mouse on

    On MacOSx: can use set instead of setw

  • Capture tmux scrollback to a file: link
    • prefix + :, then type: capture-pane -S -3000 + return
    • prefix + :, then type: save-buffer path/to/logfile + return

Conda virtual environment

Python tips

  1. slots: The special attribute slots allows you to explicitly state which instance attributes you expect your object instances to have, with the expected results:
    • faster attribute access.
    • space savings in memory.
  2. Install a jupyter kernel inside an virtual environment 
    python -m ipykernel install --user --name <name of virtualenv>
  3. Show description of a function/module, type: 
    help(<name of function/module>)
  4. Quick GPU test for Tensorflow 
    srun -J "$F" --pty --gres=gpu:1 -c 4 python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

R tips

  • Install Jupyter Notebook support for R 
    conda install -c r r-essentials
  • Update R & transfer old installed packages to new version https://stackoverflow.com/a/17625368/11524628

Templates

Presentations

Design

Shortcuts

  • Mathpix Snapping Tool:
    • Ubuntu: Ctrl + Alt + M (ensure Mathpix is active)
    • MacOS: Ctrl + ⌘ + M

Remove Watermark from pdf

Written on December 28, 2019