Coding fundamentals

Recording

The following is a recording on this topic, I hope you find it useful!

Feel free to watch at 1.5x or 2x speed.

Note: If you find something missing or incorrect in the recording, then please let me know!

Computer code types

  • Markup & Mark-down languages format & structure content (e.g., .html, .md).
  • Scripted languages are interpreted at runtime (flexible & user-friendly but slower)
    • Run via an interpreter: An interpreter is software that reads and executes code line by line, translating high-level code into machine-readable instructions in real-time, enabling dynamic execution and testing.
    • Scripted languages in DSAN include R, Python, and bash (shell script, .sh)
  • Compiled languages (i.e. programming languages) are translated into machine code before execution, offering performance optimization.
    • e.g. C, C++ (cpp), Fortran (still very common in hard sciences), Java

Variable scope

  • Variable scope refers to where a variable is accessible & can be used within a program. It defines the portion of code where the variable’s value is valid and recognizable.
  • Namespace: In computer science, a namespace is a container that holds a set of identifiers (such as variable and function names) to manage organization within a codebase. R uses namespaces to manage variable names, preventing clashes between packages and user-defined variables.
  • Global Scope: Variables defined outside of functions are accessible throughout the entire scrip
  • Local Scope: Variables defined within a function are typically only accessible within that function
  • Encapsulation: Local scope helps prevent unintended variable conflicts & enhances modularity.

IF YOU EVER HAVE WEIRD BUGS IN .ipynb RESTART YOUR KERNEL AND DO “RUN ALL”

Conda

Conda Overview

  • Conda is a package & environment manager used in data science & software development, simplifying installation and management of packages & dependencies.
  • Easy python package control: seamlessly switch between various python interpreters (conda environments), each with different packages, all on the same machine
  • Enables smooth collaboration in data science, research, and development projects.

Why use conda?

  • Sandboxing is isolating software applications from the system to prevent potential harm, enabling safe testing and execution
  • Conda facilitates MULTIPLE python interpreters (environments) for various projects

Common Conda commands

  • You DO NOT need to run these, but they are useful and included as reference
  • Deactivate current environment
    • conda deactivate
  • Activate environment named “myenv”
    • conda activate myenv
  • To list all of your environments, in your terminal window or an Anaconda Prompt
    • conda info --envs
  • To remove an environment named “myenv”, in terminal or an Anaconda Prompt, run:
    • conda remove --name myenv --all
  • Rename and environment
    • conda rename -n old_name new_name

Conda cheat-sheet

Source: click here for original PDF

Common coding paradigms

Running scripts

  • There are MANY ways to run the scripts that you encounter in data science
    • .qmd, .rmd, and .ipynb require human interaction, e.g. manually running code cells.
      • They are great for Communication and education but not for automated tasks
    • .py and .R can be triggered to run automatically.
      • They are better at “hands-off” automated workflows and complicated software development projects
  • Common coding methods
    • .py \(\rightarrow\) just python code \(\rightarrow\) we can run from terminal with python filename.py
      • In this case the python command points to a particular python interpreter
      • Each conda environment is a different interpreter (translator)
    • .R or .Rscript \(\rightarrow\) just R code \(\rightarrow\) we can run from terminal run with Rscript filename.R
      • In this case the Rscript command points to the R interpreter
    • .rMarkdown = Markdown + R-code \(\rightarrow\) we can run using Rstudio
      • Rstudio “points” at the same R interpreter used by the Rscript command
    • .qmd = Markdown + R or Python code \(\rightarrow\) we can run using Rstudio or VSC
    • .ipynb = Markdown + R or Python code \(\rightarrow\) we can run using Rstudio or VSC
      • Can select which interpreter to run the code with

Aside: .ipynb are JSON files

  • An .ipynb file is simply a JSON file with a specialized structural format
  • You can see this by running more eda/eda.ipynb from the command line
  • Which will output the following;

Debugging

Comments on debugging code-1

Remember, effective debugging is often a combination of systematic analysis, creativity, and persistence. Different situations may require different approaches, so adapt these best practices based on your specific context.

  • Google the error message:
    • This is always the first place to start
  • Understand the Problem:
    • Clearly define the issue or unexpected behavior you’re encountering.
    • Reproduce the problem consistently to understand its scope and context.
    • Make sure you understand the data-types involved, the “type()” command is your best friend
  • Isolate the Issue:
    • Narrow down the code section causing the problem to reduce complexity.
    • Comment out or isolate blocks of code to identify the specific cause.
    • Check data types and make sure they make sense
  • Peer Review:
    • Seek input from colleagues or peers who can offer fresh perspectives and ideas.
  • Take Breaks:
    • Sometimes stepping away and returning later can provide new insights and a clearer mindset.
  • Simplify and Test:
    • Strip away unnecessary complexity and focus on the core problem.
    • Create minimal test cases to reproduce the problem in a controlled environment.
  • Check Assumptions:
    • Challenge your assumptions about how certain parts of the code should work.
    • Verify that your understanding of the problem and its requirements is accurate.

Comments on debugging code-2

Remember, effective debugging is often a combination of systematic analysis, creativity, and persistence. Different situations may require different approaches, so adapt these best practices based on your specific context.

  • Check Inputs and Outputs:
    • Verify input data and parameters for correctness.
    • Monitor output values and results to spot discrepancies.
  • Use Debugging Tools:
    • Utilize integrated development environment (IDE) tools like breakpoints, watches, and variable inspection.
    • Use debugging libraries or third-party tools for more advanced debugging.
  • Print and Log:
    • Insert print statements or log messages at critical points to trace code execution and variable values.
  • Search for Documentation:
    • Consult official documentation, forums, and resources related to the programming language, libraries, or frameworks you’re using.
  • Avoid Hasty Changes:
    • Make deliberate changes while debugging, and test each change to understand its impact.
  • Divide and Conquer:
    • Split the code into smaller parts and test them individually to identify the problematic section.
  • Read Error Messages:
    • Carefully read error messages and stack traces to understand where the issue originates.