Data.Trek 2020 / Mar 06 2020

Gabriel Dansereau, AAÉBINUM and

Francis Banville

Master [Julia]

PoisotLab/data-trek-2020master

This repo is mounted by: Julia

1. Data types, data structures and indexing

1.1 Basics

Object, assignment, functions, how to comment and get help

0.6s

Julia

x = 2

0.2s

Julia

x + 2

0.2s

Julia

x * 3 # we can use Julia as a calculator!

0.2s

Julia

log(1) # Functions help us execute things; we usually have to provide arguments

0.4s

Julia

0.0

?log # Don't know how a function works? Ask for help

1.3s

Julia

1.2 Data types

Numeric (integers or doubles)

my_integer = 1 # Naming objects tips

0.2s

Julia

my_float = 1.0

0.2s

Julia

1.0

String

my_string = "blue"

0.6s

Julia

"blue"

Boolean

my_boolean = true

0.6s

Julia

true

1.3 Data classes

Arrays

Arrays are ordered collection of elements.

A 1-dimensional array is a vector.

my_vector = ["blue", "bleu", "azul", "blau"] # 1-dimensional array

2.0s

Julia

4-element Array{String,1}: "blue" "bleu" "azul" "blau"

is_french = [false, true, false, false]

1.9s

Julia

4-element Array{Bool,1}: 0 1 0 0

What happens when we add 1 to a logical vector?

is_french .+ 1

1.4s

Julia

4-element Array{Int64,1}: 1 2 1 1

In Julia, arrays can contain elements of different types.

my_any_array = ["blue", true, 1, false, 3.0]

1.0s

Julia

5-element Array{Any,1}: "blue" true 1 false 3.0

A 2-dimensional array is a matrix.

reshape(my_vector, (lines, columns))			# reshapes my_vector

my_matrix = reshape(my_vector, (2,2)) # 2-dimensional array

1.0s

Julia

2×2 Array{String,2}: "blue" "azul" "bleu" "blau"

Arrays can also contain other arrays.

my_array_of_arrays = [my_vector, is_french, my_matrix]

1.5s

Julia

3-element Array{Array,1}: ["blue", "bleu", "azul", "blau"] Bool[0, 1, 0, 0] ["blue" "azul"; "bleu" "blau"]

Tuples

(name_element1 = element1, name_element2 = element2, name_element3 = element3)

my_tuple = (word = my_vector, french = is_french)

0.8s

Julia

(word = ["blue", "bleu", "azul", "blau"], french = Bool[0, 1, 0, 0])

Data frames

DataFrame(vector1, vector2)	# bind vectors with the same length

using DataFrames

4.0s

Julia

my_df = DataFrame(word = my_vector, french = is_french)

3.0s

Julia

word	french
blue	false
bleu	true
azul	false
blau	false

4 items

1.4 Dimensions

size(matrix) # returns the dimensionslength(object) # returns the number of elements

size(my_vector)

0.7s

Julia

length(my_vector)

Shift+Enter to run

Julia

size(my_matrix)

0.8s

Julia

length(my_matrix)

0.3s

Julia

Can you guess the value of the length of my_array_of_arrays?

size(my_array_of_arrays)

0.4s

Julia

length(my_array_of_arrays)

Shift+Enter to run

Julia

1.5 Indexing

# 1-dimensionalvector[index]# 2-dimensionalmatrix[row, column]dataframe[row, column]# Multi-dimensionalarray[element][...]tuple[element][...]

my_vector[2]

0.3s

Julia

"bleu"

my_df[2, 1]

0.2s

Julia

"bleu"

my_array_of_arrays[1]

0.5s

Julia

4-element Array{String,1}: "blue" "bleu" "azul" "blau"

my_array_of_arrays[1][2]

0.2s

Julia

"bleu"

1.6 Slicing

my_matrix = [34 9; 						 6 5; 						 3 50;             43 27;             98 100]

1.2s

Julia

5×2 Array{Int64,2}: 34 9 6 5 3 50 43 27 98 100

my_matrix[my_matrix[:, 1] .> 5, :]

0.6s

Julia

4×2 Array{Int64,2}: 34 9 6 5 43 27 98 100

2. Files

Absolute paths

C:/Users/RonBumblefootThal/Documents/RFolder/MyFirstProject/Draft/IDon'tKnowWhatI'mDoing/etc.R

Relative Paths

~/I_love_my_project/CoolCode.R

2.1 Working directories

readdir()

0.6s

Julia

27-element Array{String,1}: ".dockerenv" ".empty" ".nextjournal" "bin" "boot" "data-trek-2020" "dev" "etc" "home" "lib" ⋮ "runtimes" "sbin" "shared" "srv" "sys" "tmp" "usr" "var" "volumes"

cd("data-trek-2020")

0.2s

Julia

pwd()

0.2s

Julia

"/data-trek-2020"

readdir()

0.2s

Julia

6-element Array{String,1}: ".git" "LICENSE" "README.md" "code" "data" "output"

2.2 Save/write files

Save a dataframe

soa_tour = DataFrame(country = ["USA", "UK", "FRA", "GER", "BRA"],                      frequency = [34, 9, 6, 5, 3],                      continents = ["north_america", "europe", "europe",                                    "europe", "south_america"])

0.5s

Julia

country	frequency	continents
USA	34	north_america
UK	9	europe
FRA	6	europe
GER	5	europe
BRA	3	south_america

5 items

# Function structureCSV.write(path, object)

using CSV

3.2s

Julia

CSV.write("data/clean/soa_tour.csv", soa_tour)

2.4s

Julia

"data/clean/soa_tour.csv"

2.3 Load/read files

From your PC

object = CSV.read(path)

soa_tour = CSV.read("data/clean/soa_tour.csv")

13.7s

Julia

country	frequency	continents
USA	34	north_america
UK	9	europe
FRA	6	europe
GER	5	europe
BRA	3	south_america

5 items

From url to your PC, then read

download("http://remote.repo/data/file.csv", path)object <- read.csv(path)

Metabolic rates data: http://sciencecomputing.io/data/metabolicrates.csv

download("http://sciencecomputing.io/data/metabolicrates.csv", "data/raw/metabolicrates.csv")

1.4s

Julia

"data/raw/metabolicrates.csv"

metabolic_rates = CSV.read("data/raw/metabolicrates.csv")

1.7s

Julia

3. Control Flow

You already apply control flow when you decide how to go to work during winter.

For example:

You take the metro if it's snowy
You take the metro if it's cold
You walk every other time

Now, let's put that into code!

3.1 Conditional evaluation

Simple if statement

Structure:

if (condition is true)	do expressionend

Example:

weather = "snowy"

0.2s

Julia

"snowy"

if weather == "snowy"  println("Take the metro!")end

0.3s

Julia

weather = "clear"

0.2s

Julia

"clear"

if weather == "snowy"  println("Take the metro!")end

0.1s

Julia

If/else statement

Structure:

if (condition)	expression 1else  	expression 2end

Example:

if weather == "snowy"  println("Take the metro!")else  println("Let's walk!")end

0.5s

Julia

Nested if /else statement

if (condition 1)	expression 1  if (condition 2)    expression 2  endend

Example:

temperature = -15

0.2s

Julia

-15

if weather == "snowy"  println("Take the metro!")else  if temperature < -20  	println("Take the metro!")	else    println("Let's walk!")  endend

0.4s

Julia

Simpler alternative:

if weather == "snowy"  println("Take the metro!")elseif temperature < -20  println("Take the metro!")else  println("Let's walk!")end

0.4s

Julia

Adding a condition:

if weather == "snowy" || temperature < -20  println("Take the metro!")else  println("Let's walk!")end

0.5s

Julia

3.2 For loops

Simple for loops

Using for loops, you can then plan your schedule for a few days.

What we had:

weather = "snowy"temperature = -15

0.1s

Julia

-15

But what about this?

weather_vec = ["snowy", "cloudy", "snowy", "clear", "rainy"]temperature_vec = [-15, -23, -2, -40, 5]

0.2s

Julia

5-element Array{Int64,1}: -15 -23 -2 -40 5

Does the same code work?

if weather_vec == "snowy" || temperature_vec < -20  println("Take the metro!")else  println("Let's walk!")end

1.4s

Julia

Iterations

for i in iterations	expression(i)end

for i in 1:5  println(temperature_vec[i] + 2)end

0.5s

Julia

More generally:

length(temperature_vec)

0.2s

Julia

for i in 1:length(temperature_vec)  println(temperature_vec[i] + 2)end

0.4s

Julia

If statement inside for loops

Structure:

for i in iterations	if (condition) 	 	expression1  else  	expression2  endend

Example:

# Previous statementif weather_vec == "snowy" || temperature_vec < -20  println("Take the metro!")else  println("Let's walk!")end

0.4s

Julia

# Will this work?for i in 1:length(weather_vec)	if weather_vec == "snowy" || temperature_vec < -20	  println("Take the metro!")	else	  println("Let's walk!")	endend

0.5s

Julia

3.3 Extras

Some logical operators

Comparisons:

less than (<)
more than (>)
less than or equal to (<=)
more than or equal to (>=)
equal to (==)
not equal to (!=)

Logic:

not x (!x)
x or y (x || y)
x and y (x && y)

4. Functions

4.1 Syntax and arguments

Basic syntax

function functionname(argument1, argument2)		# Name and arguments 	result = expression		# What the function does  return result # What the function returnsend

# Define a functionfunction temp_difference(temp1, temp2) # Name and arguments  result = temp2 - temp1 # What the function does  return result # What the function returnsend

0.8s

Julia

temp_difference (generic function with 1 method)

# Apply on valuestemp_difference(-5, -15)

0.3s

Julia

-10

# Apply on variablestemperature = [-15, -23]temp_difference(temperature[1], temperature[2])

0.1s

Julia

-8

Calling (personal) functions within functions

function absolute_temp_difference(var1, var2)  result = temp_difference(var1, var2)  abs_result = abs(result)  return abs_resultend

0.8s

Julia

absolute_temp_difference (generic function with 1 method)

absolute_temp_difference(temperature[1], temperature[2])

0.2s

Julia

4.2 Scope

Variables can exist either in global or local scope.

Remember, the element to return in our function was called abs_result.

# What will this return, outside the function?result

0.5s

Julia

Here is a second example for ecologists who like to count living things:

trees = 4 # global variablesquirrels = 10

0.2s

Julia

function count_living_things()	birds = 5 # local variable  squirrels = 20  return birds, squirrels, treesendcount_living_things() # global and local variables returned

0.8s

Julia

(5, 20, 4)

birds # does not exist in global scope

0.5s

Julia

squirrels

0.2s

Julia

4.3 Integration

Combining functions and control flow

Let's come back to our previous example about transportation according to the weather.

Here is the forecast for the week and the weekend:

# Week forecastweather_week = ["snowy", "cloudy", "snowy", "clear", "rainy"]temperature_week = [-15.0, -23.0, -2.0, -40.0, 5.0]# Weekend forecastweather_weekend = ["snowy", "rainy"]temperature_weekend = [-3.0, 2.0]

1.3s

Julia

2-element Array{Float64,1}: -3.0 2.0

Now, let's build a function that will work with either the week or weekend forecasts.

It will look like:

function transportation	for all days of the week/weekend  	if (snowy or cold)    	take metro    else    	walk    end  endend

# Previous for loopfor i in 1:length(weather_vec)  if weather_vec[i] == "snowy" || temperature_vec[i] < -15.0    println("Take the metro")  else    println("Just walk")  endend

0.4s

Julia

function choose_transportation(weather, temperature)  for i in 1:length(weather)    if weather[i] == "snowy" || temperature[i] < -15.0      println("Take the metro")    else      println("Just walk")    end  endend

0.8s

Julia

choose_transportation (generic function with 1 method)

# Plan for the weekchoose_transportation(weather_week, temperature_week)

0.5s

Julia

# Plan for the weekendchoose_transportation(weather_weekend, temperature_weekend)

0.3s

Julia

4.4 Exercise - Planning the week

Exercise to integrate the following:

Functions
Control flow
Files

Write a function to read a file if it exists, downloading it first if it does not exist.
Apply the choose_transportation function to the data in the file

# Pseudocodefunction (file, url)	if (file exists)  	read file  else  	download file    read fileend

# Useful functions?isfile?CSV.read?download

Forecast data url: https://docs.google.com/spreadsheets/d/e/2PACX-1vSdqGDzfGPygowYRaffMTsVHQz1nejPPyjE2Q1yYIRKPUfhayMTcCMhdzqfbea5IeYKi82aW4NDas_G/pub?gid=0&single=true&output=csv

function read_if_exists(filename, url)  if isfile(filename)    CSV.read(filename)  else    download(url, filename)    read_if_exists(filename, url)  endend

0.8s

Julia

read_if_exists (generic function with 1 method)

using CSVfilename = "metabolicrates.csv"url = "https://docs.google.com/spreadsheets/d/e/2PACX-1vSdqGDzfGPygowYRaffMTsVHQz1nejPPyjE2Q1yYIRKPUfhayMTcCMhdzqfbea5IeYKi82aW4NDas_G/pub?gid=0&single=true&output=csv"read_if_exists(filename, url)

8.2s

Julia

forecast = read_if_exists(filename, url)

0.6s

Julia

choose_transportation(forecast.weather, forecast.temperature)

Shift+Enter to run

Julia

4.5 Extras

Default values

# Define functionfunction add_and_multiply(var1, var2; var3 = 1)  result = (var1 + var2) * var3  return resultend

0.8s

Julia

add_and_multiply (generic function with 1 method)

add_and_multiply(1.0, 2.0) # multiplies by 1, as default

0.3s

Julia

3.0

add_and_multiply(1.0, 2.0, 2.0) # returns error

0.6s

Julia

add_and_multiply(1.0, 2.0; var3 = 2.0) # proper syntax

0.2s

Julia

6.0

Controlling return

function add_and_multiply(var1, var2; var3 = 1)  addition_result = var1 + var2  multiplication_result = addition_result * var3  return (added = addition_result, multiplied = multiplication_result)end

0.2s

Julia

add_and_multiply (generic function with 1 method)

add_and_multiply(1.0, 2.0, var3 = 2.0)

0.8s

Julia

(added = 3.0, multiplied = 6.0)

Master [Julia]

1. Data types, data structures and indexing

1.1 Basics

1.2 Data types

Numeric (integers or doubles)

String

Boolean

1.3 Data classes

Arrays

Tuples

Data frames

1.4 Dimensions

1.5 Indexing

1.6 Slicing

2. Files

Absolute paths

Relative Paths

2.1 Working directories

2.2 Save/write files

Save a dataframe

2.3 Load/read files

From your PC

From url to your PC, then read

3. Control Flow

3.1 Conditional evaluation

Simple if statement

If/else statement

Nested if /else statement

3.2 For loops

Simple for loops

If statement inside for loops

3.3 Extras

Some logical operators

4. Functions

4.1 Syntax and arguments

Basic syntax

4.2 Scope

4.3 Integration

4.4 Exercise - Planning the week

4.5 Extras

Default values

Controlling return

Runtimes (1)