aws developer associate test IAM quick note

iam

  • components: user, group, role, policy
  • user has many groups
  • role is defined for aws resources
  • policy can be used to define role, user, group access
  • policy is a json file defining one or more permissions

create a user

  1. console
    • need user name and password
    • can reqeust to reset password
  2. programtic
    • access key id, access key secret
    • only able to see the secret when creating the user, need to download the csv for further reference

create a role

  1. can create from a aws resource
    • e.g EC2 giving s3 read permission, any instance has this role attached can access S3

exam tips

  • iam is global, does not relate to region
  • root account is the account first created, shouldn’t use this account normally
  • always use MFA on root account

Numpy学习笔记 (1)

np.array

  • construct ndarray from python array
  • ndim: 返回值有几个括弧
  • dtype: 也是object,指定ndarray的型, 必须整个数列都是一个型的
    a = np.array([1, 2, 3,4,5], ndmin = 2, dtype=complex)
    # => [[ 1.+0.j,  2.+0.j,  3.+0.j]]
    

np.dtype

  • dtype是object而不是常数
  • np.int32是type class,和np.dtype不同
    dt = np.dtype(np.int32)
    

ndarray.shape

  • overwrite shape can reshape the array, reshape后维度不会变, 原来的array会被覆盖
  • reshape会返回新的array
    a = np.array([[1,2,3],[4,5,6]])
    a.shape = (3,2)
    b = a.reshape((6, 1))
    

ndarray.ndim

  • 返回维度(int)

新建np array

  1. np.empty (default dtype float 64)
  2. np.ones (default dtype float 64)
  3. np.zeros (default dtype float 64)
    • shape can be (1, 2) or [1, 2]
    • 如果只指定int,默认是int64

从既存的array创建

  • 原来的array和ndarray一定是两个object
    1. np.asarray (list or tuple)
  • !⚠️tuple的元素对齐返回值不同
    np.asarray(1,2,3),(4,5,6)) #返回二元数组,dtype是int
    np.asarray(1,2,3),(4,5)) #返回一元数组,dtype是object
    

np.arange

  • dtype是int,输入float的时候小数点部分会被切掉
  • endpoint=False会把最后那个数也加进去
    np.arange(5, dtype = float)
    

np.linspace

  • base是log space的底,default是0(线性)
    np.linspace(10,20,5) # 起点,终点,点数
    

slice

  1. use slice method
    a = np.arange(10)
    s = slice(2,7,2)
    
  2. [start:stop:step]
    a[2:7:2]
    

    二元数组的情况下

    a = np.array([[1,2,3],[3,4,5],[4,5,6]])
    a[1] == a[1, :]
    a[:, 1] == a[..., 1] # => True
    
  3. boolean slicing
    a = np.array([[1,2,3],[3,4,5],[4,5,6]])
    a[a > 5] # => 返回一位数组
    

boradcasting

  1. 两个ndarray shape相同时,操作是element-wise
  2. 可以boardcast的情况:形状完全相同,形状(n, 1)和(n, m)的情况

Use `raise_error` matcher in Rspec

Yesterday I came accross a tricky useage of raise_error matcher in rspec, first I try to use the one line syntax is_expected.to

it { is_expected.to raise_error(SomeError) }

but I got an eror which stops the test before the test is finished.

It is confusing in the first time, because is_expected.to is shortcut for expect(subject).

Then I found this excellent explanation in Github isses.

So expect(subject) is not a block, subject is executed instantly, so rspec will stop before evaluating raise_error matcher.

In order to make it work, we can make subject itself a block

subject { -> { raise SomeError } }

it { is_expected.to raise_error(SomeError) }

But this syntax is creepy, instead it’s better to use

subject { raise SomeError }

expect { subject }.to raise_error(SomeError)

Serialize Non-ActiveRecord Model Using ActiveModel::Serializer in Rails

Recently I’ve been working in backend which connects to another API, you may need to send a specific format of JSON data which is not the same as database resource.

Create A Serializable Resource

Currently I’m using active_model_serializers 0.10.0. From the documentation we know ActiveModel::Serializer wraps a serializable resource, to create a serializable resource we need to create a plain old Ruby object inherits ActiveModelSerializers::Model, then create a corresponding serializer.

For example:

class MyModel < ActiveModelSerializers::Model
  def initialize(title:, body:)
    @title = title
    @body = body
  end

  def converted_body
    # do something
  end
end
class MyModelSerializer < ActiveModelSerializers::Serializer
  attributes :converted_body
end

Handling Nested JSON

You can use has_many, has_one method in ActiveModel::Serializer to handle nested json, although it doesn’t mean the same with Rails model relations.

For example:

class MyModel < ActiveModelSerializers::Model
  def initialize(object: object)
    @title = title
    @body = body
  end

  def 

  def related_models
    related_models.map do |related_model|
      RelatedModel.new(related_model: related_model)
    end
  end
end
class MyModelSerializer < ActiveModelSerializers::Serializer
  has_many :related_models
end

ActiveModelSerializers::Serializer will automatically look for a serializer named RelatedModelSerializer.

Reference

Find more explainations in https://github.com/rails-api/active_model_serializers/tree/0-10-stable#what-does-a-serializable-resource-look-like


Plot Financial Data Using Matplotlib (super basic)

import libraries

In order to plot the data, we need to import libraries for dataframe and plot, also set plot inline if you’re using a notebook.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

plot stock price

Code is super simple, you just need to set the index of the dataframe to timestamp. If you find the figure to small, use pylab to set the figure size.

from pylab import rcParams
rcParams['figure.figsize'] = 50,13
apple_data = market_train_df[market_train_df['assetCode'] == 'AAPL.O']
apple_data.set_index('time')['close'].plot(grid = True)

compare multiple plots

Then I’m trying to sanitize the data by looking at the figure it self, I want to plot all suspicious assets.

for code in suspicious_asset_code:
    apple_data = market_train_df[market_train_df['assetCode'] == code]
    apple_data.set_index('time')['close'].plot(grid = True)

This only gives me one figure with all assets, it’s ugly. I want to plot them one by one.

Subplot will give you figures put up together, use

# treat the whole figure as a grid, specify how many rows and columns you wanna plot and the index of the current sub plot
fig.add_subplot(row, columns, index)

Then you’ll get graphs all together to detect abnormal data.


   Tags
life ( 4 )
music ( 1 )
programming ( 14 )
spirituality ( 1 )
中文 ( 2 )
language-learning ( 1 )
python ( 1 )
aws ( 1 )
study-note ( 1 )

About Me

I'm a university student in the last year, also a web developer. I love exporing cultures, learning languages and making BGM music!