Notes Certified Solutions Architect - Associate - Part 1/2

I have decided to share all my notes that I have written when I was preparing for AWS certified solutions Architect – Associate.
In this article “Part 1 of 2”, you will find notes related to these AWS services: IAM, S3, AWS Organization, CloudFront and other storage services.

1. Overview:

A region is a physical location with two or more Availability zones
Availability zone is one or more datacenters.
Edge location: AWS endpoints for caching purpose (CloudFront: Amazon CDN)
Trusted Advisor : great AWS service that checks your account and infrastructure, and provides you with best practices, serivce limits information…

2. Identity Access Management: IAM

Allows you to manage users and their level of access to the AWS console.

- Users: end users
- Groups: group of users
- Policies: permissions (using JSON)
- Roles: assigned to AWS ressources.

3. S3:

Safe place where to store your files (objects), from 0 Bytes to 5 TB

You get an HTTP 200 code when you upload a file successfully .

Objects can have:

Key (name of the object)
Value (data)
Version ID.
Metadata (data about data you are storing)
Subresources: Access control list, torrents.

S3 Naming convention:

No uppercase nor underscore
3-63 characters long
Not an IP and it must start lowercase letter or number

S3 is a universal namespace and it’s unique.

S3 has the following features:

Tiered storage available
Lifecycle mgmt
Versionning
Encryption
MFA Delete (multi factor auth)
Secure data using ACL and bucket policies.
Signed URLs: URLs that are valid only for a limited time (ex: premium video service for logged in users)

S3 storage classes:

S3 standard (99.99% availability, 99.99999999% durability..)
S3 IA (infrequently Accessed)
S3 one zone – IA
S3 Intelligent Tiering
S3 Glacier (for data archiving, 99.999999999% durability of archives )
S3 Glacier Deep Archive (retrieve data in 12hours)

S3 Pricing Tiers:

You pay per:

Storage
Requests and data retrieval
Data transfer
Mgmt & replication

Most expensive: S3 standard, then followed by:

S3 IA
then S3 Intelligent Tiering
then S3 one zone IA
then S3 glacier
and finally S3 glacier deep archive..

S3 Encryption:

Two types of encryption:

Encryption in Transit: SSL/TLS
Encryption at Rest (server side):
- S3 managed keys -SSE -S3,
- AWS Key Management Service,
- Server side encryption with customer provided keys SSE-C
Client side encryption

S3 Security:

User based: IAM policies.
Resource base:
- Bucket policies, used to:
  - Grant public access to the bucket
  - Force a bucket to be encrypted at upload
  - Grant access to another account (Cross Account)
- Object ACL,
- Bucket ACL.

IAM Roles vs Resource Based Policy:

When you assume a role (user, application, or service) you give up your original permissions and take the permissions assigned to the role.
An explicit Deny overrides an Allow
If there is an Allow wether in Identity Based Policy or Resource Based Policy and nothing in the order side, the result will be “Allow”.

4. AWS organization and consolidated billing

Allows you to have multiple accounts and consolidate them.
Create new organizations, then invite an account…

Use Service Control Policies (SCP) to:

Whitelist or blacklist IAM actions
Applied at the Root, OU or Account level

Use cases:

Restrict access to certain services (for example: cant use EMR)
Enforce PCI compliance by explicitly disabling services

Moving Accounts:

To migrate accounts from one organization to another:

Remove the member from the old organization
- Send an invite to the new organization
- Accept the invite to the new organization from the member account

If you want the master account of the old organization to also join the new organization, do the following:

Remove the member accounts from the organizations using procedure above
- Delete the old organization
- Repeat the process above to invite the old master account to the new org

5. AWS Resource Access Manager (RAM):

Share AWS resources that you own with other AWS accounts
Share with any account or within your Organiszation
Avoid resource duplication!
VPC Subnets:
- Allow to have all the resources launched in the same subnets
- Must be from the same AWS Organizations.
- Cant share security groups and default VPC
- Participants can manage their own resources in there
- Participants cant view, modify, delete resources that belong to other participants or the owner.
AWS Transit Gateway
Route53 Resolver Rules
License Manager configurations

3 different ways share S3 bucket accross accounts:

using bucket policies and IAM.
using bucket ACLs & IAM.
Cross-account IAM roles.
Create a role / another AWS account / paste ID..

Cross Region Replication
- Create a new bucket, then onglet mgmt / replication /…
- Versionning must be enabled on both src and dest buckets.
- Files in an existing bucket are not replicated automatically
- All subsequent updated files will be replicated automatically
- Delete markers are not replicated
- Deleting individual versions or delete markers will not be replicated
- use cases: compliance, lower latency access, replication across accounts

Tips:

Policy generator: https://awspolicygen.s3.amazonaws.com/policygen.html

S3 CORS:

If you request data from another S3 bucket, you need to enable CORS.
Cross Origin Resource Sharing allows you to limit the number of websites that can request your files in S3 (and limit your costs)

Consistency Model

Read after write consistency for PUTS of new objects:
- As soon as an object is written, we can retrieve it, ex: PUT 200 -> GET 200)
- This is true, except if we did a GET before to see if the object existed (ex: GET 404 -> PUT 200 -> GET 404) – eventually consistent
Eventual Consistency for DELETES and PUTS of existing objects
- If we read an object after updating, we might get the older version (ex: PUT 200 -> PUT 200 -> GET 200 (might be older version))
- If we delete an object, we might still be able to retrieve it for a short time (ex: DELETE 200 -> GET 200)

S3 Transfer acceleration:

Users upload to edge, then transfer to AWS backbone.
Check url on google for speed of S3 Acceleration:
https://s3-accelerate-speedtest.s3-accelerate.amazonaws.com/

6. CloudFront: Amazon CDN

Edge location: location where content will be cached (216 points of presence globally).
origin: origin of all the files that CDN will distribute: S3 bucket, EC2 instance…
distribution: name given to the CDN.
web distribution: for web site.
RTMP: used for media streaming.
you can also write on edge location.
DDOS protection, integration with Shield, AWS Web App Firewall.
Can expose external HTTPS and can talk to internal HTTPS backends.
CloudFront is the best way for static content (updates, images..), more scalable and cheaper.
Field-level encryption allows you to enable your users to securely upload sensitive information to your web servers (Personally identifiable information (PII)

CloudFront Geo Restriction:

Can restrict who can access your distribition by using whitelist or blacklist.
Country is determined using a 3rd party Geo IP database
Use case: copyright laws to control access to content

CloudFront vs S3 Cross Region Replication (CRR):

CloudFront Signed URL / Signed Cookies:

You want to distribute paid shared content to premium users over the world
We can use CloudFront Signed URL / Cookie. We attach a policy that:
- Includes URL expiration
- Include IP ranges to access the data from
- Trusted signers (which AWS account can create signed URLs)
How long should the URL be valid for?
- Shared content (movie, music): make it short (a few minutes)
- Private content (private to the user): you can make it last for years

Signed URL: access to individual files (one signed URL per file)

Signed Cookies: access to multiple files (one signed cookie for many files)

CloudFront Signed URL vs S3 Pre-Signed URL:

Snowball:

Physical data transport solution that helps moving TBs or PBs of data in or out of AWS
Alternative to moving data over the network (and paying network fees)
Secure, tamper resistant, uses KM 256 bit encryption
Tracking using SNS and text message. E-ink shipping label.
Pay per data transfer job
Use cases: large data cloud migrations, DC decommission, disaster recovery.
If it takes more than a week to transfer over the network, use Snowball devices.
Snowball Edge is a 100TB data transfer device.
Snowmobile is an Exabyte scale data transfer:
- Transfer exabytes of data (1EB = 1000 PB = 1M TB)
- Each Snowmobile has 100 PB of capacity (use multiple in parallel)

Better than Snowball if you transfer more than 10 PB.

7. Storage gateway:

service that connects an on-premises software appliance with cloud based storage, it comes as an appliance (hardware) or vm.

3 types of storage gateway:
- File gateway (NFS / SMB) stored on S3
  - Configured S3 buckets are accessible using the NFS and SMB protocol.
  - Supports S3 standard, S3 IA, S3 One Zone IA.
  - Bucket access using IAM roles for each File Gateway
  - Most recently used data is cached in the file gateway
  - Can be mounted on many servers.
- Volume gateway: (iSCSI)
  - Block storage using iSCSI protocol backed by S3
  - Backed by EBS snapshots which can help restore on premise volumes.
  - Cached volumes: low latency access to most recent data
  - Store volumes: entire dataset is on premise, scheduled backups to S3.
- Tape gateway (VTP: Virtual Tape Library)
  - Some companies have backup processes using physical tapes
  - With Tape Gateway, companies use the same processes but in the cloud.
  - Virtual Tape Library (VTL) backed by Amazon S3 and Glacier
  - Backup data using existing tape based processes (and iSCSI interface)
  - Works with leading backup software vendors.

Exam tip: Read the question well, it will hint at which gateway to use:

On premise data to the cloud è Storage gateway
File access / NFS -backed by S3) è File gateway
Volumes / Block storage / iSCSI (backed by S3 with EBS snapshots) è volume gateway
VTL Tape solution / Backup with iSCSI -backed by S3 and Glacier ) è Tape gateway

Athena:

Serverless service to perform analytics directly against S3 files
Use SQL language to query the files
Has a JDBC / ODBC driver
Charged per query and amount of data scanned
Supports CSV, JSON, ORC, Avro,..
Use cases: BI, analytics, reporting, analyze & query, VPC Flow Logs, ELB Logs, CloudTrail trails,..
Exam tip: Analyze data directly on S3 è use Athena

Athena vs Macie:

Athena: interactive query service which enables you to analyse and query data located in S3 using std SQL, it also can query log files, generate business reports, …
Macie: security service which uses ML (machine learning) and NLP (natural language processing) to discover and protect sensitive data stored in S3: PPI (Personally Identifiable Information), it’s great for PCI-DSS, preventing ID theft..

8. Advanced Amazon S3:

S3 MFA-Delete:

To use MFA Delete, enable versioning on the S3 bucket
You’ll need MFA to permanently delete an object version, suspend versioning on the bucket.
Has to be done on CLI not on console (only the root can enable it or disable it).

S3 Default Encryption vs Bucket Policies:

To encrypt a file in S3 bucket, just enable default encryption.
Bucket policies are evaluated before “default encyption”

S3 Access Logs:

For audit purpose
Any request made to S3, from any account, authorized or denied will be logged into another S3 bucket
That data ca be analyzed using data analysis tools like Athena.
S3 pre-signed URLs:

Can generate pre-signed URLs using SDK or CLI
- For download (easy, can use the CLI)
- For uploads (harder, must use the SDK)
Valid for a default of 3600s, can change timeout with –expires in {TIME_BY_SECONDS] argument
Users given a pre-signed URL inherit the permissions of the person who generated the URL for GET / PUT.

Examples:

Allow only logged in users to download a premium video on your S3 bucket
Allow an ever changing list of users to download files by generating URLs dynamically
Allow temporarily a user to upload a file to precise location in our bucket

S3 Performance:

Baseline Performance:
- S3 scale automatically to high request rates, latency 100-200ms
- Your app ca achieve at least 3500 PUT/COPY/POST/DELETE and 5500 GET/HEAD requests per second per prefix in a bucket.
KMS Limitation:
- If you use SSE-KMS, you may be imapcted by the KMS limits
- When you upload, it call the GenerateDataKey KMS API
- When you download, it calls the Decrypt KMS API
- Count towards the KMS quota per second (5500, 10000, 3000 req/s based on region)
- You cant request a quota increase for KMS
Multi Part upload:
- Recommended for files > 100MB, must use for files > 5GB
- Can help parallelize uploads (divied in parts and speed up transfers)
S3 Transfer Acceleration (upload only)
- Increase transfer speed by transferring file to an AWS edge location which will forward the data to the S3 bucket in the target region using AWS backbone.
- Compatible with multipart upload
S3 Byte range Fetches
- Parallelize GETs by requesting specific byte ranges
- Better resilience in case of failures
- Can be used to speed up downloads
- Can be used to retrieve only partial data (for example the head of a file)

S3 Select & Glacier Select:

Retreive less data using SQL by performing server side filtering
Can filter by rows & columns (simple SQL statements, server side filtering)
Less network transfer, less CPU cost client side.

S3 Object & Glacier Vault Lock:

Read the S3 FAQ before taking the exam, it will really help you, https://aws.amazon.com/fr/s3/faqs/

9. EC2:

Pricing models:

On Demand: pay per hour or second with no commitment. Low cost, flexibility, short term, for dev/testing, predictible price.
Reserved: significant discount (1y-3y) app for steady state, requires capacity, …:

– Convertible reserved instances: long workloads with flexible instances. – Scheduled reserved instances: example: every Thu between 3 and 6 pm.

Spot: bid whatever price you want for instance, apps that have flexible start & end times…
Dedicated instances: no other customers will share your hardware, billing is based on instance..
Dedicated hosts: book an entire physical server, for regulatory (no multi-tenant support), great for licensing,

EC2 instance Types – main ones:

R: apps that need a lot of RAM -in memory caches
C: apps that need good CPU – compute / databases
M: apps that are balanced (think medium) – general / web app
I: apps that need good local I/O (instance storage) – databases.
G: apps that need a GPU – video rendering / machine learning.
T2/T3: burstable instances (up to a capacity)
T2/T3: unlimited burst

Termination protection is turned off by default, you must turn it on.
On an EBS-backed instance, the default action for root volume is to be deleted when the instance is terminated.
You can encrypt the root volume.

EBS: 5 different types of EBS storage:

General purpose SSD
Provisionned iops SSD
Throughput optimized HDD
Cold HDD (lowest )
EBS magnetic

AMI Types (EBS vs instance store):

For EBS volumes: the root device for an instance launched from the AMI is an EBS volume created from an EBS snapshot.
For Instance store volumes: the root device for instance launched from the AMI is an instance store volume created from a template stored in S3.
Instance store volume is an Ephemeral storage: if stopped, you will lose all your data.
Instance store is physically attached to the machine (EBS is a network drive).
Instance Store Pros:
- Better I/O perofrmance
- Good for buffer / cache / scratch data / temporary content.
Instance Store Cons:
- On stop or termination, the instance store is lost.
- You cant resize the instance store
- Backups must be operated by the user.
EBS RAID Options (do it on OS not AWS):
- RAID0: increase performance (lots of iops), but more risk (if 1 disk fail, data lost).
- RAID1: increase fault tolerance (mirroring).
- RAID5 (not recommanded for EBS)
- RAID6 (not recommanded for EBS)

Cross Account AMI Copy:

You can share an AMI with another AWS account
Sharing an AMI does not affect the ownership of the AMI
If you copy an AMI that has been shared with your account, you are the owner of the target AMI in your account
To copy an AMI that was shared with you from another account, the owner of the source AMI must grant you read permissions for the sotrage that backed the AMI, either the associated EBS snapshot (for an Amazon EBS-backed AMI) or an associated S3 bucket (for an instance store backed AMI)
Limits:
- You cant copy an encrypted AMI that was shared with you from another account. Instead, if the underlying snapshot and encryption key were shared with you, you can copy the snapshot while re-encrypting it with a key of your own. you own the copied snapshot, and can register it as a new AMI.
- You cant copy an AMI with an associated billingProduct code that was shared with you from another account. this includes windwos AMIs and AMIs from the AWS Marketplace. To copy a shared AMI with a billing Product code, launch an EC2 instance in your account using the shared AMI and then create an AMI from the instance.

ENI vs ENA vs EFA:

ENI: elastic network interface, virtual network card
- For basic networking, create a management network, use a network appliance in your VPC..
- Low cost
EN: Enhanced networking: uses single root I/O virtualization, there is 2 types:
- ENA elastic network adapter: 100Gbps
- VF virtual function: 10Gbps
Elastic Fabric Adapter: for machine learning or high performing computing (HPC) or OS bypass.

EBS encryption:

Data in transit between an instance and an encrypted volume is also encrypted
Encryption is supported on all Amazon EBS volume types

CloudWatch:

Monitor performances:
- Compute:
  - EC2 instances.
  - Autoscaling groups
  - Elastic Load Balancers
  - Route53 Health checks
- Storage & content delivery:
  - EBS volumes
  - Storage gateways
  - CloudFront
- Dashboard are global, it can contains graphs from different region. Exam question
- HowTo do it: Add a widget, add a line, configure it (from the region where your instance is running), look for cpu usage, rename it to show the zone name, save dashboard
- Pricing: 3 dashboards (up to 50 metrics) for free
- 3$/dashboard/month afterwards
- CloudWatch Logs can store logs of EC2… but you have to configure a retention, else you’ll pay.
Host level metrics consist of: cpu, network, disk, status check.
Default 5 min
Detailed monitoring 1min
CloudWatch is all about performance & CloudTrail is all about auditing.
AWS config: record/track config changes and AWS resrouce compliance
Free for Basic Monitoring Metrics, 10 Detailed Monitoring Metrics and 3 dashboards.

AWS Config:

Helps with auditing and recording compliance of your AWS resources
Helps record configurations and changes over time
Possibility of storing the configurtion data into S3 (analyzed by Athena)
Questions that be solved by AWS Config:
- Is there unrestricted SSH access to my security groups?
- Do my buckets have any public access?
- How has my ALB configuration changed over time?
You can receive alerts (SNS notifications) for any changes.
AWS Config is per region service
Can be aggregated across regions and accounts
Pricing: no free tier, 2$ per active rule per region per month.

CloudWatch vs CloudTrail vs Config Example for an Elastic LB:

CloudWatch:
- Monitoring incoming connections metric
- Visualize error codes as a % overt ime
- Make a dashboard to get an idea of your LB performance ..
Config:
- Track security group rules for the LB
- Track configuration changes for the LB
- Ensure an SSL certificate is always assigned to the LB -compliance)
CloudTrail:
- Track who made any changes to the LB with API Calls.

EFS: Elastic File System

it grows automatically, great for file server or sharing data between EC2 instances
You pay for what you use

EFS support NFSv4 protocol
You only pay for the storage you use
Can scale up to the petabytes
Can support 1000s of concurrent NFS connections
Data is stored across multiple AZ”s within a region
Read After write consistency

Amazon FSx for Windows and for Lustre:

FSx for windows: built on windows server, use SMB and supports AD users, DFS (Distributed File System).. . Centralized storage for windows, Sharepoint, IIS or other native app for microsoft:
- Can be accessed from your on-premise infrastructure
- Can be configured to be MultiAZ (HA)
- Data is backed up daily to S3
EFS: is linux only, use it if you need distribution..
FSx for Lustre: for linux, if you process large data sets, millions of IOPS, machine learning, High Performance Computing (HPC), video processing, electronic design automation, financial modeling.

Use cases: S3 vs EBS vs EFS

S3: web serving and content management, media and entertainment, backups; big data analytics, data lake.
EBS: boot volumes, tansactional and NoSQL databases, data warehousing & ETL.
EFS: web serving and content management, entreprise applications, media and entertainment, home directories, database backups, developer tools, container storage, big data analytics.

Storage Comparision:

S3: Object Storage
Glacier: Object Archival
EFS: Network File System for linux instances, shared across AZ.
FSx for windows: NFS for Windows servers
FSx for Lustre: HPC linux file system
EBS volumes: Network storage for one EC2 instance at a time.
Instance Storage: Physical storage for your EC2 instance (high IOPS)
Storage Gateway: file gateway, volume gateway (cache & stored), Tape Gateway
Snowball/Snwomobile: to move large amount of data to the cloud, physically.
Database: for specific workloads, usually with indexing and querying.

EC2 Placement Groups:

Clustered placement group: within a single Availability Zone
- Pros: Low Network Latency / High network throughput
- Cons: if the rack fails, all instances fails at the same time.
- Use case: Big data job that needs to complete fast.
Spread placement group: individual instances placed in different hardware (rack..), for single instance
- Pros:
  - Can span across AZs
  - Reduce risk of simultaneous failure
  - EC2 instances are on different physical hardware
- Cons:
  - Limited to 7 instances per AZ per placement group
- Use case:
  - App that needs to maximize high availability.
  - Critical apps where each instance must be isolated from failure from each other
- Partitioned Placement Group: think multiple instances in partition group.
  - Up to 7 partitions per AZ
  - Up to 100s of EC2 instances
  - The instances in a partition do not share racks with the instances in the other partitions.
  - A partition failure can affect many EC2 but won’t affect other partitions.
  - EC2 instances get access to the partition: information as metadata
  - Use case: HDFS, HBase, Cassandra, Kafka

EC2 Hibernate.

The in-memory (RAM) state preserved.
The instance boot is much faster (os is not stopped/restarted)
Under the hood:the RAM state is written to a file in the root EBS volume
The root EBS volume must be encrypted
Supported instance families: C, M and R.
Instance RAM size must be less than 150GB
Available for OnDemand and Reserved Instances.
An instance cannot be hibernated more than 60days
Use cases:
- long running processing
- saving the RAM state
- services that take time to initialize.

This article covers these aws services: RDS, Aurora, DynamoDB, ElastiCache, Route53.

RDS:

Relational databases on AWS: (OLTP: OnLine Transaction Processing)

2 features:
- Multi-AZ for disaster recovery ⇒ failover is automatic.
- Read replicas ⇒ failover is manual
Up to 5 Read Replicas
Within AZ, Cross AZ or Cross Region.
Replication is ASYNC, so reads are eventually consistent.
Replicas can be promoted to their own DB.
Applications must update the connection string to leverage read replicas.

RDS Read Replicas – use cases:

You have a production database that is taking on normal load.
You want to run a reporting app to run some analytics
You create a Read Replica to run the new workload there.
The production app is unaffected
Read replicas are used for SELECT (read) only kind of statements (not INSERT, UPDATE, DELETE)

RDS Read Replicas – Network Cost:

In AWS there is a network cost when data goes from one AZ to another.
To reduce the cost, you can have your Read Replicas in the same AZ.

RDS Multi AZ (Disaster Recovery)

SYNC replication
One DNS name – automatic app failover to standby
Increase availability
Failover in case of loss of AZ, loss of network, instance or storage failover.
No manual intervention in apps
Not used for scaling
Note: the Read Replicas can be setup as Multi AZ for Disaster Recovery (DR)
DynamoDB: Amazon noSQL.
Redshift: amazon data warehouse. (OLAP OnLine Analytical Processing)
ElastiCache: webservice that makes it easy to deploy, operate, and scale an in memory cache in the cloud., it speeds up performance of existing db. (frequent identical queries)

RDS runs on virtual machines.
You can’t login to these operating systems however
patching of RDS os and db is amazon responsibility.
RDS is not serveless (except AURORA)

Backup with RDS:

Automated backup (data & logs) by default and stored on S3.
DB Snapshot (manually)
Aurora: compatible with Mysql and PostgreSQL
- start with 10GB, scales in 10GB increments to 64TB (storage autoscaling)
- 2 copies of your data are contained in each AZ, with minimum of 3 AZ, 6 copies of your data.
Three types:
- Aurora Replicas (currently 15)
- Mysql replicas 5
- PostgreSQL replicas 1

Point in time database recovery:
- The default DB security group is applied to the new DB instance
You can restore up to the last 5 minutes

Read Replicas:

Can be multi-AZ
Used to increase performance
Must have backups turned on
Can be in different regions
Can be Mysql, PostgreSQL, mariaDB, oracle, Aurora
Can be promoted to master, this will break the Read Replica

Mult-iAZ:

used for DR
you can force failover from one AZ to another by rebooting an RDS instance.

RDS Security – Encryption

At rest encryption:
- Possibility to encrypt the master & read replicas with AWS KMS – AES-256 encryption
- Encrption has to be defined at launch time
- If the master is not encrypted, the read replicas cannot be encrypted.
- Transparent Data Encryption (TDE) available for Oracle and SQL Server.
In flight encryption:
- SSL certificates to encrypt data to RDS in flight
- Provide SSL options with trust certificate when connecting to database.
- To enforce SSL:

PostgreSQL: rds.force_ssl=1 in the AWS RDS Console (Parameter Groups) MySQL: with the DB: grant usage on *.* to ‘mysqluser’@’%’ require ssl; RDS Security – Network & IAM

Network Security
- RDS databases are usually deployed with a private subnet, not in a public one.
- RDS security works by leveraging security groups (the same concept as for EC2 instances) – it controls wich IP / security group can communicate with RDS
Access Management
- IAM policies help control who can manage AWS RDS (through the RDS API)
- Traditional Username and Password can be used to login into the database.
- IAM based authentication can be used to login into RDS MySQL & PostreSQL.
Enable secure mysql user access with short-lived credentials è Create the MySQL user accounts to use the AWSAuthenticationPlugin with IAM

DynamoDB: (Serverless) The basics of DynamoDB: Amazon NoSQL solution.

stored on SSD storage
Spread across 3 geographically distinct data centres.
Eventual Consistent Reads (Default)
Best read performance.
Strongly Consistent Reads:
Read updated data. (less than 1s).
Multi-master database
DynamoDB global tables replicate your data automatically across 4 AZ of your choice of AWS Regions and automatically scale capacity to accommodate your workloads.
DynamoDB is serverless, it doesn’t requeire any maintenance.

Provisioned Throughput

Table must have provisioned read and write capacity units
Read Capacity Units (RCU): throughput for reads (0.00013 $ per RCU)
- 1 RCU: 1 strongly consistent read of 4KB per second
- 1 RCU: 2 eventually consistent read of 4KB per second
Write Capacity units (WCU): throughput for write (0.0005 $ per WCU)
- 1 WCU: 1 write of 1KB per second
Option to setup auto-scaling of throughput to meet demand
Throughput can be exceeded temporarily using “burst credit”
If burst credit are empty, you’ll get a “ProvisionedThroughputException”
It’s then advised to do an exponetial backoff retry

DynamoDB Advanced features:

DAX: DynamoDB Accelerator: Seamless cache for DynamoDB, no app re-write
DynamoDB Streams: changes in DynamoDB (create,update..) end up in a DynamoDB Stream
Transactions: All or nothing type of operations
On Demand: No capacity planning needed (WCU / RCU)

RedShift (OLAP):

Fast, fully managed data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and existing BI tools.
Petabyte-scale data
Runs complex analytic queries against petabytes of structured data.
Load data from Amazon S3 and perform analytics queries.
RedShift Spectrum can analyze data directly in Amazon S3
S3 Select can also be used to fast analysis of data stored in S3.

Config:

Single node 160Gb
Multi Node: leader node (manage client connection and receive queries and compute node that store and perform computation)

Backups:

enabled by default with 1 day retention period
Max retention period is 35d
Redshift always attempts to maintain at least three copies of your data (original, replica on compute node and backup on S3)
can also asynchronously replicate your snapshot to S3 to another region for DR.

Security:

Encrypted (SSL / AES256 / KMS)

Aurora:

Aurora storage automatically grows in increments of 10GB up to 64TB.
Failover in Aurora is instantaneous It’s HA native.
Aurora costs more thant RDS (20% more), but is more efficient. >>> check this info

Aurora HA and Read scaling:

6 copies of your data across 3 AZ:
- 4 copies out of 6 needed for writes.
- 3 copies out of 6 needed for reads
- Self healing with peer to peer replication
- Storage is striped across 100s of volumes.
One Aurora instance takes writes (master)
Automated failover for master in less than 30s
Master +up to 15 Aurora Read Replicas serve reads
Support for Cross Region Replication

Aurora DB Cluster:

Use Autoscaling and shared volume.
Writer Endpint: pointing to the master.
Reader Endpoint: helps connection Load Balancing

Features of Aurora:

Automatic failover, Backup and Recovery
Isolation and security, Industry compliance
Push button scaling, Automated patching with Zero Downtime
Advanced Monitoring, Routine Maintenance
Backtrack: restore data at any point of time without using backups.

Aurora Security:

Similar to RDS because uses the same engines.
Encryption at rest using KMS
Automated backups, snapshots and replicas are also encrypted.
Encryption in flight using SSL (same process as MySQL or Postgres)
Possibility to authenticate using IAM token (same method as RDS)
You are responsible for protecting the instance with security groups.
You can’t SSH (same as RDS)

Aurora Serverless:

Automated database instantiation and autoscaling based on actual usage.
Good for infrequent, intermittent or unpredictable workloads.
No capacity planning needed
Pay per second, can be more cost-effective.

Global Aurora:

Aurora Cross Region Read Replicas:
- Useful for DR
- Simple to put in place
Aurora Global DB (recommended)
- 1 Primary region (read / write)
- Up to 5 secondary (read only regions, replication lag is less than 1 second.
- Up to 16 Read Replicas per secondary region.
- Helps for decreasing latency
- Promoting another region (for DR) has an RTO of < 1 minute.

Elasticache:

Use it to increase db and web app performance
Redis is Multi-AZ
you can do backups and restores of Redis
if you need to scale horizontally, use Memcached
Application queries ElastiCache, if not available, get from RDS and store in ElastiCache.
Helps relieve load in RDS
Cache must have an invalidation strategy to make sure only the most current data is used in there.
User session store:
- User logs into any of the application
- The application writes the session data into ElastiCache
- The user logs to another instance of your application
- The instance retrieves the data and the user is already logged in

REDIS VS MEMCACHED: ElastiCache – Cache Security:

All caches in ElastiCache
- Support SSL in flight encryption
- Do not support IAM authentication
- IAM policies on ElasiCache are only used for AWS API-level security
Redis AUTH
- You can set a “password/token” when you create a Redis cluster
- This is an extra level of security for your cache (on top of security groups)

Memcached:

Supports SASL based authentication (advanced)

Patterns for ElastiCache:

Lazy Loading: all the read data is cached, data can become stale in cache
Write Through: Add or update data in the cache when written to a DB (no stale data)
Session Store: store temporary session data in a cache (using TTL features)

Database Types:

RDBMS (SQL/OLTP): RDS, Aurora, great for joins
NoSQL database: DynamoDB (~JSON), ElastiCache (key / value pairs), Neptune (graphs) – no joins, no SQL
Object Store: S3 (for big objects) / Glacier (for backups / archives)
DataWarehouse (SQL analytics / BI): Redshift (OLAP), Athena
Search: ElasticSearch (JSON) – free text, unstructured searches
Graphs: Neptune – displays relationships between data.

Neptune:

Fully managed graph database
When do we use Graphs?
- In relationship data
- Social Networking: users, friends with users, replied to comment on post of user and likes other comments.
- Knowledge graphs (wikipedia)
High available across 3 AZ, with up to 15 read replicas
Point in time recovery, continuous backup to S3
Support for KMS encryption at rest + HTTPS

DB Types based on the five pillars: (for Solutions Architect) RDS:

Operations: small downtime when failover happens, when maintenance happens, scaling in read replicas / ec2 instance / restore EBS implies manual intervention, application changes.
Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, authorizing users in DB using SSL.
Relaibility: Multi AZ feature, failover in case of failures.
Performance: depends on EC2 instance type, EBS volume type, ability to add Read Replicas, Doesn’t auto-scale.
Cost: Pay per hour and on provisione EC2 and EBS.

Aurora:

Operations: less operations, auto scaling storage
Security: AWS responsible for OS security, we are responsible for setting up KMS, security groups, IAM policies, authorizing users in DB using SSL.
Realiability: Multi AZ, highly available, possibly more than RDS, Aurora, Serverless option
Performance: 5x performance due to architectural optimizations. Up to 15 Read Replicas (only 5 for RDS)
Cost: Pay per hour based on EC2 and storage usage. Possibly lower costs compared to Entreprise grade databases such as Oracle.

ElastiCache:

Operations same as RDS
Security: AWS responsible for OS secyrity, we are respsonsible for setting up KMS, security groups, IAM policies, users (Redis Auth), using SSL.
Reliability: Clustering, Multi AZ
Performance: Sub-millisecond performance, in memory, read replicas for sharding, very popular cache option
Cost: Pay per hour based on EC2 and storage usage.

DynamoDB:

Operations: no operations needed, auto scaling capability, serverless.
Security: full security through IAM policies, KMS encryption, SSL in flight.
Reliability; Multi AZ, backups
Performance: single digit milisecond performance, DAX for caching reads, performance doesn’t degrade if your application scales.
Cost: Pay per provisionned capacity and storage usage (no need to guess in advance any capacity – can use auto scaling)

S3:

Operations: no operations needed.
Security: IAM, Bucket policieis, ACL, encryption (server/client), SSL.
Reliability; 99,99999999999% durability, 99,99% availability, Multi AZ, CRR
Performance: scales to 1000s of read / writes per second, transfer acceleration / multipart for big files.
Cost: Pay per storage usage, network cost, requests number.

Athena:

Operations: no operations needed, serverless.
Security: IAM policies, S3 security.
Reliability; Managed services, uses Presto engine, highly available.
Performance: queries scale based on data size.
Cost: Pay per query / per TB of data scanned, serverless

Redshift:

Operations: similar to RDS
Security: IAM, VPC, KMS, SSL (similar to RDS)
Reliability: highly available, auto healing features.
Perfomance: 10x performance vs other data warehousing, compression
Cost: pay per node proviosned, 1/10^th of the cost vs other warehouses.

Neptune:

Operations: similar to RDS
Security: IAM, VPC, KMS, SSL (similar to RDS) + IAM Authenticiation
Reliability: MultiAZ, clustering
Perfomance: best suited for graphes, clustering to improve performance.
Cost: pay per node provisioned (similar to RDS)

ElasticSearch:

Operations: Similar to RDS
Security: Cognito, IAM, VPC, KMS, SSL
Reliability; Multi AZ, clustering
Performance: based on ElasticSearch project (opensource), PetaByte scale
Cost: pay per node provisioned (similar to RDS)

10. Route53:

ELB do not have predefined IPv4 addresses you resolve to them using a DNS name
Given the choice, always choose an Alias record instead of CName.
Common DNS types/records: SOA, NS, A, CName, MX, PTR.
Traffic flow: routes end users to the endpoint that should provide the best user experience

CNAME vs Alias: Can use Alias records for:

API Gateway custom regional API or edge-optimized API
VPC interface endpoint
CloudFront distribution
Elastic Beanstalk environment
ELB load balancer
Global Accelerator
S3 bucket that is configured as a static website

Route53 Routing policies available on AWS:

Simple routing: Use for a single resource that performs a given function for your domain, for example, a web server that serves content for the example.com website.
weighted routing: spreads traffic between several services via a round-robin algorithm, Helpful to test 1% of traffic on new app version for example.
latency based routing: routes users to the AWS region that provides the lowest latency
failover routing: redirects users to an alternative service in case of outage
geolocalisation routing: Use when you want to route traffic based on the location of your users.
geo proximity routing (traffic flow only): Use when you want to route traffic based on the location of your resources and, optionally, shift traffic from resources in one location to resources in another.
multivalue answer routing: Use when routing traffic to multiple resources, up to eight healthy records for each Multi Value query.

Health Checks

Have X heatlh checks failed ==> unhealthy (default 3)
After X heatlh checks passed ==> health (default 3)
Default health Check Interval: 30s (can set to 10s – higher cost)
About 15 health checkers will check the endpoint health.
one request every 2s on average.
Can have HTTP, TCP and HTTPS health checks (no SSL veritifcation)
Possibility of integrating the health check with CloudWatch
Health checks can be linked to Route53 DNS queries !

11. Security:

KMS: Key Management Service.

Fully integrated with IAM authorization, EBS, S3, Redshift, RDS, SSM..
KMS can only help in encrypting up to 4KB of data per call If data > 4 KB, use envelope encryption
Encrypt sensitive like: database passwords, private key of ssl certificates, credentials to external service.
Never ever store your secrets in plaintext, espacially in your code!

SSM Parameter Store.

Secure Storage for configuration and secrets
Optional seamless encryption using KMS
Serverless, scalable, durable, easy SDK, free.
Configuration management using patch & IAM
Notification with CloudWatch events

AWS Secrets Manager:

Newer service, meant for storing secrets
Capability to force rotation of secrets every X days
Automate generation of secrets on rotation (uses Lambda)
Secret storing, RDS integration

CloudHSM:

KMS: AWS manage the software for encryption
CloudHSM: AWS provisions ecncryption hardware
Dedicated hardware.

AWS Shield

AWS Shield Standard: free service for attacks protection
AWS Shield Advanced:
- DDoS mitigation (3.000usd),
- Protect against more sophisticated attack on EC2, ELB, CloudFront, Global Acceleraor and Route53
- 24/7 access to AWS DDoS response Team DRP.

AWS WAF:

Protect web app from web exploits L7, not DDoS protection.
Deploy on Application LB, API Gateway, CloudFront
Allows 3 different behaviours:
- Allow all requests except the ones you specify
- Bloc all request except the ones you specify
- Count the requests that match the properties you specify

12. VPC: Virtual Private cloud.

VPC Architecture:

A VPC is a logical datacenter in AWS.
Consists of Internet Gateway, route tables, NACL, subnet and security groups.
VPC Peering: connect one vpc with another.
No transitive peering: means each vpc must be peered with another, you can not route traffic to vpc2 (from vpc1) through vpc3.
CIDR Calulator: http://cidr.xyz

Exam Tip:

If you need 29 IP addresses for EC2 isntances, you cant choose a Subnet of size /27 (32 IP)
You need at least 64 IP, subnet size /26 (64-5=59 >29, but 32-5=27<29)

NAT instance: single EC2 instance.
NAT gateway: highly available gateway for your private instances to communicate to interne

NAT gateways:

Redundant inside the AZ
Preferred by the enterprise
Starts at 5Gbps and scales currently to 450Gbps
No need to patch
No associated with sec groups
Automatically assigned a public ip address
Remember to update your route tables
No need to disable src/dest checks.
Create a NAT gateway per AZ.

Security Groups VS Network ACLs: DNS Resolution in VPC

In a default VPC instances will be assigned a public and private DNS hostname
In a non-default VPC instances will be assigned a private but not a public DNS hostname

Not all IP Traffic is monitored:

Traffic generated by instances when they contact Amazon DNS server. if you use your

DNS server, then all traffic to that DNS server is logged

Traffic generated by a Windows instance for Amazon Windows license activation.
Traffic to/from 169.254.169.254 for instance metadata
DHCP traffic
Traffic to the reserved IP address for the default VPC router.

Bastion Hosts:

Computer that is connected to an untrusted network and private network.

You have to Harden (secure) this computer.

A NAT Gateway or NAT instance is used to provide internet traffic to EC2 instances in a private subnet.
A Bastion is used to securely administer instances (using ssh, rdp).
You cannot use a NAT gateway as a Bastion host.

Exam Tip: Make sure the bastion host only has port 22 traffic from the IP you need, not from the security groups of your other instances. Site to Site VPN:

Virtual Private Gateway:
- VPN Concentrator on the AWS side of the VPN connection
- VGW is created and attached to the VPC from which you want to create the Site to Site VPN Connection
- Possibility to customize the ASN
Customer Gateway:
- Software application or physical device on customer side of the VPN connection
- IP Address:
  - Use static, internet routable IP address for your customer gateway device
  - If CGW behind NAT (with NAT-T), use the public IP address of the NAT

Direct Connect:

Cloud service solution used to establish a connection between AWS and your DC.
Useful for high throughput worloads (ie iots of network traffic)
Or if you need a stable and reliable secure connection.

Setting up Direct Connect steps.

Create virtual interfaces in the Direct Connect console. This is a PUBLIC virtual interface.
VPC console / VPN connections, Create a customer gateway.
Create a virtual private gateway.
Attach the virtual private gateway to the desired VPC.
Select VPN connections and create new VPN connections.
Select the virtual private gateway and the customer gateway.
Once the vpn is available, set up the vpn on the customer gateway or firewall.

!! learn the steps, it might be on the exam, Egress Only Internet Gateway:

Egress means outgoing, it’s like NAT but only for IPv6.
IPv6 are all public.
Therefore all out instances with IPv6 are publicly accessibly
Egress Only Internet Gateway gives our IPv6 instances access to the internet, but they wont be directory reachable by the internet
After creating an Egress Only Internet Gateway, edit the route tables.

– Global Accelerator.

A service in which you create an accelerator to improve availability and performance of your app for your local and global users.
Works with Elastic IP, EC2 instances, ALB, NLB, public or private
Consistent Perofmrance:
- Intelligent routing to lowest latency and fast regional failover
- No issue with client cache (because the IP doesn’t change)
- Internal AWS network
Health check:
- Global Accelerator perofms a health checks for your applications
- Helps make your application global (failover less than 1 minute for unhealthy)
- Great for DR (thanks to the health checks)
Security:
- Only 2 external IP need to be whitelisted
- DDos protection thanks to AWS Shield

It includes the following components:

Static ip addresses: provides you two static ip @ (it uses Anycast IP)
Accelerator: it directs traffic to optimole endpoint.
DNS name: it assigns you a default dns name.
Network zone: like Availability Zone.
Listener: processes inbound connections from clients to global accelerator.
Endpoint group: associated with a specific AWS region.
Endpoint: can be network LB, app LB, EC2 instances or Elastic IP addresses.

Global Accelerator vs CloudFront

They both use the AWS global netowrk and its edge locations around the world
Both services integrate with AWS shield for DDoS protection

CloudFront:

Improves perofrmance for both cacheable content (such as images and videos)
Dynamic content (such as API acceleration and dynamic site delivery)

Global Accelerator:

Improves performance for a wide range of apps over TCP or UDP
Proxing packets at the edge to apps running in one or more AWS Regions
Good fit for notHTTP use cases, such as gaming (UDP), IoT (MQTT) or VoIP
Good for HTTP use cases that require static IP address
Good for HTTP use cases that requere deterministic, fast regional failover.

VPC Endpoints:

Enables to privately connect your VPC to supported AWS services and VPC endpoint services powered by PrivateLink without requiring an internet gateway, NAT device, VPN connection or AWS direct connect. (staying at aws network without leaving it to internet)
Two types:

– Interface endpoint – Gateway endpoint (Amazon S3 and DynamoDB) AWS PrivateLink (VPC Endpoint Services)

More secure & scalable way to expose a service to 1000s of VPC (own or other accounts)
Does’nt require VPC peering, internet gateway, NAT, route tables..
Requires a network load balancer (Service VPC) and ENI (Customer VPC)
If the NLB is in multiple AZ, and the ENI in multiple AZ, the solution is fault tolerant.

EC2 Classic & AWS ClassicLink (deprecated)

EC2-Classic: instances run on a single network shared with other customers
Amazon VPC: your instances run logically isolated to your AWS account

ClassicLink allows you to link EC2-Classic instances to a VPC in your account

Must associate a security group
Enables communication using private IPv4 addresses
Removes the need to make use of pubic IPv4 addresses or Elastic IP addresses

AWS VPN CloudHub:

Provide secure communication between sites, if you have multiple VPC connections
Low cost hub and spoke mode for primary or seconday netowrk connectivity between locations
It’s a VPN connection so it goes over the public internet.

VPC Summary:

CIDR: IP range
VPC: Virtual Private Cloud è we define a list of IPv4 & IPv6 CIDR
Subnet: tied to an AZ, we define a CIDR
Internet Gateway: at the VPC level, provide IPv4 & IPv6 Internet Access
Route Tables: must be edited to add routes from subnets to the IGW, VPC Peering Connections, VPC Endpoints,..
NAT instance: gives internet access to instances in private subnets. Old, must be setup in a public subnet, disable Source/destination check flag.
NAT Gateway: managed by AWS, provides scalable internet access to private instances, IPv4 only
Private DNS + Route53: enable DNS Resoluton + DNS hostnames (VPC)
NACL: Stateless, subnet rules for inboud and outbound, don’t forget ephermeral ports
Secrity Groups: Stateful, operate at the EC2 instance level.
VPC Peering: Connect 2 VPC with non overlapping CIDR, non transitive
VPC Endpoints: Provide access to AWS Services (S3, DynamoDB, CloudFormation, SSM) within VPN
VPC Flow Logs: Can be setup at the VPC / Subnet / ENI Level for ACCEPT and REJECT traffic, helps identifiying attacks, analyze using Athena or CloudWatch Log Insights
Bastion Host: Public instance to SSH into, that has SSH connectivity to private instances
Site to site VPN: setup a Customer Gateway on DC, a Virtual Private Gateway on VPC and site to site VPN over public internet
Direct connect: setup a VPGateway on VPC, and establish a direct private connection to an AWS Direct Connect Location
Direct connect Gateway: setup a Direct Connect to many VPC in different regions
Internet Gateway Egress: like a NAT gateway but for IPv6
Private Link/VPC Endpoint Services:
- Connect serivces privately from your service VPC to customers VPC
- Doesn’t need VPC peering, public internet, NAT gateway, route tables.
- Must be used wih Network Load Balancer & ENI
ClassicLink: connect EC2 Classic instances privately to your VPC
VPN CloudHub: hub and spoke VPN modem to connect your sites.

Continue your learning: Notes Part2/2
The 2nd part contains these aws services: High Availability HA, Elastic Beanstalk, Applications: SQS, SNS, API Gateway, Kinesis, Amazon MQ, Lambda, Lambda@Edge, AWS SAM (Serverless Application Model), Advanced IAM (Identity and Access Management), AWS Cognito, AWS Directory Service, AWS SSO, Disaster Recovery & Mitigation services, CI-CD: Continuous Integration / Continuous Delivery, AWS ECS,

Notes Certified Solutions Architect – Associate – Part 1/2