AWS Cloud Services Quick Reference
Data engineers, is it different to choose between the ample number of services AWS offers?
🤟🏽 Here's a breakdown of the AWS cloud services:🔹 𝐃𝐚𝐭𝐚 𝐈𝐧𝐠𝐞𝐬𝐭𝐢𝐨𝐧:
• Amazon Kinesis: Real-time streaming data collection & processing
• AWS IoT Core: Ingests data securely from IoT devices
• AWS Lambda: Event-driven data ingestion & lightweight processing
• AWS Data Migration Service (DMS): Migrates databases to AWS with minimal downtime
• AWS Glue Crawlers: Automatically discovers & catalogs data sources
• AWS AppFlow: Seamlessly transfers data between AWS & SaaS applications
• AWS DataSync: Accelerates data transfer from on-premises or other clouds to AWS
• AWS Snowball: Physical device for large-scale data transfer into AWS
🔹 𝐃𝐚𝐭𝐚 𝐒𝐭𝐨𝐫𝐚𝐠𝐞:
• Amazon S3 Glacier: Low-cost, long-term archival storage
• Amazon EFS: Managed file storage for scalable access
• Amazon EBS: Block storage for EC2 instances & high-performance workloads
• Amazon DynamoDB: Fully managed NoSQL database for fast and flexible storage
• AWS Storage Gateway: Hybrid cloud storage integration for on-premises environments
🔹 𝐏𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 & 𝐂𝐨𝐦𝐩𝐮𝐭𝐚𝐭𝐢𝐨𝐧:
• AWS Glue: Serverless ETL (Extract, Transform, Load) service for data preparation
• Amazon EMR: Managed Hadoop, Spark, & other big data frameworks
• AWS Lambda: Serverless compute for event-driven processing tasks
• Amazon Kinesis Data Analytics: Real-time analytics on streaming data
• AWS Step Functions: Orchestrates complex workflows & data pipelines
• AWS Glue DataBrew: Visual interface for no-code data transformation & cleansing
• Amazon SageMaker: End-to-end machine learning model building, training, & deployment
🔹 𝐃𝐚𝐭𝐚 𝐖𝐚𝐫𝐞𝐡𝐨𝐮𝐬𝐢𝐧𝐠 & 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞:
• Amazon Redshift: Scalable, managed data warehouse for analytics
• AWS Lake Formation: Simplifies setup & governance of secure data lakes
• AWS Glue Data Catalog: Central metadata repository for data assets
• Amazon RDS: Managed relational database (supports MySQL, PostgreSQL, Oracle, SQL Server, Aurora)
• Amazon Aurora: High-performance, MySQL & PostgreSQL compatible relational database
• Amazon DynamoDB: Fast & flexible NoSQL database service
• Amazon OpenSearch Service: Managed search & analytics engine
🔹 𝐕𝐢𝐬𝐮𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 & 𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧:
• Amazon QuickSight: Scalable business intelligence & data visualization tool
• Amazon Athena: Serverless, interactive query service for data in S3 using SQL
• Amazon CloudWatch: Monitoring, logging, & observability for AWS resources and applications
• Amazon Managed Grafana: Advanced dashboards & visualization for operational data
• Amazon OpenSearch Dashboards: Interactive dashboards for search & analytics data
As data engineers, select services for each stage based on data volume, latency, processing complexity, & integration needs.
#AWS, #Cloud, #QuickReferenceSheets,
Posts
0 comments