• Should You Use Kubernetes?

• CQRS. Command Query Responsibility Segregation

• WebSockets.  API Architecture style

• API Design: 51 ✔️DO's and ❌DON'Ts

• Functional Requirements (FR) vs Non-functional (NFR)

1. Should You Use Kubernetes?

Kubernetes (or K8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes is an immensely powerful tool, but whether you should use it depends on your specific needs and constraints. Here are some considerations.

Why You Should Consider Kubernetes:
🚀 Automated Deployment: Enables streamlined and consistent application rollouts.
📈 Scalability: Effortlessly scale applications based on demand, ensuring efficient utilization of resources.
⛑ Self-Healing: Provides automatic recovery from application or infrastructure failures.
🌍 Portability: Ensures consistent deployments across various environments, be it on-premises or different clouds.
🔀 Service Discovery & Load Balancing: Offers efficient traffic management and container discoverability.
🌱 Resource Efficiency: Ensures optimal use of infrastructure resources, potentially saving costs.
💽 Storage Orchestration: If you require seamless integration with various storage solutions.
🔐 Secrets Management: Securely manage sensitive data, separate from application configurations.
🔄 Batch Execution: Efficiently manage batch processing tasks, beneficial for data processing workloads.
🔌 Extensibility: Adapt and customize according to specific needs using its vast ecosystem.

Why You Might Hesitate:
🤔 Complexity: The steep learning curve can be intimidating and may demand significant effort and time.
🐌 Resource Overhead: Might be overkill for small applications due to its resource demands.
🔒 Security Concerns: Requires intricate configurations to ensure security, and mistakes can be costly.
🕸️ Networking Issues: Network configurations, especially in hybrid or on-premises environments, can be challenging.
💾 Storage Challenges: Stateful applications have added complexities, even though Kubernetes has provisions for them.
⚙️ Upgrade Concerns: Upgrades may introduce incompatibilities or require significant planning.
☁️ Vendor Dependencies: There's a potential for cloud-specific lock-in, especially when using provider-specific extensions.
🛠️ Tool Fragmentation: The vast ecosystem can be overwhelming, making it hard to choose the right tools.
📊 Monitoring & Logging: The native capabilities may not be sufficient, necessitating external tool integrations.
🕴️ Operational Costs: It might demand specialized teams for management, especially at a larger scale.

Your decision should weigh the pros and cons based on your organization's specific needs, the scale of operations, and available resources.

❓ Does Kubernetes seem like the right fit for your current and future goals? 🤷‍♂️

2. CQRS. Command Query Responsibility Segregation

2.1. Definition.

CQRS (Command Query Responsibility Segregation) - a software architectural pattern that segregates operations that modify data (commands) from operations that retrieve data (queries).
CQRS is a powerful pattern for certain types of applications but may be an overkill for simpler ones. Its adoption should be weighed against the specific needs of the project, considering factors like system complexity, team expertise, and performance requirements. It's not a one-size-fits-all solution and is best applied when the benefits outweigh the costs and complexity.

2.2 Concepts 💡
- Command: Represents actions or changes, modifies data but does not return data, typically results in a change to the application’s state.
- Query: Represents data retrieval requests, returns data but does not modify it.

2.3. Benefits:
- Scalability: Independent scaling of read and write operations, useful for high load scenarios.
- Flexibility: Different data storage choices for read and write.
- Simplified Code: Segregation allows for a cleaner codebase.
- Enhanced Security: Applying distinct security measures for read and write operations.

2.4. Disadvantages:
- Complexity: Can be overkill for simple CRUD apps.
- Learning Curve: Steep for newcomers.
- Dev Effort: More code due to separation.
- Data Duplication: Risk of double storage.
- Misapplication: Risk without domain knowledge.
- Costs: Increased infrastructure expenses.

2.5. Challenges:
- Data Sync: Balancing data between command and query sides.
- Eventual Consistency: System might not be immediately consistent post-command.
- Testing: Ensuring data integrity during command failures.
- Errors: Complex error handling due to segregation.

2.6. Common Use Cases:
- Highly Scalable Systems. Systems needing separate read and write scaling.
- Complex Business Logic. Distinct domain logic for writing and reading.
- Event-Driven Architectures. Often combined with Event Sourcing.

2.7. Related Patterns & Concepts:
- Event Sourcing: Storing changes as events, great with CQRS for system history.
- Domain-Driven Design (DDD): Focuses on modeling business domains, CQRS can be a key pattern within DDD.

❓ Does CQRS seem like the right fit for your current project? 🤷‍♂️

3. gRPC. API Architecture style

gRPC, an exceptional open-source framework crafted by Google, is revolutionizing the realm of microservices. Let's delve into the reasons behind its rising prominence::

3.1. Message formats:
◉ Protocol Buffers

3.2. 🌟Advantages:
✅ High Performance: Uses Protocol Buffers    
✅ Language Agnostic: Supports multiple languages    
✅ Bi-Directional Streaming: Real-time communication    
✅ Deadlines/Timeouts: Clients set RPC wait times    
✅ Pluggable: Supports authentication, load balancing    
✅ Built-in Authentication: Uses SSL/TLS

3.3. 😕. Disadvantages:
❌ Learning Curve: Needs understanding of gRPC
❌ Browser Support: Limited without gRPC-Web
❌ HTTP/2 Dependent: Not supported everywhere
❌ Complexity: Might be overkill for simple services
❌ Tooling: Less extensive than REST
❌ Interoperability: Limited with non-gRPC systems

3.4. 📋Use Cases:
◉ Microservices: Efficient inter-service communication
◉ Real-time Apps: Supports real-time data exchange
◉ Mobile Clients: Optimized communication for mobile backends
◉ Point-to-Point: Direct system-to-system communication
◉ Polyglot Systems: Supports multiple programming languages
◉ Cloud-Native Apps: Integrates with cloud-native platforms
◉ Service Mesh: Used in architectures like Istio
◉ Browser Clients (gRPC-Web): Web apps communicate with gRPC services
◉ IoT Communication: Efficient server-to-IoT device communication

❓ Guys, have you integrated gRPC in your projects? How did it transform your workflow?

4. API Design: 51 ✔️DO's and ❌DON'Ts.

APIs are the backbone of modern applications, enabling seamless integration and communication between systems.
❓ But how can we ensure our APIs are user-friendly, efficient, and secure?

4.1. Requirement Gathering
✔️DO: Identify all potential users of the API. Example: E-commerce API users could be customers, administrators, and suppliers.
❌DON'T: Assume you know all the requirements without consulting stakeholders

4. 2. Identify the Main Objects/Entities
✔️DO: Break down the application into core entities. Example: Users, Products, Orders, etc
❌DON'T: Overcomplicate with too many entities initially

4.3. Define the Endpoints
DO: Use clear, intuitive endpoint names. Example: GET /products to retrieve products
❌DON'T: Use ambiguous endpoint names. Bad Example: GET /getp

4.4. Resource Names (Nouns)
✔️DO: Use nouns. GET /products @app.route('/products', methods=['GET']) def get_products():
❌DON'T: Use verbs. Bad Example: GET /getProducts @app.route('/getProducts', methods=['GET']) def get_products():

4.5. Plurals
✔️DO: GET /users/123
❌DON'T: Bad Example: GET /user/123

4.6. Idempotency
✔️DO: Ensure methods like PUT and DELETE are idempotent. Multiple identical requests should have the same effect as one request. Multiple PUT /products/123 with the same data shouldn't create multiple products.
❌DON'T: Allow multiple identical requests to produce different outcomes. Bad Practice: Each PUT /products/123 creating a new product

4.7. Versioning
✔️DO: GET /v1/products
❌DON'T: Bad Example: GET /productsNew

4.8. Soft Deletion
✔️DO: DELETE /products/123 // Marks as deleted, doesn't actually remove GET /products?includeDeleted=true
❌DON'T: Immediately and permanently delete data without a backup or recovery option.

4.9. Pagination
✔️DO: For endpoints that can return a large number of results, always implement pagination GET /products?page=2&limit=10
❌DON'T: Return all records without pagination. Bad Practice: GET /users returning thousands of records.

4.10. Sorting
✔️DO: Allow sorting by different fields /products?sort=price_asc
❌DON'T: Lack flexibility in sorting options

4.11. Filtering
✔️DO: Allow filtering on relevant fields. /products?category=electronics
❌DON'T: Return all data without filtering options

For additional 40 DOs and DON'Ts, refer to the slide.

5. Functional Requirements (FR) vs Non-functional (NFR)

5.1.🛠 Functional Requirements:
🔹 Definition: Describe the specific functionalities or tasks a system should perform.
🔹 Nature: Focus on WHAT the system must do.
🔹 Examples:
📝 User registration
🔍 Search functionality
💳 Payment processing
🔹 Focus: Features and capabilities.
🔹 Documentation: Use cases, user stories, functional specifications.
🔹 Validation: Through functional testing like unit testing and integration testing.
🔹 Change Frequency: Can change based on business needs and user feedback.
🔹 Stakeholders: Often of primary interest to users, business analysts, and developers.
🔹 Measurement: Can be validated as either 'met' or 'not met'.

5.2.🌟 Non-Functional Requirements:
🔹 Definition: Define how a system should behave, setting standards for system performance, reliability, etc.
🔹 Nature: Focus on HOW the system does it.
🔹 Examples:
⚡ Response time
📈 Scalability
🔒 Security
👥 Usability
🔹 Focus: Quality attributes or system properties.
🔹 Documentation: Performance benchmarks, SLAs, quality attributes.
🔹 Validation: Through non-functional testing like performance testing and security audits.
🔹 Change Frequency: Typically remain consistent unless there's a change in system architecture or technology.
🔹 Stakeholders: Typically of interest to system architects, administrators, and sometimes users.
🔹 Measurement: Often quantified through metrics (e.g., 99.9% uptime, response time under 2 seconds).

For more details refer to a diagram.

⚙️ Requirements management tools:
#JIRA #Accompa #EnterpriseArchitect #Innoslate #Polarion #ModernRequirements #IBMDOORSNext #Codebeamer #JamaSoftware #Visure #ClickUp #SpiraTeam #Xebrio #Wrike #GatherSpace #IrisIntelligence #AquaALM #Aha #ReQtest #Orcanos #PerforceHelixRM #ReqView

❓ Which requirements management tools do you use in your projects or organization?

#Agile #APIDesign #APIEndpoints #APIFramework #APIGateway #APIIntegration #APIManagement #APIOptimization #APIPerformance #APIRateLimiting #APIRequest #APIResponse #APISecurity #APITesting #APIs #APIVersioning #APIAuthentication #APIAuthorization #AWS #Azure #Backend #BackendDev #BackendDevelopment #BestPractices #BigData #CleanCode #Cloud #CloudComputing #CloudArchitecture #CodeLife #Coding #CodingTips #CommandQueryResponsibilitySegregation #ContinuousDelivery #ContinuousIntegration #CQRS #CRUD #DataModeling #DataScience #DDD #DesignPatterns #Developer #DevOps #DevTools #Docker #DomainDrivenDesign #EnterpriseArchitecture #EnterpriseSolutions #EventDriven #EventSourcing #Filtering #Frontend #gRPC #GUIDs #ITArchitecture #Integration #Java #JavaScript #Jenkins #K8s #Kubernetes #Linux #Localization #MachineLearning #Microservices #Modernizing #OWASP #Pagination #Permissions #Programming #Python #RateLimiting #RoleBasedAccess #SOLIDPrinciples #SQL #Scalability #Security #Software #SoftwareArchitecture #SoftwareDevelopment #SoftwareEngineering #SoftwarePatterns #Sorting #SystemArchitecture #SystemDesign #TechInnovation #TechStrategy #TechTalk #TechWorld #UserExperience #WebDevelopment #XML

• AWS, Azure, GCP, Cloud agnostic offerings based on data type and needs

• Virtualization vs Containerization

• WebSockets.  API Architecture style

• Python List Methods

• Back-of-the-envelope estimation

1. AWS, Azure, GCP, Cloud agnostic offerings based on data type and needs

When selecting a database or data store, it's essential to consider the nature of your data. Structured data typically requires databases that can handle defined schemas and relationships. Semi-structured data, which might include formats like JSON or geospatial data, often benefits from more flexible database solutions. Unstructured data, such as text or images, might be best stored in specialized storage solutions or data lakes. Each major cloud provider, including AWS, Azure, and GCP, offers a variety of databases tailored to these data types. Additionally, there are cloud-agnostic options available for those seeking platform-independent solutions. For a comprehensive breakdown of offerings based on data types, please refer to the provided table.

2. Virtualization vs Containerization

2.1. Virtualization:

🔹Level: Operates at the hardware level.
🔹Technology: Uses hypervisors (Type 1 like VMware vSphere, Microsoft Hyper-V or Type 2 like Oracle VirtualBox) to create and run virtual machines (VMs).
🔹Components: Each VM has:
- Its own full OS instance.
- A virtual copy of all the hardware that the OS requires.
🔹Isolation: VMs are fully isolated from each other.
🔹Resource Overhead: Generally, VMs have more overhead due to the duplication of full OS instances and emulated hardware.
🔹Use Cases: Ideal for running multiple instances of entirely different operating systems or different configurations of the same OS.

2.2. Containerization:

🔸Level: Operates at the OS level.
🔸Technology: Uses container platforms (like Docker, containerd, or rkt) to create and run containers.
🔸Components: Containers:
- Share the host OS kernel.
- Package only the application and its dependencies.
🔸Isolation: Containers are isolated from each other but share the same OS kernel.
🔸Resource Overhead: Containers are lightweight with minimal overhead since they share the same OS kernel and avoid hardware emulation.
🔸Use Cases: Ideal for microservices, scalable cloud-native applications, and situations where you want to maintain consistent environments between development, testing, and production.

2.3. Key Differences:

🟢Efficiency: Containers are generally more resource-efficient than VMs.
🟢Startup Time: Containers start faster as they don't need to boot an entire OS.
🟢Portability: Containers encapsulate dependencies, making them highly portable across different stages of development and even between cloud environments.
🟢Management Tools: Virtualization uses tools like VMware vCenter, while containerization has orchestration tools like Kubernetes.

While both virtualization and containerization allow for environment isolation, they differ in their approach, efficiency, and use cases. The choice between them depends on the specific needs of a project or infrastructure.

How do security concerns differ between containerization, which shares an OS kernel, and virtualization with separate OS instances?

❓ How do security concerns differ between containerization, which shares an OS kernel, and virtualization with separate OS instances?

3. WebSockets.  API Architecture style

Ever wondered how your favorite chat apps or live trading platforms update so swiftly? The magic lies in WebSockets, an API architecture style enabling real-time, two-way communication. 🔄

3.1. Message formats:

◉ Text
◉ Binary
◉ JSON
◉ XML
◉ Custom

3.2. Advantages:

✅ Full-Duplex: Bidirectional communication
✅ Real-Time: Ideal for instant data exchange
✅ Reduced Latency: Faster data transfer
✅ Efficient: No overhead of constant connections
✅ Persistent Connection: Stays open for data
✅ Standardized: Consistent across implementations

3.3. Disadvantages

❌ Complexity: More challenging than traditional HTTP
❌ Firewall Issues: Some might block WebSockets
❌ Fallback Needs: For unsupported environments
❌ Resource Use: Can consume more server resources
❌ Security: Requires careful implementation
❌ Lacks Metadata: Unlike HTTP headers

3.4. 📋 Use cases

◉ Chat Applications: Real-time messaging between users
◉ Online Gaming: Multiplayer games with instant interactions
◉ Live Sports Updates: Real-time score and game updates
◉ Financial Trading Platforms: Live updates on stock prices and trades
◉ Collaboration Tools: Real-time document editing and collaboration
◉ Notifications: Instant alerts and updates for users
◉ Live Feeds: Streaming news, social media, or other feeds
◉ Remote Device Control: Controlling IoT devices in real-time
◉ Monitoring & Dashboards: Live updates on system or data metrics
◉ Interactive Seminars & Webinars: Real-time audience interaction

❓ With the rise of real-time applications, WebSocket has become a go-to solution for many. How do you balance the benefits of persistent connections in WebSockets with potential scalability challenges?

4. Python List Methods

Understanding these list methods is essential for efficient data manipulation in Python:

1. 📌 append() - Adds an element to the end of the list.
2. ➕ extend() - Appends elements from another list (or any iterable) to the end.
3. ⬅️ insert() - Inserts an element at a specified position.
4. ❌ remove() - Removes the first occurrence of the specified element.
5. 💥 pop() - Removes and returns the element at the specified position. If no index is specified, it removes and returns the last element.
6. 🚫 clear() - Removes all elements from the list.
7. 🔍 index() - Returns the index of the first occurrence of the specified element.
8. 🔢 count() - Returns the number of occurrences of the specified element.
9. 🔤 sort() - Sorts the list in ascending order by default. You can also specify reverse order or a custom key function.
10. 🔁 reverse() - Reverses the order of the list.
11. 📋 copy() - Returns a shallow copy of the list.

❓ How do you find the index of a value's second occurrence in a Python list?

5. Back-of-the-envelope estimation

"Back-of-the-envelope estimation" refers to a rough calculation or estimation made quickly using simplified assumptions. The term originates from the idea of jotting down a quick calculation on any available piece of paper, such as the back of an envelope. The goal is to get a ballpark figure or order of magnitude, rather than an exact number.

Here's how you can approach a back-of-the-envelope estimation:

🔘 Define the Problem: Clearly state what you're trying to estimate.
🔘 Simplify Assumptions: Make broad assumptions that simplify the problem. These assumptions should be reasonable but don't need to be precise.
🔘 Break Down the Problem: Divide the problem into smaller, more manageable parts.
🔘 Use Round Numbers: Instead of using precise numbers, use rounded numbers that are easier to work with.
🔘 Calculate: Perform the calculation using the simplified assumptions and rounded numbers.
🔘 Review & Adjust: Reflect on the result. Does it seem reasonable? If not, adjust your assumptions and recalculate.
🔘 Document Assumptions: Always note down the assumptions you made, so you (or someone else) can understand the context of your estimation.

Example 1:
🔘 Define:
- Estimation Goal: Determine the Queries Per Second (QPS) our server needs to handle based on user activity.
🔘 Simplify:
-User Behavior: Assume each user makes 2 queries per day.
- Active Users: Assume 10 million daily active users.
🔘 Break Down:
- Calculate the total number of queries made by all users in a day.
- Convert the daily query count to a per-second rate.
🔘 Round Numbers:
- For simplicity:
- Use the exact numbers provided: 2 queries and 10 million users.
- Round the number of seconds in a day from 86,400 to 100,000.
🔘 Calculate:
- Total Queries Per Day: 2×10,000,000=20,000,000
- 2×10,000,000=20,000,000 QPD
- Convert QPD to QPS using the rounded number:
- 20,000,000 QPD/100,000 seconds/day = 200 QPS
🔘 Review:
The result indicates that the server should handle approximately 200 QPS based on the given user activity and rounded seconds. This value can be cross-checked with server capabilities and real-world data.
🔘 Document:
- Assumptions Made:
- Each user makes 2 queries per day.
- There are 10 million daily active users.
- For simplicity, we rounded 86,400 seconds in a day to 100,000 seconds.

❓ How would you use a back-of-the-envelope estimation to predict the number of cars passing through a busy intersection in a day, given only the traffic flow in one hour?

#systemdesign #softwareengineering #coding #dev #Webhooks #API #BackOfTheEnvelope #Estimation #SystemDesign #QuickCalculation #RoughEstimate #DesignAssumptions #InfrastructurePlanning #CapacityEstimation #SystemScaling #PerformanceEstimation #Relational #JSON #Geospatial #TimeSeries #Graph #KeyValue #BlobStorage #DataWarehouses #DataLakes #TextSearch #Ledger #InMemory #structureddata #SemiStructureddata #Unstructureddata #AWS #Azure #GCP #DataStorage #BigData #NoSQL #SQL #kubernetes #k8s #container #VMware

• Networking & Content Delivery services in AWS, Azure, GCP, and OCI

• Multi-Master Replication

• SOAP API Architecture style

• Linux File System Structure

• OAuth 2.0 and OpenID Connect (OIDC)

1. Networking & Content Delivery services in AWS, Azure, GCP, and OCI

Networking & Content Delivery services in AWS: Approximate match rating with corresponding services in Azure, GCP, and OCI.

Networking & Content Delivery services play pivotal roles in ensuring seamless, secure, and efficient operations.

❓ How do these services align with corresponding services in Azure, Google Cloud Platform (GCP), and Oracle Cloud Infrastructure (OCI), and to what extent?

Some services have a direct equivalent, others might not have a one-to-one match, and the degree of match is based on the closest functionality.

🔘 Amazon API Gateway:
A service for developers to create, secure, and manage APIs, facilitating access to back-end services.

🔘 Amazon #CloudFront:
A global CDN delivering data and apps with low latency. Seamlessly integrated with AWS and offers flexible pricing.

🔘 Amazon Route 53:
A cloud DNS service translating domain names to IP addresses, enhancing connectivity to AWS resources.

🔘 AWS Verified Access:
Provides secure application access for corporate users without a VPN, based on user and device data.

🔘 Amazon #VPC:
Enables creation of a private AWS Cloud segment with a customizable virtual network.

🔘 Amazon VPC Lattice:
Manages service-to-service connectivity with defined policies for traffic and access.

🔘 AWS App Mesh:
Standardizes and controls AWS microservices communication, ensuring application reliability.

🔘 AWS Cloud Map:
Tracks and updates the location of dynamic resources, bolstering application uptime.

🔘 AWS Direct Connect:
Forms a dedicated connection between user premises and AWS, optimizing costs and bandwidth.

🔘 AWS Global Accelerator:
Boosts global application performance by routing traffic via the AWS network.

🔘 AWS PrivateLink:
Secures data by preventing public internet exposure and offers private AWS connectivity.

🔘 AWS Private 5G:
Integrates cellular technology into networks, with AWS supplying essential components.

🔘 AWS Transit Gateway:
Links Amazon VPCs and on-site networks via a central gateway, streamlining operations.

🔘 #AWS #VPN:
Combines AWS Site-to-Site and Client VPN, ensuring secure connections across the AWS ecosystem.

Approximate Match Rating (1-5) with equivalent services in #Azure, #GCP, #OCI.
1 = "Not a Direct Match, but Closest Available"
2 = "Somewhat Similar"
3 = "Moderately Similar"
4 = "Very Similar"
5 = "Almost Identical"

2. Multi-Master Replication

Multi-master replication is a database replication method where multiple nodes or computers store data and can update it. All nodes can respond to data queries. The system ensures data modifications made by one node are propagated to others, maintaining data consistency. It contrasts with primary-replica replication, where only one "master" node can modify data, and failover clustering, where only one server actively interacts with clients.

🌟 2.1. Advantages:

1. 🔄 Availability: Continuous operation even if one master fails.

2. 🌍 Distributed Access: Nodes can be geographically distributed, reducing latency.

3. ⚖️ Load Sharing: Handles more write traffic than a single node.

4. 🛡️ No Single Point of Failure: Multiple masters ensure continuous operation.

5. ⚡ Low Latencies: By placing nodes closer to users, write latencies are minimized.

6. 🔧 Database Upgrades: Facilitates upgrades without significant downtime.

😕 2.2. Disadvantages and Concerns:

1. 📉 Consistency Issues: Asynchronous replication can lead to eventual consistency, compromising ACID guarantees.

2. 🐢 Performance Concerns: Data synchronization can strain network bandwidth, leading to sluggish performance.

3. ⚠️ Conflict Resolution: Simultaneous updates can lead to data conflicts. Handling these conflicts becomes crucial in a multi-master setup.

4. 🧩 Complexity: Setting up and managing multi-master replication can be more complex than other replication methods.

🛠️ 2.3. Implementations:

1. 📂 Directory Services: Many use #LDAP and implement multi-master replication, such as Active Directory, #OpenDS/#OpenDJ, #OpenLDAP.

2. 🗃️ Database Management Systems: #Amazon #Aurora, #Apache #CouchDB, #ArangoDB, #Cloudant, #eXtremeDB Cluster, #Oracle, #Microsoft #SQL, #MySQL, #MariaDB, #PostgreSQL, #Ingres.

📌 2.4. Use Cases Examples:

1. 🤝 Collaborative Tools: Google Docs, where multiple users edit simultaneously.

2. 📴 Offline Databases: Clients work offline and sync when online.

Multi-master replication offers scalability, availability, and geographical distribution advantages. However, it also presents challenges like conflict resolution, consistency issues, and potential performance degradation. It's crucial to consider these factors based on specific needs and use cases.

❓ What challenges are you encountering with multi-master replication?

3. SOAP API Architecture style

SOAP, a protocol from the early days of web services, remains relevant in specific scenarios. Remember when banking systems relied heavily on SOAP for secure, reliable communication? 🏦 Financial institutions are using SOAP for transactions due to its robust security features.

3.1. Message formats:

• XML

3.2. ✅ Advantages:

1. 🔗 Standardized Protocol: Protocol-agnostic

2. 🚫 Error Handling: Integrated mechanisms

3. 🔒 Security: Adheres to WS-Security standards

4. 💾 ACID Transactions: Guarantees data integrity

5. 💡 Stateful Operations: Retains state

6. 🔍 Extensibility: Compatible with WS-* standards

7. 🌍 Language Neutral: Compatible across all languages.

3.3. ❌ Disadvantages

1. 📜 Verbosity: Large XML payloads

2. ❌ Complexity: More challenging to implement than REST

3. 🔗 Tight Coupling: Modifications can disrupt integrations

4. 🐢 Performance: Reduced speed because of XML parsing

5. 🔒 Less Flexibility: Rigid standards

6. 🎓 Learning Curve: Not the same as REST

7. 📉 Declining Popularity: Limited contemporary tools/resources.

3.4. 📋 Use cases

1. 🏢 Enterprise Solutions: Integration for expansive enterprise systems.

2. 💰 Financial Sector: Safe and ACID-adherent banking transactions. 📡 Telecommunications: Management of network devices.

3. 🩺 Healthcare: Exchange of medical records.

4. ✈️ Airline Services: Flight reservations and information systems.

5. 🌐 Web Solutions: Uniform public APIs.

6. 👥 CRM Platforms: Integration of customer management tools.

7. 💳 Payment Channels: Protected payment transactions.

8. 📦 Delivery Services: Shipment tracking and details.

9. 💾 Legacy System Adaptation: Contemporary interface for dated, intricate systems.

❓ What API architecture style are you guys currently utilizing in your system?

4. Linux File System structure

The Filesystem Hierarchy Standard (FHS) outlines the structure for Unix-like systems. Popularized by Linux distributions, it's also adopted by other Unix-like systems and maintained by the Linux Foundation. The latest version, 3.0, was released on 3 June 2015.

• 🌳 Rooted at /, the FHS organizes files and directories, even if they reside on different devices.

• 📁 Presence of certain directories depends on specific subsystems.

• 🐧 While many directories are common across Unix-like systems, the FHS descriptions are specific to Linux.

❓ Following the FHS, where should Linux app binaries and user-specific config files reside?

5. OAuth 2.0 and OpenID Connect (OIDC)

5.1. OAuth 2.0:

• 🎯 Purpose: OAuth 2.0 is primarily an authorization framework that allows third-party applications to obtain limited access to a user's resources without exposing their credentials. It's designed for token-based authorization.

• 🔑 Tokens: Uses access and refresh tokens.

• 🔄 Flows: Authorization Code Flow, Implicit Flow, Client Credentials Flow, Resource Owner Password Credentials Flow.

5.2. OAuth 2.0 Authorization Code Flow (on the image):

1️⃣ User Resource Request
2️⃣ Client Redirect
3️⃣ User Login & Permission Grant
4️⃣ Authorization Code Issuance
5️⃣ User Provides Authorization Code
6️⃣ Access Token Request
7️⃣ Access Token Response
8️⃣ Resource Request with Access Token
9️⃣ Resource Response

5.3. OpenID Connect (OIDC):

• 🎯 Purpose: OIDC is a simple identity layer built on top of the OAuth 2.0 protocol. While OAuth 2.0 is about resource access and sharing, OIDC is about user authentication. It provides apps with a way to get basic profile information about the authenticated user.

• 🆔 ID Token: OIDC introduces a new type of token - the ID Token. This token is a JWT (JSON Web Token) and contains information about the authenticated user.

• 📋 UserInfo Endpoint: OIDC provides a standard endpoint where apps can retrieve claims (user attributes) about the authenticated user.

• 🔄 Flows: OIDC supports the same flows as OAuth 2.0 but adds authentication steps to them. The most commonly used flow in OIDC is the Authorization Code Flow with the addition of the ID Token.

• 📜 Standard Claims: OIDC defines a standard set of claims (user attributes). Examples include sub (subject identifier), name, email, and more.

5.4. OAuth 2.0 + OIDC:

• 🛠️ OIDC is built on top of OAuth 2.0. In essence, while OAuth 2.0 provides the framework for access delegation, OIDC adds an identity layer on top, allowing for user authentication in addition to the OAuth 2.0 authorization functionalities.

• 🌐 It's common for modern applications to use both OAuth 2.0 for authorization and OIDC for authentication. This combination allows apps to both authenticate users and request permissions to access resources on their behalf.

❓ How are you implementing and utilizing OAuth 2.0 and OIDC in your applications?

• GraphQL API Architecture style

• Netflix. What it’s built on

• Comparison of URI, URL, and URN

• 27 Microservices Best Practices

• Master-Slave Replication

1. GraphQL API Architecture style

1.1. Message formats:

• JSON

1.2. ✅ Advantages:

1. Flexible Data 📦: Request only needed data, reducing network load.

2. Strong Typing 🔍: Clear client-server contract via a defined schema.

3. Single Endpoint 🎯: One endpoint for all interactions simplifies API usage.

4. Real-time Updates ⚡: Subscriptions support live data reflection.

5. Introspection 🔮: Self-documenting APIs for easier discovery.

6. Declarative Fetching 📋: Describe data needs without backend specifics.

7. Batch & Cache 🚀: Combine requests and optimize caching.

8. Smooth Evolution 🌱: Add features without breaking existing ones.

1.3. ❌ Disadvantages

1. Complexity 🌀: Can be overkill for simple APIs.

2. Performance ⏳: Nested queries may slow response times.

3. Caching 🧩: Fine-grained queries complicate caching.

4. Rate Limiting 🚦: Trickier with a single endpoint.

5. Learning Curve 📚: New query language to master.

6. Over-fetching 🎒: Risk of requesting excess data.

7. Errors ❗: Ambiguous or partial failures.

8. File Uploads 💾: Not natively supported.

9. Overexposure 🔓: Potential data security risks.

1.4. 📋 Use cases

1. Dynamic Data 📱: Tailor data for different clients (web, mobile, IoT).

2. SPAs 🖥️: Efficient data fetching for Single Page Applications.

3. Real-time Apps ⚡: Push updates for chats, scores, stocks.

4. Microservices 🕸️: Unified data layer across services.

5. API Gateway 🚪: Intermediary between clients and backends.

6. REST Wrapper 🔄: Add flexibility to existing REST APIs.

7. Rapid Prototyping 🚀: Speed up development with introspection.

8. CMS 📄: Flexible content querying and management.

9. E-commerce 🛍️: Manage complex product and user data.

10. Social Aggregators 🌐: Streamline fetching from multiple sources.

❓ What API architecture style are you guys currently utilizing in your system?

2. Netflix. What it’s built on

Netflix has a complex and evolving tech stack that supports its massive global streaming service.

1. Frontend

1.1. Web: Node.JS (client side), HTML 5, JS, React
1.2. Mobile: Kotlin (Android), Swift (iOS)

Communication via Federated GraphQL

2. Backend.

2.1. Cloud Infrastructure: AWS - majority of Netflix's cloud infrastructure

2.2. Microservices Architecture: Microservices (1000+ microservices), Netflix ZuuL (API GW), Netflix Eureka (Service Discovery)

2.3. Data Storage & Databases
2.3.1. Relational databases: MySQL, CockroachDB
2.3.2. NoSQL databases: Cassandra, Amazon DynamoDB
2.3.3. Content/Streaming: S3, Elastic Transcoder
2.3.4. Search Engines: ElasticSearch
2.3.5. Caching: EVCache

2.4. Big Data & Analytics
2.4.1. Stream processing systems: Kafka, Flink
2.4.2. Batch processing systems: Apache Spark
2.4.3. Data warehousing solutions: Amazon Redshift, Snowflake
2.4.4. Data Visualization: Tableau
2.4.5. Real-time analytics: Druid
2.4.5. To enhance data performance: Iceberg

2.5. Machine Learning: TensorFlow

2.6. CDN: Netflix Open Connect

2.7. Networking
2.7.1. LBs: Amazon ELB, Netflix Zuul
2.7.2. Service Mesh: Envoy

2.8. Scripting & Automation: Python, Groovy
2.9. Incident Management: PagerDuty

3. DevOps.

3.1. CI/CD: Spinnaker, Jenkins
3.2. Configuration Management: Archaius
3.3. Monitoring & Observability
3.3.1. Time-series monitoring: Apache Atlas
3.3.2. Logging systems: ELK stack
3.3.3. Tracing tools: Zipkin
3.4. Resilience & Chaos Engineering: Chaos Monkey
3.5. Version Control: Git, GitHub
3.6. Infrastructure as Code: Terraform
3.7. Collaboration & Communication: Slack, Jira, Confluence
3.8. Build Automation & Dependency Management: Gradle
3.9. Enhance the build and release process: Nebula

4. Sources:
4.1. https://netflixtechblog.com/

4.2. https://aws.amazon.com/solutions/case-studies/innovators/netflix/

4.3. https://github.com/Netflix

❓Are there any components or technologies that need to be corrected?

3. Twenty-seven (27) Microservices Best Practices

Efficiently architecting, building, and maintaining microservices is vital to optimize both human and infrastructure resources. For successful microservice implementation, it's essential to adopt practices that ensure flexibility, efficiency, and scalability:

1. Independent Deployment and Scalability: Deploy and scale each microservice separately.
2. Domain-Driven Design: Model services based on business domains to ensure alignment with business capabilities.
3. Loose Coupling & High Cohesion: Minimize inter-service dependencies and group related functionalities.
4. API Versioning & Backward Compatibility: Ensure older versions of services or clients continue to function with newer releases.
5. Service Discovery: Leverage solutions like Consul or Eureka, allowing dynamic contraction and expansion without a central agent.
6. Centralized Configuration: Manage configurations across services centrally.
7. Health Checks: Implement health endpoints for monitoring service health.
8. Distributed Tracing: Trace requests across services for diagnostics.
9. Circuit Breakers & Automatic Retries: Ensure systems are fault-tolerant and can handle service interruptions.
10. Data Consistency: Ensure data consistency across services using patterns like Saga.
11. Centralized Logging: Centralize logs for easier debugging.
12. API Gateway: Route requests and handle authentication centrally.
13. Authentication & Authorization: Implement access restriction protocols to ensure only authentic users access services.
14. Rate Limiting & Request Caching: Prevent excessive requests and reduce load on services.
15. Database Per Service: Dedicate a database to each service for decoupling.
16. Event-Driven & Asynchronous Architecture: Use message queuing services like RabbitMQ and leverage asynchronous protocols for better performance.
17. Automated Testing: Implement comprehensive testing for each service.
18. CI/CD & DevOps Practices: Automate processes and ensure development teams own and support changes from development to end-of-life.
19. Containerization & Orchestration: Use Docker and Kubernetes for deployment consistency.
20. Single Responsibility Principle: Each service should cater to one business capability.
21. Statelessness: Design microservices to be stateless for scalability.
22. Security: Ensure data encryption, API, and network security.
23. Code Maturity: Keep code within a service at a similar maturity level.
24. Separate Builds: Each microservice should have its own build and deployment pipeline.
25. Micro Frontends: Develop and deploy frontend components independently.
26. Documentation: Keep API documentation up-to-date.
27. Monitoring & Alerts: Use tools like Prometheus for monitoring and set up alerts.
28. Backup & Disaster Recovery: Regularly backup data and have recovery processes in place.

❓ Which best practices are you implementing for your microservices architecture?

4. Comparison of URI, URL, and URN.

Comparison of URI, URL, and URN.

4.1. URI (Uniform Resource Identifier)
- syntax: scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]
- Purpose: To identify a resource
- Persistence: Can be either persistent or transient
- Definition: A generic term for any type of name or address referring to a resource.
Example: mailto:ezvizi.com@gmail.com?subject=Ad Placement Request?&cc=ezvizi.com@gmail.com&bcc=ezvizi.com@gmail.com

4.2. URL (Uniform Resource Locator)
- syntax: scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]
- Purpose: To locate a resource
- Persistence: Typically transient; can change if the resource moves
- Definition: A specific type of URI that describes where a resource is located and how to access it.
Example: https://user:password@www.ezvizi.com:8080/products/page1?item=123#section2

4.3. URN (Uniform Resource Name)
- syntax: urn:<namespace-identifier>:<namespace-specific-string>
- Purpose: To name a resource uniquely
- Persistence: Meant to be persistent; doesn't change even if the resource's location changes.
- Definition: A specific type of URI that provides a persistent identifier for a resource without implying its location or how to access it.
Example: urn:isbn:0451450523

❓ Can a URI be both a URL and a URN?

5. Master-Slave Replication

The master handles both reads and writes, sending write replicas to its slaves, which are read-only. Slaves can further replicate in a hierarchical manner. If the master fails, the system shifts to read-only until a slave becomes the master or a new master is set up.

🌟 Advantages:
✅ Reading from slaves doesn't impact the master.
✅ Backups don't heavily affect the master.
✅ Slaves can resync to the master without downtime.

😕 Disadvantages:
❌Additional hardware and complexity due to replication.
❌All writes must go through the master.
❌Potential data loss and downtime if the master fails.
❌More read slaves can lead to increased replication delays.

❓ In a master-slave replication setup, if the master becomes isolated from the network due to a network partition but remains operational, and one of the slaves is promoted to a master, what strategies can be employed to handle data inconsistencies when the original master is reconnected?

If you possess insights or educational content on the following topics:

• Scalability

• Distributed Systems

• Architecture of Scalable Applications

• System Design Essentials

• Design Problems

• Machine Learning System Design

We would be honored to showcase your expertise to our audience!

Submission Guidelines:

• Content Length: Between 600-2000 characters.

• Attach 1 relevant image/animation/video link.

• Don't forget to include your LinkedIn link for proper attribution.

Please forward your submissions to ezvizi.com@gmail.com. It's a fantastic opportunity to engage with our community, which is currently in the double digits and growing steadily. Let's collaborate and enrich the knowledge pool together!

Please note, not all submissions will be published; only approved content will be featured.

What you get:

• A dedicated sponsor section in our weekly email.

• Permanent placement in our archive, ensuring long-term visibility.

• A catchy title (up to 88 characters)

  • Ex.: Win a Free Pass: Dive into the Future of Software Engineering!

• Engaging body text (less than 700 characters).

  • Ex.: Exciting times ahead for software enthusiasts! Join us at the premier Software Engineering Conference of the year. Dive deep into the latest trends, innovations, and breakthroughs in the tech world. Engage in enlightening discussions with industry leaders, witness hands-on demos, and network with peers who share your passion. Whether you're a seasoned developer or just starting out, this is the place to be for all things software. Don't miss out on this opportunity to elevate your skills, expand your horizons, and be part of the future of software engineering!

• A vibrant image with your logo included (max dimensions: 1600 x 800). Ex.

  

• A compelling call to action link and text to drive engagement.

  • Ex. call to action text: Secure Your Spot Now! Unleash the Future of Tech!

  • Ex. call to action link: [Link goes here]

This is a unique opportunity to position your brand front and center to our engaged readership. Interested? Reach out to us at ezvizi.com@gmail.com for more details.

• Webhooks API Architecture style

• Load Balancing Methods. Part 1 (7 Methods)

• HTTP Status Codes

• Fundamental Latency Metrics to Remember

• Git commands. Part 1 (21 commands)

1. Webhooks API Architecture style

1.1. Message formats:

• JSON

1.2. ✅ Advantages:

• Real-Time Notifications 🚨: Immediate alerts without polling.

• Event-Driven ⚡: Supports dynamic, event-based interactions.

• Reduced Server Load 📉: Sends data only on events.

• Configurable 🔧: Tailored to specific events.

• Simple to Use 🟢: Generally straightforward implementation.

• Asynchronous Operations 🔄: Non-blocking data transfers.

• Scalable 📈: Adapts easily with growing needs.

1.3. ❌ Disadvantages

• Security Concerns 🔒: Public endpoints may pose risks if not secured.

• Error Handling ❌: Challenges with endpoint failures or errors.

• Latency Issues 🐢: Network delays can affect real-time nature.

• Resource Use 🔄: Each call consumes server resources.

• Debugging 🔍: Complexity increases with third-party services.

• Limited Filtering 🚫: Some providers may lack detailed options.

• Ordering 📦: No guaranteed sequence for events.

1.4. 📋 Use cases

• Real-Time Notifications 🚨: Immediate alerts for events like new posts.

• CI/CD 🛠: Trigger builds, tests, and deployments on code changes.

• E-Commerce Updates 🛍: Notify on order placements, payments, or stock changes.

• CMS 📝: Alerts on content changes.

• Social Media Integration 📲: Updates on posts and interactions.

• Monitoring & Analytics 📊: Alerts for system or security issues.

• Chatbots 💬: Notifications for new messages.

• Payments 💳: Status updates on transactions.

• IoT Device Communication 🌐: Send data when specific conditions are met by IoT devices.

• Collaboration Tools Integration 🤝: Notify on task updates or new messages in collaboration apps.

❓ What API architecture style are you guys currently utilizing in your system?

2. Load Balancing Methods: Round Robin, Sticky Round Robin, Weighted Round Robin, IP Hash, Generic Hash, Least Connections, Least Time

More detailed:

2.1. 🔁Round Robin

• Distributes requests sequentially across all servers in the pool. Simple and predictable.

• Static

• Pros:

  • Easy Setup 🟢: Minimal configuration

  • Even Distribution 🔁: Suitable for testing.

  • No Monitoring Needed 📊: Rotates through servers.

  • Predictable 🔄: Can anticipate server handling.

• Cons:

  • Server Overload Risk ⚠️: Can push servers to overload if they're already heavily loaded.

  • Requires Similar Server Capacity 📏: Best when all servers have roughly the same capacity.

  • Content Uniformity Needed 📋: Requires all servers to host the same content.

• Use Cases:

  • Testing Environments 🧪: Balanced request distribution for testing.

  • Stateless Applications 🔄: For independent, session-less requests.

  • Uniform Server Capacities 📏: When all servers have similar resources.

  • Microservices 🚀: Even distribution across stateless services.

• Example (NGINX):

  • upstream backend {

       server s1.dmn.com;

       server s2.dmn.com;

       server s3.dmn.com;

    }

2.2. 🟢🔁  Sticky Round Robin

• Distributes requests sequentially but ensures a user's subsequent requests stick to the initially assigned server, maintaining session persistence.

• Static

• Pros:

  • Session Persistence 📌: Maintains user session data.

  • Better User Experience 🚀: No session timeouts or data loss.

  • Simpler App Design 🛠️: Assumes user requests hit the same server.

• Cons:

  • Potential Imbalance ⚖️: Uneven load distribution.

  • Scaling Issues 📈: Challenges when adding new servers.

  • Server Failure Impact 🔥: Disruptions if a sticky server fails.

• Use Cases:

  • Web Apps with Sessions 🛒: Maintains user carts and preferences.

  • Authentication Systems 🔐: Keeps users logged in across requests.

  • Multi-step Forms 📝: Remembers data across form pages.

  • Streaming Services 🎥: Consistent server connection during streams.

  • Online Gaming 🎮: Tracks player states and scores consistently.

• Example (NGINX+):

  • upstream backend {

    server s1.dmn.com;

    server s2.dmn.com;

    server s3.dmn.com;

    sticky cookie srv_id expires=2h
domain=.dmn.com path=/;

    }

2.3. ⚖️🔁Weighted Round Robin

• Distributes requests based on assigned server weights, favoring servers with higher capacities or priorities.

• Static

• Pros:

  1. Adaptive Distribution 📊: Suits servers with different capacities.

  2. Flexibility 🔄: Adjusts weights as server performance changes.

  3. Efficient ⚙️: Maximizes resource utilization without overloading.

• Cons:

  1. Complexity 🧩: Requires monitoring and weight adjustments.

  2. Potential Imbalance ⚠️: Incorrect weights can lead to overloads.

• Use Cases:

  1. Mixed Server Capacities 🏢: When servers have varying resources.

  2. Dynamic Environments 🌪️: Adapting to changing server performance.

  3. Traffic Prioritization 🚦: Directing more traffic to higher-performing servers.

• Example (NGINX):

  • upstream backend {  

    server s1.dmn.com weight=3;   

    server s2.dmn.com weight=2;
server s3.dmn.com weight=1;

    }

2.4. 🔗#️⃣IP Hash

• Distributes requests based on a hash of the client's IP address, ensuring consistent routing to the same server for a specific IP.

• Static

• Pros:

  • Session Persistence 📌: Clients consistently directed to the same server.

  • Predictable Distribution 🔀: Based on client IP hash, ensuring even load.

• Cons:

  • Limited Flexibility ⛓️: Hard to adjust once set up.

  • Imbalance Risk ⚠️: Some servers might get more traffic if certain IP ranges are more active.

• Use Cases:

  • Stateful Applications 🛒: Where session data needs to be retained.

  • Geo-specific Content 🌍: Serving content based on client's geographic location.

  • Security & Monitoring 🛡️: Easier tracking and management of client sessions.

• Example (NGINX):

  • upstream backend {

    ip_hash;  

    server s1.dmn.com weight=3;
server s2.dmn.com weight=2;
server s3.dmn.com weight=1;

    }

2.5. 🔳#️⃣Generic Hash

• Distributes requests using a hash of customizable inputs, offering flexible and consistent routing based on various data points.

• Static

• Pros:

  • Versatility 🌐: Hashes on diverse inputs, including text, variables, or combinations like IP-port pairs or URIs.

  • Uniform Distribution 🔀: Aims for even distribution across servers.

• Cons:

  • Complexity 🧩: Requires careful selection of hash function.

  • Potential Imbalance ⚠️: Hash collisions or poor hash functions can skew distribution.

• Use Cases:

  • Custom Inputs 🛠️: Hash based on specific application data or headers.

  • Dynamic Environments 🌪️: Where inputs for distribution change frequently.

  • Cache Distribution 💾: Ensuring cached content is evenly distributed.

• Example (NGINX):

  • upstream backend {

    hash $request_uri;  

    server s1.dmn.com weight=3;
server s2.dmn.com weight=2;
server s3.dmn.com weight=1;

    }

2.6. 🔌🔽Least Connections

• Directs traffic to servers with the fewest active connections, optimizing for server availability.

• Dynamic

• Pros:

  1. Adaptive 🌐: Directs traffic to less-busy servers.

  2. Efficiency ⚙️: Maximizes server utilization without overburdening.

• Cons:

  1. Delayed Reaction ⏱️: Might not account for sudden server load spikes.

  2. Potential Overhead 📊: Requires monitoring of active connections.

• Use Cases:

  1. Varying Server Capacities 🏢: Balances load in mixed-capacity environments.

  2. High Traffic Sites 🚦: Distributes large volumes of requests effectively.

  3. Real-time Applications 🎮: Where quick response times are crucial.

• Example (NGINX):

  • upstream backend {

    least_conn;

    server s1.dmn.com;

    server s2.dmn.com;

    server s3.dmn.com;}

2.7. ⏱️🔽Least Time

• Routes traffic to servers with the quickest response times, ensuring faster user experiences.

• Dynamic

• Pros:

  1. Fast Responses 🚀: Prioritizes quickest servers.

  2. Dynamic Adaptation 🌐: Adjusts based on server response times.

• Cons:

  1. Monitoring Overhead 📊: Requires continuous tracking of server latencies.

  2. Fluctuation Risk ⚠️: Rapid changes in response times can lead to frequent server switches.

• Use Cases:

  1. User Experience 🛍️: Ensures users get the fastest server response.

  2. High Demand Applications 🎥: For services like video streaming where latency matters.

  3. Variable Server Performance 📉: Balances in environments with fluctuating server speeds.

• Example (NGINX+):

  • upstream backend {

    least_time header;

    server s1.dmn.com;

    server s2.dmn.com;

    server s3.dmn.com;

    }

❓ What combination of load balancing methods are you guys currently implementing in your system?

3. HTTP Status Codes

HTTP status codes are three-digit responses sent by servers to indicate the outcome of a request. They categorize the result into five broad classes: informational (1xx), successful (2xx), redirection (3xx), client errors (4xx), and server errors (5xx). Understanding these codes is crucial as they provide insight into the success or failure of an HTTP request, helping diagnose issues, optimize user experience, and ensure smooth communication between client and server.

Short version:

and more detailed:

1xx: Informational 🔄

• 100 Continue 📤

• 101 Switching Protocols 🔄

• 102 Processing ⏳

• 103 Early Hints 💡

2xx: Successful ✅

• 200 OK 🆗

• 201 Created 🆕

• 202 Accepted 🔄

• 203 Non-Authoritative Information ℹ️

• 204 No Content 🚫

• 205 Reset Content 🔄

• 206 Partial Content ⏳

• 207 Multi-Status 📊

• 208 Already Reported 📢

• 226 IM Used 🔄

3xx: Redirection ➡️

• 300 Multiple Choices 🤔

• 301 Moved Permanently ➡️

• 302 Found 🔄

• 303 See Other 👀

• 304 Not Modified 🔄

• 305 Use Proxy 🚧

• 307 Temporary Redirect ➡️

• 308 Permanent Redirect ➡️

4xx: Client Errors ❌

• 400 Bad Request 🚫

• 401 Unauthorized 🔒

• 402 Payment Required 💰

• 403 Forbidden 🚷

• 404 Not Found 🕳️

• 405 Method Not Allowed ❌

• 406 Not Acceptable 🚫.

• 407 Proxy Auth Required 🔒

• 408 Request Timeout ⏰

• 409 Conflict ⚠️

• 410 Gone 🕳️

• 411 Length Required ❗

• 412 Precondition Failed ❌

• 413 Payload Too Large 📦

• 414 URI Too Long 📏

• 415 Unsupported Media Type ❌

• 416 Range Not Satisfiable 🚫

• 417 Expectation Failed ❌

• 418 I'm a Teapot ☕: April Fools' joke

• 419 Page Expired ⏳

• 420 Method Failure/Enhance Your Calm 🚫

• 421 Misdirected Request ❗     

• 422 Unprocessable Entity ❌(WebDAV)

• 423 Locked 🔒 (WebDAV)

• 424 Failed Dependency ❌(WebDAV)

• 425 Too Early ⏰

• 426 Upgrade Required ⬆️

• 428 Precondition Required ❗

• 429 Too Many Requests 🚫

• 430 HTTP Status Code 🚫

• 431 Headers Too Large 📏

• 440 Login Time-Out ⏰

• 444 No Response 🚫

• 449 Retry With 🔁

• 450 Blocked by Parental Controls 🔒  

• 451 Legal Reasons ⚖️

• 460 Client Closed Connection Prematurely 🚫

• 463 Too Many Forwarded IP Addresses 🚫  

• 494 Request Header Too Large 📏

• 495 SSL Certificate Error ❌

• 496 SSL Certificate Required 🔒

• 497 HTTP to HTTPS ❌

• 498 Invalid Token ❌

• 499 Token Required/Client Closed 🚫

5xx: Server Errors🚨

• 500 Internal Server Error 🚨

• 501 Not Implemented ❌

• 502 Bad Gateway 🚧

• 503 Service Unavailable ⛔

• 504 Gateway Timeout ⏰

• 505 HTTP Version Not Supported ❌

• 506 Variant Also Negotiates 🔄

• 507 Insufficient Storage 💾(WebDAV)

• 508 Loop Detected 🔁(WebDAV)

• 509 Bandwidth Limit Exceeded 📊

• 510 Not Extended ➕

• 511 Network Authentication Required 🔒

• 520 Unknown Error ❓

• 521 Server Is Down ⛔

• 522 Timeout ⏰

• 523 Origin Unreachable 🚧

• 524 Timeout ⏰

• 525 SSL Handshake Failed 🔒

• 526 Invalid SSL Certificate ❌

• 527 Railgun Listener to Origin 🚄

• 529 Service Overloaded ⚠️

• 530 Site Frozen ❄️

• 598 Network Read Timeout ⏰

• 599 Network Connect Timeout ⏰

❓ Imagine you're designing a new HTTP status code that indicates a server has understood the request but refuses to fulfill it, not due to authorization issues but because it deems the request to be potentially harmful or malicious. What would you name this status code, and how would you differentiate its use from the existing 403 Forbidden and 451 Unavailable For Legal Reasons codes?

4. Fundamental Latency Metrics to Remember

The numbers provided are approximate, based on figures from Peter Norvig’s article: http://norvig.com/21-days.html#answers.

It's essential to have a general understanding of the orders of magnitude rather than the exact values for the following reasons:

1. Performance Optimization: Helps identify system bottlenecks and areas for improvement.

2. System Design: Guides decisions about data placement and communication strategies in distributed systems.

3. Realistic Expectations: Sets achievable benchmarks for system operations.

4. Efficient Coding: Enables developers to write latency-aware code, reducing wasteful operations.

5. Debugging: Assists in quickly spotting performance anomalies.

6. User Experience: Directly impacts how users perceive application responsiveness.

7. Cost Efficiency: Faster operations can lead to savings, especially in cloud environments.

8. Educated Trade-offs: Allows for informed decisions when balancing performance, cost, and reliability.

❓ Guys, if you had to design a real-time collaborative document editing platform (similar to Google Docs) where multiple users from around the world can edit a document simultaneously, which latency numbers would be most critical to consider, and how would they influence your design choices to ensure a seamless user experience?

5. Git commands. Part 1 (21 commands).

Use case: Developing a new feature for a web application.

• Git config: Before starting, Alice sets her Git username and email.

git config --global user.name "Alice Smith" git config --global user.email "alice.smith@example.com"

• Git init: Alice initializes a new Git repository for her project.

git init my-web-app

• Git clone: Bob wants to collaborate with Alice. He clones her repository.

git clone https://github.com/alice/my-web-app.git

• Git status: Alice checks the status of her files.

git status

• Git add: Alice adds a new file to the staging area.

git add index.html

• Git commit: Alice commits her changes with a message.

git commit -m "Added homepage."

• Git push: Alice pushes her changes to the remote repository.

git push origin master

• Git branch: Bob creates a new branch to work on a feature.

git branch feature-navbar

• Git checkout: Bob switches to his new branch.

git checkout feature-navbar

• Git merge: Alice merges Bob's feature branch into the master branch.

git merge feature-navbar

• Git pull: Bob pulls the latest changes from the remote repository.

git pull origin master

• Git log: Alice views the commit history.

git log

• Git show: Bob checks the details of the last commit.

git show

• Git diff: Alice checks the differences between her working directory and the last commit.

git diff

• Git tag: Alice tags the current commit as a new release.

git tag v1.0

• Git rm: Bob removes a file from the repository.

git rm old-file.txt

• Git stash: Alice temporarily saves her changes to work on something else.

git stash

• Git reset: Bob undoes the last commit, keeping the changes in the working directory.

git reset HEAD~1

• Git revert: Alice undoes a specific commit by creating a new commit.

git revert commit_id

• Git remote: Bob checks the remote repositories connected to his local repository.

git remote -v

• Git fetch: Alice fetches the latest changes from the remote repository without merging.

git fetch origin

❓ Guys, imagine you're working on a feature branch in Git and you've made several commits. You then realize that one of the commits in the middle introduced a critical bug. Without reverting or affecting the subsequent commits, how would you use Git commands to isolate and address just that specific commit?

• REST API Architecture style

• Caching strategies Part 1 (5 strategies)

• Differences Between API Gateway and Load Balancer

• Linux commands. Part 1 (37 commands)

• Diagram as Code

1. REST API Architecture style

1.1. Message formats:

• XML

• JSON

• Other

1.2. ✅ Advantages:

• Stateless: Scales easily

• Cacheable: Improved performance

• Uniform Interface: Standardized use

• Scalable: Supports high request volumes

• Interoperable: Works well with HTTP

• Simple: Uses standard HTTP methods

• Widely Adopted: Many resources available

1.3. ❌ Disadvantages

• Over/Under-fetching: Fixed data returns

• Multiple Endpoints: More requests for complex data

• Versioning Issues: Changes can break clients

• Limited Flexibility: Fixed data structures

• Performance: Multiple trips for complex data.

• Stateful Limitations: Not ideal for real-time operations

• Nested Resource Complexity: Harder to manage nested data

1.4. 📋 Use cases

• Web Applications. Backend for web apps with CRUD operations.

• Mobile Apps. Backend support for mobile app data needs.

• Public APIs. Offer services to third-party developers.

• Integration. Connect different systems or services.

• Content Delivery. Distribute content to various platforms.

• Data Storage. Backend for apps needing database access.

• E-Commerce. Manage products, orders, and user accounts.

• Social Media Platforms. Handle user data, posts, and interactions.

• IoT Devices. Communicate with and manage IoT devices.

• Legacy System Interface. Modern interface for older systems.

❓ What API architecture style are you guys currently utilizing in your system?

2. Caching strategies: Cache Aside (Lazy Loading), Read-Through, Write-Through, Write-Around, Write-Behind (Write-Back)

2.1. 💤📌Cache-Aside (Lazy Loading)

• Data is loaded into the cache on demand. If data is not found in the cache, the application retrieves it from the datastore and then stores it in the cache.

• Pros:

  • 🎯 Only requested data is cached.

  • 🔄 Cache can be easily refreshed.

• Cons:

  • ❌ Initial cache miss incurs a latency penalty.

  • 🔄 Manual intervention required for cache updates.

2.2. 📖🔍Read-Through

• The cache is responsible for loading data from the datastore when a cache miss occurs.

• Pros:

  • 🔄 Automatic data loading.

  • 🎯 Consistent cache and datastore data.

• Cons:

  • ❌ Initial cache miss can slow down the application.

2.3. ✍️🔍Write-Through

• Data is written to both the cache and the datastore synchronously (simultaneously).

• Pros:

  • ✅ Immediate data consistency.

  • 🔄 Automatic data updates.

• Cons:

  • ❌ Slower write operations due to dual writes.

2.4. ✏️🔄 Write-Around

• Data is written directly to the datastore, bypassing the cache. The cache is updated later or during subsequent read operations.

• Pros:

  • ⚡ Faster initial write operations.

  • 🎯 Reduces cache churn for less frequently accessed data.

• Cons:

  • ❌ Subsequent reads might suffer from cache misses.

2.5. ✏️🔙Write-Behind (Write-Back)

• Data is first written to the cache, and then asynchronously written to the datastore.

• Pros:

  • ⚡ Fast write operations.

  • 🔄 Asynchronous updates optimize datastore writes.

• Cons:

  • ❌ Risk of data loss if cache fails before async write.

❓ What combination of caching strategies are you guys currently implementing in your system?

3. Differences Between API Gateway and Load Balancer

While both API Gateways and Load Balancers manage incoming requests, they serve different primary roles. In many architectures, especially in microservices, you'll often find both an API gateway and a load balancer working in tandem. An API Gateway focuses on API management, offering a suite of tools for processing and directing API calls. In contrast, a Load Balancer's main job is to distribute traffic to ensure optimal server performance and availability.

API Gateway: 🌐 Primary Role: Manages and processes API requests.

🔹 Functions:

• 🚦 Request Routing: Directs requests to the appropriate service.

• 🧩 API Composition: Aggregates data from multiple services.

• ⏳ Rate Limiting: Limits request frequency.

• 🔒 Security: Manages authentication and API keys.

• 🚀 Caching: Improves response times with stored data.

• 🔄 Request/Response Transformation: Modifies data formats.

• 📊 Analytics & Monitoring: Tracks API usage.

• 🔍 Service Discovery: Identifies available services.

• ❌ Error Handling & Retry: Manages failed requests.

• 🔧 Protocol Translation: Converts communication protocols.

🔹 Use Cases: Microservices, API management, data aggregation, securing APIs.

Load Balancer: ⚖️ Primary Role: Distributes incoming network traffic.

🔹 Functions:

• 🌊 Traffic Distribution: Prevents server overloads.

• ❤️ Health Checks: Monitors server health.

• 🔐 SSL Termination: Handles SSL/TLS processing.

• 🧬 Session Persistence: Maintains client-server connection.

• 📡 Layer 4 and Layer 7 Load Balancing: Operates at transport and application layers.

🔹 Use Cases: High availability, fault tolerance, application scaling, large-scale web traffic management.

Key Differences:

• 🎯 Focus: API gateway = API management. Load balancer = traffic distribution.

• 🔍 Granularity: API gateway = fine-grained control. Load balancer = broader network level.

• 🛠️ Features: API gateway = tailored for API management. Load balancer = traffic and health management.

In modern architectures, especially microservices, both API gateways and load balancers often work together. The load balancer manages traffic, while the API gateway handles API requests.

❓ Are you guys utilizing both API gateways and load balancers in your infrastructure?

4. Linux commands. Part 1 (37 commands).

1. File Operations 📂:

   • ls 📄: List directory contents

     • Example: ls -l

   • cp 📋: Copy files and directories

     • Example: cp source.txt destination.txt

   • mv 🚚: Move or rename files and directories

     • Example: mv oldname.txt newname.txt

   • rm 🗑️: Remove files or directories

     • Example: rm unwanted.txt

   • touch ✍️: Create an empty file

     • Example: touch newfile.txt

   • cat 📖: Concatenate and display file content

     • Example: cat file.txt

   Directory Operations 📁:

   • pwd 📍: Print working directory

     • Example: pwd

   • cd 🚪: Change directory

     • Example: cd /home/user/documents

   • mkdir 🆕📁: Make directories

     • Example: mkdir new_directory

   • rmdir 🚫📁: Remove empty directories

     • Example: rmdir empty_directory

   System Info ℹ️:

   • uname 🖥️: Display system information

     • Example: uname -a

   • top 📊: Display system tasks

     • Example: top

   • df 💽: Disk space usage of file system

     • Example: df -h

   • free 🧠: Display memory usage

     • Example: free -m

   • ps 🔄: Display process status

     • Example: ps aux

   Permissions 🔒:

   • chmod 🔑: Change file mode bits

     • Example: chmod 755 script.sh

   • chown 👤: Change file owner and group

     • Example: chown user:group file.txt

   • chgrp 👥: Change group ownership

     • Example: chgrp group file.txt

   Networking 🌐:

   • ping 📡: Send ICMP ECHO_REQUEST to network hosts

     • Example: ping google.com

   • netstat 🌐📊: Network statistics

     • Example: netstat -tuln

   • ifconfig 🌐🛠️: Display or configure a network interface

     • Example: ifconfig eth0

   • ssh 🛡️: Secure shell client (remote login program)

     • Example: ssh user@hostname

   • scp 📤: Secure copy (remote file copy program)

     • Example: scp file.txt user@hostname:/path/

   Compression/Archiving 🗜️:

   • tar 📦: Tape archiver

     • Example: tar -czvf archive.tar.gz folder/

   • gzip 🗜️: Compress or expand files

     • Example: gzip file.txt

   • gunzip 🗜️🔓: Decompress files

     • Example: gunzip file.txt.gz

   • zip 🤐: Package and compress files

     • Example: zip archive.zip file1.txt file2.txt

   • unzip 🤐🔓: Extract compressed files in a ZIP archive

     • Example: unzip archive.zip

   Package Management 📦:

   • apt-get 📥: APT package handling utility (Debian-based systems)

     • Example: sudo apt-get install package_name

   • yum 📥: Package manager for RPM-based systems

     • Example: sudo yum install package_name

   • dpkg 📥: Package manager for Debian

     • Example: sudo dpkg -i package.deb

   Text Processing 🔍:

   • grep 🔎: Search text

     • Example: grep "pattern" file.txt

   • sed ✂️: Stream editor

     • Example: sed 's/old/new/g' file.txt

   • awk 📝: Pattern scanning and text processing language

     • Example: awk '{print $1}' file.txt

   Help & Output 📘:

   • man 📚: Display manual pages

     • Example: man ls

   • echo 🔊: Display a line of text

     • Example: echo "Hello, World!"

   Process Management ❌:

   • kill ☠️: Terminate processes

     • Example: kill -9 12345 (where 12345 is a process ID)

❓ Which Linux commands do you find most frequently used in your daily operations?

5. Diagram as Code

Diagrams (Diagram as Code) allows you to create cloud system architecture diagrams using Python.

Originally designed for quick prototyping of new architectures without specialized design tools, it can also be employed to visualize or describe existing systems. The tool is versatile, supporting a wide range of major cloud providers like AWS, Azure, GCP, Kubernetes, Alibaba Cloud, and Oracle Cloud, as well as On-Premise nodes, SaaS platforms, and key programming languages and frameworks. One of its standout features is its compatibility with version control systems, enabling you to track changes to your architecture diagrams over time. Note: While Diagrams excels at creating architecture diagrams, it neither manages actual cloud resources nor generates cloud formation or terraform scripts. For a deeper dive, check out the optional 6-minute video below.

❓ Which tool or software are you guys utilizing to generate diagrams programmatically?