EZF-006. Should You Use Kubernetes?; CQRS. Command Query Responsibility Segregation; gRPC API Architecture; API Design: 51 ✔️DO's and ❌DON'Ts; Functional Requirements (FR) vs Non-functional (NFR)

Nov 04, 2023

Recap on system design for this week:

Should You Use Kubernetes?
CQRS. Command Query Responsibility Segregation
WebSockets. API Architecture style
API Design: 51 ✔️DO's and ❌DON'Ts
Functional Requirements (FR) vs Non-functional (NFR)

1. Should You Use Kubernetes?

Kubernetes (or K8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes is an immensely powerful tool, but whether you should use it depends on your specific needs and constraints. Here are some considerations.

Why You Should Consider Kubernetes:
🚀 Automated Deployment: Enables streamlined and consistent application rollouts.
📈 Scalability: Effortlessly scale applications based on demand, ensuring efficient utilization of resources.
⛑ Self-Healing: Provides automatic recovery from application or infrastructure failures.
🌍 Portability: Ensures consistent deployments across various environments, be it on-premises or different clouds.
🔀 Service Discovery & Load Balancing: Offers efficient traffic management and container discoverability.
🌱 Resource Efficiency: Ensures optimal use of infrastructure resources, potentially saving costs.
💽 Storage Orchestration: If you require seamless integration with various storage solutions.
🔐 Secrets Management: Securely manage sensitive data, separate from application configurations.
🔄 Batch Execution: Efficiently manage batch processing tasks, beneficial for data processing workloads.
🔌 Extensibility: Adapt and customize according to specific needs using its vast ecosystem.

Why You Might Hesitate:
🤔 Complexity: The steep learning curve can be intimidating and may demand significant effort and time.
🐌 Resource Overhead: Might be overkill for small applications due to its resource demands.
🔒 Security Concerns: Requires intricate configurations to ensure security, and mistakes can be costly.
🕸️ Networking Issues: Network configurations, especially in hybrid or on-premises environments, can be challenging.
💾 Storage Challenges: Stateful applications have added complexities, even though Kubernetes has provisions for them.
⚙️ Upgrade Concerns: Upgrades may introduce incompatibilities or require significant planning.
☁️ Vendor Dependencies: There's a potential for cloud-specific lock-in, especially when using provider-specific extensions.
🛠️ Tool Fragmentation: The vast ecosystem can be overwhelming, making it hard to choose the right tools.
📊 Monitoring & Logging: The native capabilities may not be sufficient, necessitating external tool integrations.
🕴️ Operational Costs: It might demand specialized teams for management, especially at a larger scale.

Your decision should weigh the pros and cons based on your organization's specific needs, the scale of operations, and available resources.

❓ Does Kubernetes seem like the right fit for your current and future goals? 🤷‍♂️

2. CQRS. Command Query Responsibility Segregation

2.1. Definition.

CQRS (Command Query Responsibility Segregation) - a software architectural pattern that segregates operations that modify data (commands) from operations that retrieve data (queries).
CQRS is a powerful pattern for certain types of applications but may be an overkill for simpler ones. Its adoption should be weighed against the specific needs of the project, considering factors like system complexity, team expertise, and performance requirements. It's not a one-size-fits-all solution and is best applied when the benefits outweigh the costs and complexity.

2.2 Concepts 💡
- Command: Represents actions or changes, modifies data but does not return data, typically results in a change to the application’s state.
- Query: Represents data retrieval requests, returns data but does not modify it.

2.3. Benefits:
- Scalability: Independent scaling of read and write operations, useful for high load scenarios.
- Flexibility: Different data storage choices for read and write.
- Simplified Code: Segregation allows for a cleaner codebase.
- Enhanced Security: Applying distinct security measures for read and write operations.

2.4. Disadvantages:
- Complexity: Can be overkill for simple CRUD apps.
- Learning Curve: Steep for newcomers.
- Dev Effort: More code due to separation.
- Data Duplication: Risk of double storage.
- Misapplication: Risk without domain knowledge.
- Costs: Increased infrastructure expenses.

2.5. Challenges:
- Data Sync: Balancing data between command and query sides.
- Eventual Consistency: System might not be immediately consistent post-command.
- Testing: Ensuring data integrity during command failures.
- Errors: Complex error handling due to segregation.

2.6. Common Use Cases:
- Highly Scalable Systems. Systems needing separate read and write scaling.
- Complex Business Logic. Distinct domain logic for writing and reading.
- Event-Driven Architectures. Often combined with Event Sourcing.

2.7. Related Patterns & Concepts:
- Event Sourcing: Storing changes as events, great with CQRS for system history.
- Domain-Driven Design (DDD): Focuses on modeling business domains, CQRS can be a key pattern within DDD.

❓ Does CQRS seem like the right fit for your current project? 🤷‍♂️

3. gRPC. API Architecture style

gRPC, an exceptional open-source framework crafted by Google, is revolutionizing the realm of microservices. Let's delve into the reasons behind its rising prominence::

3.1. Message formats:
◉ Protocol Buffers

3.2. 🌟Advantages:
✅ High Performance: Uses Protocol Buffers
✅ Language Agnostic: Supports multiple languages
✅ Bi-Directional Streaming: Real-time communication
✅ Deadlines/Timeouts: Clients set RPC wait times
✅ Pluggable: Supports authentication, load balancing
✅ Built-in Authentication: Uses SSL/TLS

3.3. 😕. Disadvantages:
❌ Learning Curve: Needs understanding of gRPC
❌ Browser Support: Limited without gRPC-Web
❌ HTTP/2 Dependent: Not supported everywhere
❌ Complexity: Might be overkill for simple services
❌ Tooling: Less extensive than REST
❌ Interoperability: Limited with non-gRPC systems

3.4. 📋Use Cases:
◉ Microservices: Efficient inter-service communication
◉ Real-time Apps: Supports real-time data exchange
◉ Mobile Clients: Optimized communication for mobile backends
◉ Point-to-Point: Direct system-to-system communication
◉ Polyglot Systems: Supports multiple programming languages
◉ Cloud-Native Apps: Integrates with cloud-native platforms
◉ Service Mesh: Used in architectures like Istio
◉ Browser Clients (gRPC-Web): Web apps communicate with gRPC services
◉ IoT Communication: Efficient server-to-IoT device communication

❓ Guys, have you integrated gRPC in your projects? How did it transform your workflow?

4. API Design: 51 ✔️DO's and ❌DON'Ts.

APIs are the backbone of modern applications, enabling seamless integration and communication between systems.
❓ But how can we ensure our APIs are user-friendly, efficient, and secure?

4.1. Requirement Gathering
✔️DO: Identify all potential users of the API. Example: E-commerce API users could be customers, administrators, and suppliers.
❌DON'T: Assume you know all the requirements without consulting stakeholders

4. 2. Identify the Main Objects/Entities
✔️DO: Break down the application into core entities. Example: Users, Products, Orders, etc
❌DON'T: Overcomplicate with too many entities initially

4.3. Define the Endpoints
DO: Use clear, intuitive endpoint names. Example: GET /products to retrieve products
❌DON'T: Use ambiguous endpoint names. Bad Example: GET /getp

4.4. Resource Names (Nouns)
✔️DO: Use nouns. GET /products @app.route('/products', methods=['GET']) def get_products():
❌DON'T: Use verbs. Bad Example: GET /getProducts @app.route('/getProducts', methods=['GET']) def get_products():

4.5. Plurals
✔️DO: GET /users/123
❌DON'T: Bad Example: GET /user/123

4.6. Idempotency
✔️DO: Ensure methods like PUT and DELETE are idempotent. Multiple identical requests should have the same effect as one request. Multiple PUT /products/123 with the same data shouldn't create multiple products.
❌DON'T: Allow multiple identical requests to produce different outcomes. Bad Practice: Each PUT /products/123 creating a new product

4.7. Versioning
✔️DO: GET /v1/products
❌DON'T: Bad Example: GET /productsNew

4.8. Soft Deletion
✔️DO: DELETE /products/123 // Marks as deleted, doesn't actually remove GET /products?includeDeleted=true
❌DON'T: Immediately and permanently delete data without a backup or recovery option.

4.9. Pagination
✔️DO: For endpoints that can return a large number of results, always implement pagination GET /products?page=2&limit=10
❌DON'T: Return all records without pagination. Bad Practice: GET /users returning thousands of records.

4.10. Sorting
✔️DO: Allow sorting by different fields /products?sort=price_asc
❌DON'T: Lack flexibility in sorting options

4.11. Filtering
✔️DO: Allow filtering on relevant fields. /products?category=electronics
❌DON'T: Return all data without filtering options

For additional 40 DOs and DON'Ts, refer to the slide.

5. Functional Requirements (FR) vs Non-functional (NFR)

5.1.🛠 Functional Requirements:
🔹 Definition: Describe the specific functionalities or tasks a system should perform.
🔹 Nature: Focus on WHAT the system must do.
🔹 Examples:
📝 User registration
🔍 Search functionality
💳 Payment processing
🔹 Focus: Features and capabilities.
🔹 Documentation: Use cases, user stories, functional specifications.
🔹 Validation: Through functional testing like unit testing and integration testing.
🔹 Change Frequency: Can change based on business needs and user feedback.
🔹 Stakeholders: Often of primary interest to users, business analysts, and developers.
🔹 Measurement: Can be validated as either 'met' or 'not met'.

5.2.🌟 Non-Functional Requirements:
🔹 Definition: Define how a system should behave, setting standards for system performance, reliability, etc.
🔹 Nature: Focus on HOW the system does it.
🔹 Examples:
⚡ Response time
📈 Scalability
🔒 Security
👥 Usability
🔹 Focus: Quality attributes or system properties.
🔹 Documentation: Performance benchmarks, SLAs, quality attributes.
🔹 Validation: Through non-functional testing like performance testing and security audits.
🔹 Change Frequency: Typically remain consistent unless there's a change in system architecture or technology.
🔹 Stakeholders: Typically of interest to system architects, administrators, and sometimes users.
🔹 Measurement: Often quantified through metrics (e.g., 99.9% uptime, response time under 2 seconds).

For more details refer to a diagram.

⚙️ Requirements management tools:
#JIRA #Accompa #EnterpriseArchitect #Innoslate #Polarion #ModernRequirements #IBMDOORSNext #Codebeamer #JamaSoftware #Visure #ClickUp #SpiraTeam #Xebrio #Wrike #GatherSpace #IrisIntelligence #AquaALM #Aha #ReQtest #Orcanos #PerforceHelixRM #ReqView

❓ Which requirements management tools do you use in your projects or organization?

#Agile #APIDesign #APIEndpoints #APIFramework #APIGateway #APIIntegration #APIManagement #APIOptimization #APIPerformance #APIRateLimiting #APIRequest #APIResponse #APISecurity #APITesting #APIs #APIVersioning #APIAuthentication #APIAuthorization #AWS #Azure #Backend #BackendDev #BackendDevelopment #BestPractices #BigData #CleanCode #Cloud #CloudComputing #CloudArchitecture #CodeLife #Coding #CodingTips #CommandQueryResponsibilitySegregation #ContinuousDelivery #ContinuousIntegration #CQRS #CRUD #DataModeling #DataScience #DDD #DesignPatterns #Developer #DevOps #DevTools #Docker #DomainDrivenDesign #EnterpriseArchitecture #EnterpriseSolutions #EventDriven #EventSourcing #Filtering #Frontend #gRPC #GUIDs #ITArchitecture #Integration #Java #JavaScript #Jenkins #K8s #Kubernetes #Linux #Localization #MachineLearning #Microservices #Modernizing #OWASP #Pagination #Permissions #Programming #Python #RateLimiting #RoleBasedAccess #SOLIDPrinciples #SQL #Scalability #Security #Software #SoftwareArchitecture #SoftwareDevelopment #SoftwareEngineering #SoftwarePatterns #Sorting #SystemArchitecture #SystemDesign #TechInnovation #TechStrategy #TechTalk #TechWorld #UserExperience #WebDevelopment #XML