Architecture Considerations When Creating Software Solutions

Recently, I wrapped my blog series about data oriented computing, and published couple blogs about data oriented computing architecture patterns and case study.

As a solution/product architect, when we design a software solution, product, or application, think of just its product architecture pattern is not enough, but also its ecosystem (connections and integrations with its environmental services), and delivery, operation model.

Based on my working experience, I want to use this blog to briefly discuss what we need to consider when creating a software solution or product:

Solution itself:

1. cloud native and hybrid cloud architecture
2. data oriented computing
3. embedded ML

Solution delivery and management model:

1. devOps/MLOps/SRE
2. business deployment model

Cloud native and hybrid cloud architecture

In order to build and operate the solution or application in a more agile, scalable, and resilient way, and take full advantage of the benefits of cloud computing environments, cloud native is the first consideration.

The term “cloud native” was defined by the Cloud Native Computing Foundation (CNCF) as an approach to build and run applications that fully exploit the advantages of the cloud computing model. The approach includes several key principles:

1. Microservices architecture: Applications are broken down into components called microservices that can be developed and deployed independently.
2. Containerization: These microservices are packaged as lightweight, portable containers that can be easily moved between different computing environments.
3. Dynamic orchestration: These microservices are managed and orchestrated dynamically, allowing for flexible scaling and efficient resource utilization.
4. DevOps practices: (will discuss this in later section).

Based on market and my experience working with customers, almost all business cases need solution to run or integrate with different cloud environments. With digital transformation, many customer data are saved in public cloud, private cloud, and even on-prem private lab. Cloud native applications need to leverage environmental services such as databases, messaging systems, and caching to offload infrastructure management, improve application performance, and fulfill certain business cases following data security policies.

The application shall be seamlessly integrate with ecosystem services and bring consistent behavior and the necessity of solution operation cross multiple different cloud environments — hybrid cloud solution.

A hybrid cloud solution is a computing environment that combines both public and private cloud infrastructure, allowing organizations to take advantage of the benefits of both.

In a hybrid cloud solution, some applications and data are hosted in a private cloud, which is typically owned and operated by the organization itself, while others are hosted in a public cloud, which is owned and operated by a third-party provider such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP).

The benefits of a hybrid cloud solution are obvious:

1. Flexibility: Organizations can bring application to their data based on factors such as performance requirements, security needs, and cost considerations.
2. Security: Organizations can keep sensitive data and applications on-premises, while still taking advantage of the public cloud’s security features, such as DDoS protection, encryption, and identity and access management.
3. Consistent: Hybrid cloud solution bring consistent behavior of data computing cross different cloud to customer.
4. Cost savings for operations and resources: Organizations can optimize their cloud spending by using the public cloud for non-sensitive workloads or for peak capacity requirements, while using the private cloud for workloads that require high performance or compliance with specific regulations. Consistent operation behavior will save customer for training and application management.

Implementing a hybrid cloud solution will need considerations of following requirements when designing an application architecture:

1. Cloud agnostic: the application and its dependencies shall not dependent on services on a specific cloud provider environment.
2. Flexible interface design using standard protocol so the application can integrate with services from different environments.
3. Consistent end user experience: CLI, dashboard or console.
4. Data can be easily moved between different cloud environments as needed.
5. Edge applications integrate with Cloud services.

Data oriented computing

To save some space, please check my blog:

Data Oriented Computing: Architecture Patterns

Embedded ML

Embedded machine learning (ML) generally refers to the use of machine learning algorithms and models in edge devices or systems. I want to extend it into a general concept for any software solution to use machine learning algorithms and models.

The goal of embedded ML is to enable these solutions and software products to make intelligent decisions. This is particularly important to resolve some business problems that not easy to resolve by traditional code logic.

To achieve embedded ML, specialized hardware and software architectures are often required to optimize the model operation, performance and resource efficiency. For example:

1. Using containerized approach to deliver and deploy ML models independent of software product components (microservices) delivery and deployment.
2. Using Model-View-Controller (MVC) pattern to separate data processing, ML model and presentation layers. This approach helps in developing the application and embedded ML models with better scalability, maintainability and ease of testing.

DevOps, MLOps, and SRE

DevOps, MLOps, and SRE (Site Reliability Engineering) are all well known and related concepts that aim to improve the operations, reliability, scalability, and efficiency of software systems.

There is overlap between these three disciplines, as they all emphasize the importance of collaboration, automation, and monitoring in software development and operations. For example, MLOps can be seen as a specialization of DevOps that focuses on machine learning models with a loop of feedback from product to improve models with retrain, while SRE can be seen as a broader discipline that encompasses both DevOps and MLOps, with a focus on ensuring the reliability of the entire system.

I won’t go into the details of these concepts, as they are well known. However, as we explore cloud native computing, data-oriented computing, and embedded ML, I want to emphasize the following points:

1. These three areas need to be seamlessly integrated. Like image below shown a pipeline of DevOps and MLOps, with SRE principles.
2. Automation is critical.
3. Data oriented: at a published blog, I suggested to rename SRE to DRE: Data Oriented Computing — DRE, not SRE.

Business deployment model

Based on the nature of the solution or product, we also need to think of its deployment model.

The buzzword for this topic is as-a-Service, for example, software-as-a-service (SaaS).

Again, I will not go into these concepts, but point out some considerations when supporting a hybrid cloud solution or product.

Most of aaS pipelines are push model: the software product go through the release pipeline, and “push” to product environments, public cloud or on-prem.

However, to support hybrid cloud deployment, specifically with edge deployment of thousands of clusters or devices, the “push” model may not be scalable, so a “pull” model can be a more viable option.

When using “pull” model, we need to setup a light-weighted agent at the edge cluster. The agent can:

1. monitor the product release and pull release to upgrade as business need.
2. operate product locally following customer configurations.

The “pull” model moves the software release management and operations to the edge sites and makes it possible to scale to thousands or more sites.

Conclusions

As I mentioned in previous blog posts on MVP and MVP, as an architect working on a software solution or product, it’s important to have a holistic vision in order to ensure the solution fits the market.

When architecture a software solution or product, we need to consider following areas to make it succeed:

Solution itself:

1. cloud native and hybrid cloud architecture
2. data oriented computing
3. embedded ML

Solution delivery and management model:

1. DevOps/MLOps/SRE and business deployment model