Some Interesting Software Stories I experienced

Created by a Stable Diffusion model trained by Andrew W. Sun

I feel fortunate to have experienced the recent technological revolutions. My career began in the early 2000s and has spanned through advancements in real-time computing, cloud-native architectures, data-oriented and data-driven computing, and more recently, the fields of AI and ML.

Over the last twenty years or so, I’ve evolved from a software engineer to a solution architect, contributing to the design and development of many services, applications, and solutions, from traditional monolithic applications to cloud-native microservices.

Along the way, as I worked with various project teams, I experienced some interesting software design, development stories that I want to share to benefit software development practices.

Digital transformation and public cloud

Digital Transformation has been a important topic of discussion among our partners and clients across various industries, from healthcare, oil & gas, to retail.

However, in my experience working with these clients, discussions about digital transformation for their businesses have mostly centered around public cloud solutions such as AWS, Azure, or GCP. Occasionally, considerations have extended to include edge deployments from these cloud providers, for example, AWS Outposts, Azure Stack, or GCP Anthos, but these discussions have been the primary focus.

Based on my experience, designing a hybrid cloud solution from beginning is often the best approach for business across various use cases and scenarios. This strategy addresses concerns related to data sovereignty, performance requirements, and other business considerations, ensuring sustainability, scalability, and future enhancements for the organization.

Hybrid cloud architecture extends beyond a simple parent-child relationship between public cloud and edge deployment. It encompasses a broader ecosystem that includes public cloud, private cloud, edge computing, and on-premises infrastructure. This holistic approach allows organizations to leverage the strengths of each environment while ensuring seamless integration and interoperability across the entire hybrid cloud landscape.

When considering hybrid cloud design and architecture, several angles should be taken into account:

1. Technology to use to facilitate integration across different environments. For instance, what platform to use to access data and how to deploy and use ML models cross public and private clouds.
2. Data sources: how data sources are distributed, on public cloud, private cloud, or on-prem. Any legacy databases?
3. Security, including data governance and model management
4. Consistent operational behaviors crossing different clouds
5. Reliability and performance from any potential end point
6. Sustainability.

Please check some of my published blogs.

Created by a Stable Diffusion model trained by Andrew W. Sun

Vision, Agile, and Two-weeks release cycle

Software project management often presents various dilemmas.

Recently, I’ve been working with a team that is developing an edge solution using a combination of traditional software engineering and machine learning technology. The team adopts an agile methodology for project release planning and management, with 2-week sprint and release cycles.

Everything seemed to be alright till one day, when I discussed with component teams about the future architecture for client extension and potential scaling of solution to meet marketing requirements, most team leaders responded with, “my team is following the agile process, we are just focusing on our component for the next couple of releases/sprints.” — so, no vision.

So, the agile emphasizes focusing on the near term, right? This wasn’t entirely the case when a high-priority request arrived from the market and the Product Manager team asked the development team to evaluate its support, the development team leaders pushed back, stating, “We are focusing on the planned items for the next couple of sprints and cannot work on this request quickly.” — so, a deviation from true “agile”.

While, team focus on sprint plan and development is totally correct, but if project architect, team leaders do not have holistic vision and future plan, many times the development work may become siloed and lead to increased technical debt. Couple years ago, I published a blog titled “MVP vs MVP”, that discussed this topic in detail.

From another point, true agility entails teams being dynamic and capable of adjusting their workload to align with market and customer requirements. This is especially crucial in today’s rapidly evolving market landscape, where changes occur swiftly and with increasing frequency. The best project management practices involve having a plan in place while also possessing the flexibility to adapt the plan according to market needs, like the example of project management story I published before.

So, a software development team with 2-weeks (or 1-week, 4-weeks, etc) sprint and release cycle does NOT mean that the team embodies the true agile spirit — their “agile” project management is more like drawing a tiger using a cat, signifying a mismatch between the methodology and its intended principles.

Real-time computing and Event-driven process

In today’s data-oriented IT landscape, real-time data processing is an unavoidable topic. There are many real-time designs approaches tailored to different business cases. However, utilizing a real-time design in a mismatched business case often leads to suboptimal business outcomes.

I worked on a project which creating a solution for real-time customer audio request processing. Upon joining the project, I was surprised to discover that the solution was utilizing an Event-driven design instead of a streaming data process approach.

When the solution was deployed on the customer cluster, its central component was an MQTT broker, which managed data flows as message events, as depicted in the image below:

However, the actual flow of customer audio request data was intended to be as follows:

When I asked about how come using an event-drive design for this straightforward streaming data flow, the team’s response was that MQTT can handle realtime events.

Yes, it is true that a message broker can handle realtime message events. In fact, many realtime data processing systems use Kafka, which is totally suitable. However, it’s important to recognize that not all real-time processing cases are best suited for a message broker. In scenarios where data flows along a specific path from one service to another, an event-driven design may introduce additional processing overhead, unnecessary failure points, and increased maintenance and resource usage.

Microservice vs Containerized Service

Microservice architecture is fundamental to cloud native solutions, as it provides flexibility and agility to adapt to rapidly changing markets.

However, containerizing a service or observing an application running in a containerized environment with multiple deployments, statefulsets, or pods does not necessarily indicate that the solution adheres to microservice design principles.

The keys to microservice architecture are:

1. Structuring an application as a collection of services
2. Services loosely coupled.
3. Services independently deployable.

I encountered many instances where, due to IT strategy, applications were mandated to run within a containerized environment such as Kubernetes. Consequently, teams broke down applications into a collection of services. However, despite this breakdown, the design often still adhered to traditional monolithic approaches, with services tightly coupled. Updates to one service generally affect others, making all services have to be released together.

I would say it as “monolith with microservice skin” (wolf in sheep’s skin :-), as it lacks the benefits of a true microservices design. Instead, it introduces additional workload for cross-service communication and maintenance, impacts the performance of the application in the field.

Push and Pull models

With microservice architecture and distributed deployment environments, many services must make design decisions regarding whether to use a push or pull approach for remote requests.

This topic has been widely discussed, and a simple Google search for “software design push vs pull model” yields tons of blogs and discussions.

In general, if we consider the side with the data/information source as the server and the side that wants to retrieve data/information as the client:

1. If the server needs to send data/information to all clients, use push. For instance, broadcasting a certain data event to all connected clients.
2. If different clients have different demands of data, use pull. For example, different edge clusters need to retrieve distinct data from a cloud service.
3. If client need some event updates from server, but server events can happen irregularly, push generally good. For example, a client listen and the server push notifications to the client.
4. If a client requires event updates from the server, but server events occur irregularly, push is generally preferred. For instance, a client listens for event update notifications, and the server pushes notifications to the client as they occur.
5. However, if events occur frequently and the client needs to manage the incoming flow, pull should be used. For example, if an event occurs every second on the server side, but the client only needs to update its state based on the latest event every minute, it’s more efficient for the client to pull the event information every minute rather than receiving notifications every second.
6. (many other best practices, please search online…)

The choice between push and pull models for communication between microservices is crucial, and failure to design it properly can lead to performance issues and hinder the scalability of services in production environments. It’s essential to carefully consider the requirements of each service and the nature of data exchange to ensure optimal performance and scalability.

Above I listed some real-world stories I experienced in my software architecture and development career. Hopefully it can be beneficial to others in the software development community.