A Project Management Story

How did I finish a six-month project within three months

Created by a Stable Diffusion model trained by Andrew W. Sun

When a team working on a project, no matter the team within a company or a community group, there are always discussions about project management, meeting tight deadlines, and risk mitigation, etc. Having participated in many such discussions, I believe that one projects I led several years ago can serve as a good example of effective project management, illustrating how to successfully navigate these challenges from end to end.

At early August 2020, I was assigned to convert an IBM Consulting project into an IBM Data and AI product (IBM Open Data for Industries, ODI), and make it onboard IBM Cloud Pak for Data (CPD). Based on marketing and partner requirements, we needed to GA it at early November 2020. Only three months to go, which is much shorter than the average time a service need to onborad CPD, 6 months to 12 months. So the schedule was tight and risk was high.

Luckily, using my experience, skills and knowledge, I successfully finished this project with following steps:

Understand the Requirements

First of all, understanding the project requirements and creating line items.

I evaluated the consulting project, its architecture, functionalities, deployment and infrastructure. In order to release it as an IBM Data and AI product and cartridge of CPD, I came up with its updated product architecture, and MVP (the minimum viable product) features as essential requirements for the release in November 2020:

1. Meet the OSDU (Open Subsurface Data Universe) milestone functionalities support
2. Microservice design and full-stack, full-lifecycle container security
3. Onboard as a cartridge of CPD
4. Cloud agnostic and deploy anywhere, including edge deployment and computing.
5. CICD pipeline

Copied from https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=industries-architecture. It keeps basic component layer out from my initial design)

Initial Planning

Based on its architecture, MVP contents and the dependencies of line items on each other, I came up with a release roadmap. The following image shows my initial plan and estimations for major required tasks to onboard CPD and GA in November 2020.

My high level project plan. Due to confidential contents, I covered the work items and owners)

As shown above, I planned many work items in an innovative, parallel way to optimize the plan and enable deliver on schedule.

With my technical skills and knowledge of CPD, I was able to finish above architecture design and project plan/roadmap within a couple weeks. By the 3rd week of August, we were able to go into execution phase.

Project execution

We used the Agile project management with weekly sprint.

I will not put details here of how we run sprint by sprint, but just mention couple key things about this execution:

Keep whole project vision and requirements in mind when doing specific dev work

It is essential to:

1. Avoid siloed work, so each work will benefit whole product/project
2. Often, you may find that one work item can fulfill multiple requirements or support each other — essentially, one stone can hit multiple birds.

This will help optimize effort and much reduce project e2e time.

Try not to have too many meetings to disrupt development:

Each sprint we only have two required and one optional meetings:

1. Every Monday, a team sprint planning meeting, required.
2. Every Friday, team sprint demo meeting, required.
3. Wednesday we host a mid-sprint status checking meeting, optional.

Anything else happened offline through slack/sametime communication or small group talk.

Break big work items into weekly feasible small items

This is another key to make project succeed. The work items are small enough so development team clearly understand what to do each sprint, and any delayed item will be a risk that need to mitigate quickly.

How to break an item dependents on the actual contents. Based on my experience, most software engineering item can be broken into smaller ones that need no more than four personal-day effort and fit into a weekly sprint.

For me, I initiated the Agile process and work item splitting about six or seven years ago when I led the transformation of IBM Streams into a Cloud Native product with a CICD pipeline. I found significant guidance from a blog. The version I read at that time was an older iteration, as attached here:

Risk management and mitigation

Risk is an inherent aspect of any project. When I led the project, my focus was on addressing the issues that cause risks and mitigating risks quickly to minimize their impact on the overall project’s progress.

Many issues generally fall into following two scopes:

(Feature) Item not work as originally designed/planned

This kind of issue happens quiet often within software development, due to technical issue, language limitations, or 3rd party functional dependency, some originally designed work items can not be supported. In this case, we need to think and quickly answer following questions:

1. Is this a must-have feature?
2. if must-have, any other approach or workaround to meet the same use case?

When releasing a product as a cartridge of CPD, the first mandatory functional support is the installation and deployment. CPD provides its own installation tool — cpd-cli. However, due to the partnership and special functional and market requests of ODI, some ODI required infrastructure (open source) services, including MinIO, Keycloak, ElasticSearch, etc., cannot be installed using cpd-cli.

As a must-have for the ODI release, I came up with a solution and secured agreement from the Product Manager, Consulting and market team quickly — to support an automated two-step deployment: the first step is to use ArgoCD to install all infrastructure services, and second step is to use cpd-cli to install ODI core services.

Extra requirements that are not in original plan

Another set of issues that happen for software engineering project are some “new” (or different) requirements that not in original plan. As marketing change quickly, this often a case for many projects.

For this kind of issues, need to think of:

1. Do we have to support this new feature, or an feature in the new proposed way? What is business gain and lost comparing with the original plan?
2. If we need to support, what is the minimum effort based on existing (or planned) work items within allowed timeline?
3. If can not contain requests, any other approach or partially support that can meet same business need?

In September, about one month into the project, due to customer requests, Product Manager asked us to have a Beta release in early October 2020 to enable customer evaluation before official GA in November.

This is a deviation from the original plan. When I got this request, I evaluated the situation and the project status, defined the scope for Beta with Product Manager and came up with an approach that enabled beta release with minimal impacts to the overall release plan — if you check my project plan in section above, at early October, team could wrap up several key features and release Beta. We successfully released Beta version one week ahead of the CPD official Beta release, with no extra development work but some paper work.

In the 4Q 2020, the OSDU Forum required the product to support backup and restore. This was not in the original ODI release MVP scope.

As OSDU required it, ODI had to support in certain way. Based on the ODI architecture I came up and microservice behavior, as well as CPD backup and restore support, I came up with an approach using CPD backup and restore tool but in a creative, cross-namespace way. After my hand-on verification, it was proven to work well. With this, we successfully claimed support for not only in-place backup and restore, but also active-passive disaster recovery, with little extra technical work.

Summary

After 3 months of creative and hard work, I led the ODI development team to successfully release IBM ODI v1.0.0 on schedule on November, 2020. For this initial release, we also covered features that were not in my original MVP plan, including quiesce, backup&restore, HA and Disaster Recovery, and scalability.

Reviewing the whole project, I feel following are critical for the project succeed:

Project leaders’ technical knowledge and skills

At time I was assigned to lead this project, I had led IBM Streams into CICD pipeline and design/led CPD SRE infrastructure support. These experiences, skills and knowledge proven to be critical to make this project successful, from initial project plan, to guiding team daily development work, and solving all issues quickly.

Innovation and be able to make decision and mitigate risk quickly

For a project with tight schedule, being able to make decision and coming up with innovative approaches for plan change, feature adjustment, solving technical issues is essential.

Team communication and coordination

This is obvious to make sure all parties and stakeholders of the project/product align to make project going smoothly.