How Microprocessor giant, Intel used Scrum in their engineering development?

Microprocessor giant Intel was using a Test Program which runs on Automated Test Equipment (ATE). It has a proprietary operating system and interface languages that prevent Intel from using industry-standard off-the-shelf software validation solutions. In essence, Intel was working in a proprietary language environment with no off-the-shelf unit test framework and no offline testing capabilities. Additionally, Intel has a long history of requirements: thrash, over-committing, missed schedules, grueling work weeks, poor morale, and high turnover rates. How they used Scrum to overcome these forms the basis of this story

In Intel, the Product Development Engineering (PDE) group exists to provide the test collateral to support cost-effective device screening and classification.

Intel’s waterfall culture led to functional silos and regular handoffs of deliverables between teams. A high turnover rate resulted from overburdening the PDE group toward the end of a project’s lifecycle.

Intel decided to introduce Scrum early on in the project and in phases.

Phase 1: PREPARING FOR SILICON

They hired an external company for Scrum training and coaching called Danube Technologies headed by Michael James and Dan Rawsthorne. Furthermore, the Process Action Team (PAT) was formed to monitor progress. Initially, six teams started following Scrum by the book.

In this phase, the biggest challenge was scaling work across scrum teams. They used the “learn, try, inspect, and adapt” approach and shared best practices that worked using an internal wiki. Although the Scrum framework became the standard means of managing requirements by the end of the first year, PAT still gave it a 50-50 chance of surviving.

The Intel teams were only building infrastructure to support silicon debugging and manufacturing. There was no outside force requesting certain features during the first year or so of the project. This made business value a difficult metric for prioritization. Therefore, the Product Owners and Business Owners tried to prioritize features with a combination of estimated business value and general priority, mostly as a dependency management strategy. By the end of the first year, Scrum had taken root within the organization and became the default framework for planning the work and management for their product.

Phase 2: SURVIVING SILICON

During this phase, the Scrum team focused on debugging Scrum Events and maintaining Scrum Artifacts. However, one team reverted to old habits and others were barely holding on. The two-week Sprints proved challenging, with some teams reducing them to just one day. Despite these difficulties, the group managed to identify teams and size, prioritizing business value, updates, and refining the product backlog. During several intense weeks of debugging and development, some teams even pulled in developers and product owners for testing and approval. Eventually, the surviving Scrum emerged stronger, expanding their sprints back to two weeks. The two weeks sprints remain in effect today.

Phase 3: PREPARING FOR MANUFACTURING

Through Scrum, they were able to identify what was slowing down their progress. The handoff between functional groups was one such issue. What they really needed were cross-functional teams to minimize handoffs. They ran a pilot test on one of the teams. As a result of this task, they were able to learn how to minimize handoffs. This presented a huge opportunity to influence the organization’s leadership and make a course correction that would allow Scrum to function better.

Outcomes:

Strong Definition of Done

Without a “real” programming language, Intel lacked a unit test framework or offline regression. In microprocessor product development, unit testing refers to testing silicon units. This challenge led them to focus on creating clear stories and defining customer satisfaction through well-written acceptance criteria (AC).

They also implemented a lightweight verification process called the “Pair Review”. This review was a collaboration between developers, POs or stakeholders. To track the effectiveness of “Pair Review” they collected simple metrics: Adds, Saves, and Escapes.

Adds: Additional AC added during the Pair Review that the developer agreed to implement in the current sprint.

Saves: Bugs caught and fixed in the current sprint

Escapes: Bugs created in previous sprints but found in the current sprint. Escapes indicate a need for improvement.

Saves: Showed the verification process was working.

Unlike verification, validation required that the story would function properly in the released product and often involved running the device on test equipment. A story passed the “Definition of Done” only when all tasks in the list were verified and validated.

No Partial Credit

To accurately predict the velocity for the next sprint, the team doesn’t consider stories that haven’t met their definition of “done.” This approach encouraged the team to prioritize verification and validation requirements in their estimates and ensures they meet their commitments. If the team committed to delivering 100%” done” but only delivered 90%, that would be considered “not done”.

Nine-day Sprints

Every two Fridays, the team conducts Review, Retrospective, and Planning Meetings during our nine-day sprints. This schedule allows the team to have a break from sprinting every other weekend, leading to improved quality of life and morale. On weekends in the middle of a sprint, team members have the option to work if necessary to meet their goals. However, after the first six months, this became a rare occurrence as they established a consistent rhythm and sustainable pace.

PO on the Team

To improve communication between Product Owners and the team, the team initially allowed POs to serve as active members of each team. However, this approach had mixed results, as some POs micromanaged the teams, hindering honest communication among team members. This resulted in teams holding secret meetings to address real organizational challenges away from the POs and functional managers. These issues, improved over time, but impaired the teams’ ability to self-organize. To prevent this, they banned the practice when building cross-functional Scrums.

Huge Backlogs

The team faced a challenge in managing the “all access backlog.” If anyone could add items to a team’s backlog at any time, it could cause overwhelming requests. Some Product Owners sought to lock down their backlogs, limiting input from other team members or stakeholders. To address this, our Scrum tool separates “new” stories from “accepted” stories. The PO can then review each new story, discuss it with stakeholders, and break it down accordingly.

Moreover, the team introduced a “freezer” for stories that wouldn’t be addressed for a few sprints. This enabled Stakeholders to see their requests in the freezer.

Tasks Take Less than a Day.

They decided not to follow estimation. Instead, stories were assigned a degree of “difficulty” in story points. While tasks were simply binary items – either “done” or “not done”. A task was generally set for less than a day. So, if someone worked on a task for more than a day, their task was likely impeded.

Executive Sponsorship

Upper management’s support proved essential to the success of the transition for both teams and managers. The organization’s manager provided crucial backing, offering incentives to leaders who took on team roles and recognizing their contributions with career credit. In addition, disincentives were put in place for anyone who undermined the process.

Changing Behaviors

The team learned that by consistently applying the principles of Scrum, they improved their efficiency. Through the ongoing negotiation of scope, prioritization, clear requirements, strict adherence to time boxes, monitoring metrics, and striving for team self-organization, Scrum has remained effective and thriving till today.

source link: https://scrumtrainingseries.com/Intel-case-study.pdf

Additional resources:

Scrum Case Studies

Why You Don’t Force an Agile Journey – Concepts and Beyond