Chapter 8 returns to the third key task of architectural design from Chapter 2: evaluating and validating the architecture before it is built.

ATAM in detail, and its lightweight version.

It is shaped by three drivers: functional requirements, non-functional (quality) requirements and business constraints (budget, schedule, regulations). An evaluation checks the architecture against all three.

The earlier a problem is found, the cheaper it is to fix: a wrong decision found on paper costs a meeting; found in code, it costs a rewrite.

It results in prioritisation of conflicting goals: stakeholders must agree which qualities matter most when they clash.

It forces a clear explanation of the architecture: the architect must be able to present and justify it.

It improves the quality of the architecture documentation, because gaps show up during the review.

It leads to improved architecture practices in the organisation over time.

ATAM: the Architecture Tradeoff Analysis Method (the main one in this chapter).

What they all have in common: they evaluate the architecture using scenarios and quality attributes. They differ in depth and focus, not in the underlying idea.

In practice that means discovering risks: places where an architectural decision affects a quality attribute of interest, often through a trade-off between qualities.

The key concept in every evaluation method is the quality attribute (Chapter 3).

In complex systems qualities can never be achieved in isolation: improving one affects others, sometimes negatively (encryption adds security but costs performance), sometimes positively (modularity helps both modifiability and testability).

So architectural decisions are really trade-offs between qualities in support of the business goals, which is exactly what ATAM analyses.

Team leader: sets up the evaluation contract with the client, forms the team, and makes sure the final report is produced and delivered. (The garbled text on the slide is overlapping lines in the PDF.)

Scenario scribe writes scenarios on the board and captures their agreed wording, stopping discussion until it is exact; the proceedings scribe records everything electronically (scenarios, the issues behind them, how each was resolved) and prints the adopted scenarios.

Steps 4–6 do the first analysis: identify architectural approaches, generate the quality-attribute utility tree, analyse the approaches against its top scenarios.

Steps 7–8 widen the view: all stakeholders brainstorm and prioritise scenarios, and the approaches are analysed again against them.

ATAM must produce at least the documentation of the selected architecture, whose key artefact is the SAD (Software Architecture Document) (Chapter 9).

And an evaluation report that recaps the ATAM process, captures the scenario analysis (quality requirements written as scenarios), and explains the candidate architectures and the rationale for choosing one, with a summary of the work done.

Section divider: slides 17–28 go through the nine steps one by one.

The 4+1 view model (Chapter 2, Lecture 3) is a good way to structure the presentation.

It covers the contextual view (relationships with people and other systems), the logical view (modules, layers), the process view (processes, threads, data flow, events) and the deployment view (CPUs, storage, devices, network).

The team analyses the architecture from step 3 in depth until it knows which patterns and approaches the architect used. These approaches are what the later analysis tests.

A utility tree is a top-down way of characterising the quality requirements: the root is “utility”, the next level the most important qualities (performance, security, availability, modifiability, maintainability), and the leaves are concrete quality scenarios.

Each leaf is rated for importance to the system’s success and difficulty to achieve (the architect’s assessment), usually H/M/L. High-importance, high-difficulty (H,H) scenarios are analysed first.

A utility tree for a car’s anti-lock braking system (ABS), with three columns: quality attribute → requirement → scenario.

Performance → R1 latency < 50 ms → “the ABS should respond in less than 50 ms after the driver hits the brake”.

Reliability → R2 system reliability > 95% → “fault tolerance of at least 94%” (note the slide’s numbers don’t quite match). Resource consumption → R3 CPU utilisation < 80% → “CPU use of the ABS software below 80%”.

The textbook utility tree with ratings (importance, difficulty). Performance: data latency → reduce storage latency on the customer DB to < 200 ms (L,M); transaction throughput → deliver video in real time (M,M).

Modifiability: new products → add CORBA middleware in < 20 person-months (H,H); change COTS → change the web UI in < 4 person-weeks (H,L).

Availability: hardware failure → power outage at site 1 redirects traffic to site 2 in < 3 s (H,H); COTS software failure → network failure detected and recovered in < 1.5 min (H,H).

Security: data confidentiality → credit-card transactions secure 99.999% of the time (H,M); data integrity → customer DB authorisation works 99.999% of the time (H,L). The (H,H) leaves go to step 6 first.

For each, it documents the relevant architecture decisions and their rationale, and classifies the findings as risks (decisions that may prevent a quality goal), non-risks (decisions judged safe), sensitivity points (decisions that strongly affect one quality’s response) and trade-offs (decisions that affect several qualities in opposite directions).

Architectural decisions with their classification: backup CPU(s) (sensitivity S2, risk R8); no backup data channel (S3, trade-off T3, risk R9); watchdog (S4, non-risk N12); heartbeat (S5, N13); failover routing (S6, N14).

Reasoning: different hardware and OS avoid common-mode failure (see risk 8); worst-case rollover is 4 seconds; failure is detected within 2 seconds from the heartbeat and watchdog rates; the watchdog is simple and proven; availability might be at risk without a backup data channel (risk 9). The architecture diagram shows primary and backup CPUs with heartbeat (1 s) to the switch CPU.

Step 7: the wider group of stakeholders brainstorms scenarios that matter for their own roles.

If new driving scenarios appear (they usually do), that may itself be a risk when the gap is large: the architect and stakeholders disagree about the system’s important goals.

The architect explains how the architectural approaches are affected by these new scenarios.

Risks, non-risks, sensitivity points and trade-offs continue to be identified, and architectural decisions are specified. It is step 6 repeated with the stakeholders’ scenarios.

It may be run entirely by people internal to the organisation. Because participants are internal and fewer, giving everyone their say and reaching shared understanding is much faster, and the steps run more quickly.

The cost: a lower level of scrutiny and objectivity, so it may not probe the architecture as deeply as a full ATAM with an external team.

The lightweight agenda. Step 1 (present ATAM) takes 0 hours, because participants already know the process; steps 2–4 take 15–30 minutes each, because everyone knows the system (1–2 scenarios are traced through the module and C&C views in step 3).

Step 5 (utility tree) takes 0.5–1.5 hours, often reusing existing scenarios; step 6 (analysis) takes the bulk, 2–3 hours, mapping the top scenarios onto the architecture.

Steps 7 and 8 are dropped (0 hours): the internal stakeholders already contributed scenarios in step 5, and all analysis happens in step 6. Step 9 (results) takes 0.5 hours to review the risks, non-risks, sensitivities and trade-offs. Total: about half a day to a day.

Recap: why and when to evaluate an architecture, its benefits, the methods (ATAM, SAAM, ARID: all scenario- and quality-based), and ATAM’s participants, nine steps, phases and outputs, plus the lightweight variant.

Next: documenting the architecture (Chapter 9).