“Waterfagile” requirements analysis in MICO

In a great collaborative R&D project such as MICO, different partners bring in what they do best, combining their know-how and competences. We are extremely happy to participate in MICO, and one of our first contributions to the project has been in the area of requirements analysis. This might come as a surprise at first, because Fraunhofer is typically associated with technology development, but there is an obvious connection: Applied research is about mediating between basic research and practical application, and activities such as requirements analysis and system design are necessary steps to “translate” and steer research into something that can will and can be applied in practice. It is a natural consequence that we get involved in respective tasks in public and industrial projects, it is simply “part of the job”.

Regarding requirements analysis, system design and integration, MICO has to deal with the typical challenges of public, collaborative R&D projects:

1. Because such projects are publicly funded, they have the obligation to achieve pre-defined goals and results, described in contractual documents, with a defined amount of resources, which they should use as efficiently as possible. As a consequence, they consist of fixed deadlines and milestones and quality metrics for official and internal deliverables and the overall project process and management that have to me met.
2. They are collaborative, with sometimes very big consortia. If everything goes well (and in our experience, it does more often than many people think), this means that partners with complementary competences can learn a lot from each other, create new ideas and concepts, and achieve things which would never be possible via bilateral cooperation. But collaboration is a challenge as well, and things can and do go wrong if they are not dealt with properly. The aforementioned planning of deadlines and milestones etc. is not only important to fulfill a pre-defined work plan. They are also important to enable the individual partners to plan resource usage, to ensure smooth interaction between partners: If one partner fails to deliver results on time which other partners depend on for their further work, significant problems and resource inefficiency can result from that. And the more partners interact in order to achieve a goal, the more this becomes an issue.
3. They are R&D, and R&D means high potential, high risk: Development is risky, and it is no surprise that concepts such as agile project management have evolved in order to address that. But research tends to be even riskier than that: By definition, research should be about something that has – at least partially – never been tried before, and is based on hypothesis which may or may not turn out to be correct. As a consequence, R&D activities cannot be expected to ever perfectly match a prepared plan. If they do, it is either an extraordinary achievement or a lack of R&D challenges.

So, as many other projects, MICO faces the challenge of meeting two contradicting goals: To stick with a plan, which requires “waterfall” elements to ensure that budget limitations, expectations and overall time planning are met. And at the same time, to include including “agile” elements to provide enough flexibility to cope with the unknown, which will always play a role because R&D is risky, and communication is difficult (and exponentially so with the amount of partners).

Requirement analysis (and afterwards, system design) is a key activity in that context, because it is the first activity to agree and act on, driving all R&D activities. It is also a difficult one, because identification, filtering and streamlining of requirements which are actionable and technically feasible an interactive process between application partners and technical partners, who may speak “different languages”. The earlier and better the requirements, the more efficient the resource usage, and the better and in line with the overall goals the results. If things go wrong here, the overall project is likely to fail.

MICO requirements analysis – showcases, user stories, datasets and technology enablers

There is no patent solution as to how this can be achieved, so for MICO, we agreed on a straightforward, fairly simple approach: A “wrapper” for fixed elements such as milestones and tasks corresponding with the description of work (DoW), and elements from agile project management, focusing on user stories, which are bound to specific projects that we called “showcases”, and supported by “technology enablers” representing various existing and – hopefully – future technological capabilities. In this manner, the methodology combines “top-down” (abstract, high-level requirements are derived into technical requirements) as well as “bottom-up” aspects (e.g. feedback regarding technical feasibility leads to the modification of high-level requirements) approaches. It is outlined in the diagram below, and the relevant entities showcases (SC), user stories (US), datasets (DS), technology enablers (TE), use-cases (UC) as functional requirements, and non-functional requirements (NF) depicted above will be described in the following.

MICO methodology

Showcases (SC) are the main starting point for requirements analysis. While not representing requirements as such, MICO showcases represent current or planned projects of Zooniverse (ZOO) and InsideOut10 (IO). They provide the context for all further requirements, and represent slightly different combinations of goals, content sets, and user communities involved. After the brainstorming phase within the MICO consortium, a set of 18 possible showcases were identified. They were then prioritized based on technical feasibility, impact and availability of datasets. This resulted in a total of 9 showcases as potential “targets” for the MICO project, which were further prioritized into two groups considering

• expected impact and industrial, scientific, and societal value
• variety and volume of cross-media content
• existence of “MICO aspects”, e.g. potential of accuracy and robustness improvements due to cross-modal analysis
• technical feasibility regarding availability of textual and audio-visual media extraction, publishing, querying, recommendation technologies, and cross-cutting technical platform requirements

Group A was considered for a first iteration under the precondition that some might be dropped during the process considering the R&D risks involved, and included

1. SC-02 News Videos [IO]
2. SC-10 Plankton Portal [ZOO]
3. SC-14 Galaxy Zoo [ZOO]
4. SC-16 Snapshot Serengeti [ZOO]
5. SC-01 Music Promotion [IO]

User Stories (US) are mapped to or derived from showcases, and serve as a starting point for requirements analysis, defining the high level requirements are essentially a technology-free process with the typical form (As a , I want (so that ), including unique ID, owner and additional information. Moreover, their relevance and interpretation can also be derived and understood from the the showcase descriptions they are mapped to. Because user stories can always be tied back to specific “demands” from a user perspective, they are used to align all further R&D work.

Technology Enablers (TE) are the technical counterpart to user stories. They are used to express, on a very high level, which technologies are required to support user stories and showcases. By linking TE to US, it can be made sure that what is “wanted” from a user perspective will be matched by enabling technologies, that technology gaps are identified, and that all technology development always serves a project goal. This was applied, for instance, for the SotA analysis work in MICO: By listing the relevant TE in the SotA sections, it was possible to quickly cross-check that (a) SotA analysis is aiming at things actually required (US) and (b) that demands (TE) stemming from US were met with corresponding SotA sections. In a similar manner, TE will serve for the upcoming system design and architecture decisions to cross-check technical work against demands.

Datasets (DS) are crucial not only to support user stories and use cases, and hence showcases, but also for training and testing purposes. Apart from supporting a showcase as such, content is crucial to train, and in some cases, develop textual and audio-visual extractors for the specific demands of a showcase, as needed for MICO. Datasets can also trigger new ideas regarding specific media extractor, publishing, querying and recommendation approaches.

A simple “waterfagile” project management approach using the MICO JIRA

Using the existing MICO project management infrastructure, at the core of which is JIRA, the aforementioned entities are used for a simple combination of waterfall and agile (hence “waterfagile”) elements:

• The work package structure, tasks within work packages, milestones and deliverables have all been created within the MICO JIRA, according to the description of work; this could be viewed as the “waterfall wrapper” for the project
• All relevant User Stories (US) and Technology Enablers (TE) for the selected Showcases (SC) have been created, including the links between them. This allows not only for simple navigation and cross-checking across entities. It also ensures that all issues are now (have to be, actually) linked to deliverables or technology enablers, ensuring alignment of work.
• US, TE and SC include respective descriptions in the JIRA to simplify the interpretation of associated issues, and they can be modified, extended, deleted or newly created during the process, to consider changes occurring during the R&D process. Following agile project management practices, issues are then assigned to sprints, representing an “agile core” for the approach.

From requirements to system design

For the further system design process, MICO uses the aforementioned entities to derive two types of concrete requirements for the system:

1. Functional requirements aka System Use Cases (UC), which serve as the starting point for system design, by providing basic flows which will then be detailed into UML sequence diagrams, specifying interaction between actors and components, and identifying component roles and descriptions, deriving APIs/interfaces, etc. They also provide a main reference for system evaluation. In contrast to User Stories (US), they are not technology-free, e.g. they imply that certain components or subsystems do exist.
2. Non-functional requirements (NF): They can be specific to a System Use Case (UC), or generic, applying to subsystems or the whole system. Examples: Usability, Testability, Performance, Scalability, Security, Privacy, Portability, Interoperability, Maintainability, Integrability, Extensibility.

Experiences so far – does it work?

So far, the described methodology has worked well to identify and describe high-level requirements. What remains to be seen is whether it also works well for the further system design and development phases, especially if significant modifications apply. We will revise the approach in a few months and report on this again.

Author: Patrick Aichroth, Fraunhofer IDMT