SOFTWARE ENGINEERING INSTITUTE | 39
Appendix A CSF Method Application Notes
The simple nature and broad applicability of CSFs lead to a variety of ways in which they can be
used and misused.
31
We found that each aspect of the method, though controllable, is inherently
subjective, and subject to error. The following sections address some of the issues associated with
the method and provide some recommendations and guidelines for addressing those issues.
Qualifying the Application
An organization that does strategic planning well is a good candidate for the CSF technique. It is
essential to look at critical success factors in the context of an entire strategic planning effort, ra-
ther than as a stand-alone endeavor. Ideally, an organization’s mission, vision, and current goals
will be articulated and well understood prior to launching a CSF activity. In addition, much of the
information collected in the context of critical success factor interviews will be inherently outside
the scope of critical success factors, despite careful questioning and attention to keeping respon-
dents on track. For example, we gathered rich information regarding recommendations, visions,
measurements, goals, and values. All the information was captured as potentially (and eventually)
valuable input, but much of the data did not relate to critical success factors. A solid strategic
planning method, such as the one described by Fogg in Team-Based Strategic Planning: A Com-
plete Guide to Structuring, Facilitating, and Implementing the Process, can provide ready recep-
tors for non-CSF oriented information [Fogg 1994].
If an organization is not adept at strategic planning, it is highly recommended that basic strategic
planning groundwork is laid first. CSFs can then be brought in to extend and improve the organi-
zation’s understanding of its operational and competitive self.
CSF Interviews
The interview process, like the CSF method, is simple and can be used much more broadly than
for developing CSFs alone.
32
Bullen and Rockart’s approach to interviewing individuals was to
collect individual CSFs that could be viewed as a group, wherein consistently referenced ideas
could be extracted and refined into a set of organizational CSFs [Bullen 1981]. But (and Rockart
encourages this) the CSF-style executive interviews can be used to elicit other insightful informa-
tion, including goals, guiding principles, barriers, visions, measures, and more. Information
beyond the scope of CSFs can be collected via these interviews either explicitly or by accident.
Open-ended questions like those used in the CSF method elicit free-form data that can be off top-
ic—and useful, or not. When recognized, the expansion of an interview question or answer can
31
In their assessment of the CSF method, Boynton and Zmud examined the previously cited difficulty of using the
CSF method and concluded that the method should be directed by a skilled analyst [Boynton 1984].
32
Bullen and Caralli provide good detail about how to formulate interview questions and conduct CSF interviews
[Bullen 1981, Caralli 2004]. This material is worth studying.