Differences

This shows you the differences between two versions of the page.

--- evaluation [2010/05/06 02:08] – mark
+++ evaluation [2010/05/06 14:15] (current) – bil
@@ Line 31: / Line 31: @@
   * Evaluation of software clustering algorithms that produce nested decompositions
+====== Evaluation methods not based on an authoritative decomposition ======
+The main drawback of methods that require an authoritative decomposition is that they assume that such a decomposition exists. To construct such a decomposit
+ion for a middle-size software system is a challenging task. Various research studies deal with the construction of an authoritative decomposition. This work addresses two questions:
+  * What is the right process for the construction of an authoritative decomposition?
+  * Why is the constructed decomposition authoritative?
+Several evaluation methods that evaluate a software clustering approach based solely on its output have been developed. Jingwei et al. in "//Comparison of Clustering Algorithms in the Context of Software Evolution//" paper suggest comparing software clustering algorithms based on two criteria:
+  * Stability
+  * Extremity of Cluster Distribution
+The main criticize of both methods is that they can only identify ``bad'' software decompositions. It is easy to develop an algorithm that produces decompositions that are stable and non-extreme, while being entirely useless for program understanding purposes. Several methods attempt evaluate software clustering algorithms based on authomatically constructed "authoritative" decompositions.
+===== Stability =====
+Stability reflects how sensitive is a clustering approach to perturbations of the input data. The intuition behind stability in software clustering is that similar clustering decompositions should be produced for similar versions of a software system. A stability function measures the percentage of changes between produced decompositions of successive versions of an evolving software system. Under conditions of small changes between consecutive versions, an algorithm should produce similar clustering.
+===== Extremity of Cluster Distribution =====
+An interesting property of a clustering algorithm is the size of the clusters it produces. The cluster size distribution of a clustering algorithm should not exhibit extremity. In other words, a clustering algorithm should avoid the following situations:
+  * The majority of files are grouped into one or few huge clusters
+  * The majority of clusters are singletons
+====== See also ======
+  * [[UpMoJo]]
+  * [[refsim|LimSim]]
+  * [[StructureEvaluation|Structure Evaluation]]