Carpenter - Closed Frequent Item Set Mining

Download

Description

A program to find closed frequent item sets with the carpenter algorithm (Pan et al. 2003), which enumerates transaction sets, in contrast to many other frequent item set mining algorithms, which enumerate item sets. Such an approach can be highly competitive in special cases, namely if there are few transactions and (very) many items, which is a common situation in biological data sets, like gene expression data. For other data sets, however, it is not a recommendable approach.

The program can also find maximal item sets, but the filtering of the closed item sets is a quick hack and not particularly efficient.

This implementation offers a variant based on transaction identifier lists according to the description in (Pan et al. 2003), although with several optimizations due to which it significantly outperforms the implementation of the Gemini package, which is provided by the authors of (Pan et al. 2003).

The default algorithm, however, is a variant based on an item occurrence counter table, which bears some vague resemblance to the horizontal approach in the RERII algorithm (Cong et al. 2004). This algorithm is the default, because it usually outperforms the variant based on transaction identifier lists.

The improved carpenter algorithm used in this program is described in the following paper:

• Finding Closed Frequent Item Sets by Intersecting Transactions
  Christian Borgelt, Xiaoyuan Yang, Ruben Nogales-Cadenas, Pedro Carmona-Saez, and Alberto Pascual-Montano.
  Proc. 14th Int. Conf. on Extending Database Technology (EDBT 2011, Uppsala, Sweden), 367-376.
  ACM Press, New York, NY, USA 2011
  edbt_11.pdf (448 kb) edbt_11.ps.gz (373 kb) (10 pages)

Some other references:

• Carpenter: Finding Closed Patterns in Long Biological Datasets
  F. Pan, G. Cong, A.K.H. Tung, J. Yang, and M. Zaki
  Proc. 9th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD 2003, Washington, DC), 637-642
  ACM Press, New York, NY, USA 2003
• Mining Frequent Closed Patterns in Microarray Data
  G. Cong, K.-L. Tan, A.K.H. Tung, F. Pan
  Proc. 4th IEEE Int. Conf. on Data Mining (ICDM 2004, Brighton, UK), 363-366
  IEEE Press, Piscataway, NJ, USA 2004

More information about frequent item set mining, implementations of other algorithms as well as test data sets can be found at the Frequent Itemset Mining Implementations Repository.