Connections between fuzzy inference systems and kernel machines

Abstract

This paper explores the connection between fuzzy inference systems and kernel methods. Particularly, we explore the connection between the fuzzy-basis-function expansion of non-singleton fuzzy systems and regularized kernel methods with positive-definite kernels. We show that positive-definite kernel on fuzzy sets, i.e., a real-valued function defined on the set of fuzzy sets, is the core concept enabling such a connection. Furthermore, we rewrite the fuzzy-basis-function (FBF) expansion of non-singleton fuzzy systems in terms of a kernel-basis-function expansion with positive-definite kernels on fuzzy sets. The consequence of doing this is that fuzzy systems can be trained by regularized kernel methods, and the advantages of doing this are: 1) fuzzy systems can have more resistance to the curse of dimensionality, 2) they can explicitly regularize the function being approximated, 3) they can satisfy the Representer Theorem of kernel methods, which bounds the number of learned rules to the number of training observations, and, 4) they can learn functions in the functional space induced by the kernel on fuzzy sets. Consequently, from this perspective fuzzy systems can be regarded as kernel methods. Different from previous works, this connection relates the FBF-expansion directly to kernel methods without dropping the denominator of the expansion; also, thanks to the use of kernels on fuzzy sets, we provide an algorithm that makes it possible to drop the restriction of having fuzzy sets that share some of their antecedent parameters, e.g., Gaussian membership functions with the same variance in rule antecedents as is usually done in related works.