Mixture of Gaussian models and bayes error under differential privacy

Abstract

Gaussian mixture models are an important tool in Bayesian decision theory. In this study, we focus on building such models over statistical database protected under differential privacy. Our approach involves querying necessary statistics from a database and building a Bayesian classifier over the noise added responses generated according to differential privacy. We formally analyze the sensitivity of our query set. Since there are multiple methods to query a statistic, either directly or indirectly, we analyze the sensitivities for different querying methods. Furthermore we establish theoretical bounds for the Bayes error for the univariate (one dimensional) case. We study the Bayes error for the multivariate (high dimensional) case in experiments with both simulated data and real life data. We discover that adding Laplace noise to a statistic under certain constraint is problematic. For example variance-covariance matrix is no longer positive definite after noise addition. We propose a heuristic method to fix the noise added variance-covariance matrix.