Policy based Security Management in an accepted practice in the industry, and required to simplify the administrative overhead associated with security management in complex systems. However, the growing dynamicity, complexity and scale of modern systems makes it difficult to write the security policies manually. Using AI, we can generate policies automatically. Security policies generated automatically can reduce the manual burden introduced in defining policies, but their impact on the overall security of a system is unclear. In this paper, we discuss the security metrics that can be associated with a system using generative policies, and provide a simple model to determine the conditions under which generating security policies will be beneficial to improve the security of the system. We also show that for some types of security metrics, a system using generative policies can be considered as equivalent to a system using manually defined policies, and the security metrics of the generative policy based system can be mapped to the security metrics of the manual system and vice-versa.