In this paper we investigate dually flat structure of the space of positive definite matrices induced by a class of convex functions called V-potentials, from a viewpoint of information geometry. It is proved that the geometry is invariant under special linear group actions and naturally introduces a foliated structure. Each leaf is proved to be a homogeneous statistical manifold with a negative constant curvature and enjoy a special decomposition property of canonically defined divergence. As an application to statistics, we finally give the correspondence between the obtained geometry on the space and the one on elliptical distributions induced from a certain Bregman divergence.