Database-As-A-service providers typically use replication to meet the performance and availability guarantees demanded by their customers. A crucial problem in this context is that of placing the replicas on machines in such a way as to meet these guarantees while optimally utilizing the available resources, despite having incomplete or erroneous a priori knowledge of the workload characteristics. In contrast to previous work, we incorporate this uncertainty by proposing online algorithms that make little or no assumptions about workload characteristics. In this paper, we provide a formal definition of variants of the replica placement problem, as well as a wide spectrum of criteria to evaluate proposed solutions. We also designed and evaluated a number of new algorithms for the online replica placement problem. We show that one of our algorithms, RkC, which is based on the notion of the 'power of two random choices', outperforms others with respect to these evaluation criteria.