Smart environment at holy places integrates various intelligent devices, infrastructures, and services to monitor and control pilgrims' activities with efficiency. Smart pilgrim management requires aggregation of surrounding informatics for sustainable development of pilgrimage tourism. Some conventional sensing techniques, such as sensor networks are used to gather real time data. The management of large congregation is a primary factor in ensuring human's safety. In this paper, we investigate how sensor, mobile, and cloud technologies can be coupled together to address the pilgrim's safety, especially sensing, storing and analyzing the enormous number of pilgrims' whereabouts information in the smart environment. The existing solutions require a tag to track pilgrim. These tags need expensive sensors to detect them which results in high infrastructure cost. We develop a portable sensing system, named Smart Pilgrim, which uses pilgrims' smartphone whenever possible instead of tags. For sensing pilgrims' smartphones, we develop a sensing unit consisting of an Android based smartphone and an external Wi-Fi adapter. An extensive set of experiments is performed to evaluate the proposed system. The system developed is cost efficient, and will be helpful in congregation management.