Continuous, accurate, and real-time smartphone-based localization is a promising technology for supporting independent mobility of people with visual impairments. However, despite extensive research on indoor localization techniques, localization technologies are still not ready for deployment in large and complex environments such as shopping malls and hospitals, where navigation assistance is needed most. We identify six key challenges for accurate smartphone localization related to the large-scale nature of the navigation environments and the user's mobility. To address these challenges, we present a series of techniques that enhance a probabilistic localization algorithm. The algorithm utilizes mobile device inertial sensors and Received Signal Strength (RSS) from Bluetooth Low Energy (BLE) beacons. We evaluate the proposed system in a 21,000 m2 shopping mall that includes three multi-story buildings and a large open underground passageway. Experiments conducted in this environment demonstrate the effectiveness of the proposed technologies to improve localization accuracy. Field experiments with visually impaired participants confirm the practical performance of the proposed system in realistic use cases.