Ridesharing services, such as Uber and Didi, are enjoying great popularity; however, a big challenge remains in guaranteeing the safety of passenger and driver. State-of-the-art work has primarily adopted the cloud model, where data collected through end devices on vehicles are uploaded to and processed in the cloud. However, data such as video can be too large to be uploaded onto the cloud in real time. When a vehicle is moving, the network communication can become unstable, leading to high latency for data uploading. In addition, the cost of huge data transfer and storage is a big concern from a business point of view. As edge computing enables more powerful computing end devices, it is possible to design a latency-guaranteed framework to ensure in-vehicle safety. In this paper, we propose an edge-based attack detection in ridesharing services, namely SafeShareRide, which can detect dangerous events happening in the vehicle in near real time. SafeShareRide is implemented on both drivers’ and passengers’ smartphones. The detection of SafeShareRide consists of three stages: speech recognition, driving behavior detection, and video capture and analysis. Abnormal events detected during the stages of speech recognition or driving behavior detection will trigger the video capture and analysis in the third stage. The video data processing is also redesigned: video compression is conducted at the edge to save upload bandwidth while video analysis is conducted in the cloud. We implement the SafeShareRide system by leveraging open source algorithms. Our experiments include a performance comparison between SafeShareRide and other edge-based and cloud-based approaches, CPU usage and memory usage of each detection stage, and a performance comparison between stationary and moving scenarios. Finally, we summarize several insights into smartphone based edge computing systems.