The food industry such as meat producers and plant product processors have tremendous interest in detecting pathogenic organisms such as E.coli, Listeria, and Salmonella in very low concentrations down to a single cell. These pathogenic organisms when they are in the right environment can start multiplying exponentially. For example, E.coli cells can double every 20 minutes posing a tremendous danger for their growth in over many hours. We have designed an optical device that attaches to a smartphone providing an imaging and processing device that achieves an optical resolution of 1 micron. The optics is engineered to reduce aberrations in the system. We also developed a smartphone application that can track microbeads and bacteria in the video frames in real time using computer vision algorithms. We extract individual bacterial image segments in these videos to train neural network models to detect and differentiate different types of bacteria such as E.coli and B.subtilis. These trained models can detect and discriminate E.coli from B.subtilis with high accuracy of more than 80%. This approach has the potential to train different types of bacteria with a multiclass neural network classifier by training them with images from different genera and species of bacteria. Such a classifier can detect them in a wild sample containing many types of bacteria with low-cost smartphone optical device.