Health monitoring of civil infrastructures is a key application of Internet of things (IoT), while edge computing is an important component of IoT. In this context, swarms of autonomous inspection robots, which can replace current manual inspections, are examples of edge devices. Incorporation of pretrained deep learning algorithms into these robots for autonomous damage detection is a challenging problem since these devices are typically limited in computing and memory resources. This study introduces a solution based on network pruning using Taylor expansion to utilize pretrained deep convolutional neural networks for efficient edge computing and incorporation into inspection robots. Results from comprehensive experiments on two pretrained networks (i.e., VGG16 and ResNet18) and two types of prevalent surface defects (i.e., crack and corrosion) are presented and discussed in detail with respect to performance, memory demands, and the inference time for damage detection. It is shown that the proposed approach significantly enhances resource efficiency without decreasing damage detection performance.