Recent literature reports have shown the ability to manipulate Cu-Zn cation ordering for Cu2ZnSnSe4 (CZTSe) via low temperature treatments. Theoretical arguments suggest that one of the major roadblocks to higher VOC - significant band tailing - could be improved with increased cation order; however, few direct measurements have been reported and significant device improvements have not yet been realized. This report investigates electrical properties, defects, and devices from quenched and slow-cooled single crystals of CZTSe. The extent of disorder was characterized by Raman spectroscopy as well as x-ray diffraction, where the change in Cu-Zn order can be detected by a changing c/a ratio. Quenched samples show higher acceptor concentrations, lower hole mobilities, and a lower-energy photoluminescence (PL) peak than crystals cooled at slower rates, consistent with a reduction in the bandgap. In addition, samples quenched at the highest temperatures showed lower PL yield consistent with higher quantities of deep defects. Devices fabricated using slow-cooled CZTSe single crystals showed improved efficiencies, most notably with increased VOC; however, low temperature intensity-dependent photoluminescence measurements continue to indicate the existence of potential fluctuations. We discuss the possibility that potential fluctuations in slow-cooled samples may be related to the inability to achieve a long range order of the Cu-Zn sub-lattice resulting in local regions of high and low levels of cation order, and consequent local variations in the bandgap. The presence of significant potential fluctuations, even after the slow-cooling step, suggests the difficulty in eliminating band-tailing in this system, and thus, additional approaches may be needed for significant reduction of the VOC deficit.