With the exponentially increased demands for large bandwidth, it is important to think about the best network platform as well as the security and privacy of the information in communication networks. Millimeter (mm)-waves and terahertz (THz) with high carrier frequencies are proposed as the enabling technologies to overcome Shannon’s channel capacity limit of existing communication systems by providing ultrawide bandwidth signals. Mm-waves and THz are also able to build wireless links compatible with optical communication systems. However, most solid-state components that can operate reasonably efficiently at these frequency ranges (100 GHz–10 THz), especially sources and detectors, require cryogenic cooling, as is a requirement for most quantum systems. Here, we show that secure mm-waves and THz quantum key distribution (QKD) can be achieved when the sources and detectors operate at cryogenic temperatures down to T = 4 K. We compare single-input single-output and multiple-input multiple-output (MIMO) continuous variable THz quantum key distribution (CVQKD) schemes and find the positive secret key rate in the frequency ranges between f = 100 GHz and 1 THz. Moreover, we find that the maximum transmission distance could be extended, the secret key rate could be improved in lower temperatures, and achieve a maximum secret communication distance of more than 5 km at f = 100 GHz and T = 4 K by using 1024 × 1024 antennas. Our results for the first time show the possibility of mm-waves and THz MIMO CVQKD with the system operating at temperatures below T = 50 K, which may contribute to the efforts to develop next-generation secure wireless communication systems and quantum internet for applications from intersatellite and deep space to indoor and short-distance communications.