This paper is based on automatically detecting hazardous (dangerous) and non-dangerous objects from terahertz images. This paper has two specific contributions. First, we trained a neural network to automatically analyze dangerous and non-dangerous items, which can be used to experiment with terahertz images generated by a prototype terahertz video system. The system comprises a terahertz video database of people hiding dangerous and non-dangerous objects under their clothing. Secondly, VGG-19 (Visual Geometry Group-19) is trained on dangerous and non-dangerous objects from our database. After training, the accuracy received was 99.6% for safe items and 85.85% for dangerous items. We tested the network with various categories of objects not included in the training set and found that most were correctly identified as dangerous and non-dangerous. Also, we have identified some of the critical issues that need to be addressed to make this technology more accessible and widely used. Our work can pave the way for future research in this field and help address the challenges associated with terahertz imaging technologies. The paper describes some preliminary terahertz video surveillance experiments necessary for developing a natural terahertz video surveillance system.