Polythene has always been a threat to the environment since its invention. It is non-biodegradable and very difficult to recycle. Even after many awareness campaigns and practices, Separation of polythene bags from waste has been a challenge for human civilization. The primary method of segregation deployed is manual handpicking, which causes a dangerous health hazards to the workers and is also highly inefficient due to human errors. In this paper I have designed and researched on image-based classification of polythene bags using a deep-learning model and its efficiency. This paper focuses on the architecture and statistical analysis of its performance on the data set as well as problems experienced in the classification. It also suggests a modified loss function to specifically detect polythene irrespective of its individual features. It aims to help the current environment protection endeavours and save countless lives lost to the hazards caused by current methods.