Learning to Search for Job Shop Scheduling via Deep Reinforcement Learning

Recent studies in using deep reinforcement learning (DRL) to solve Job-shop scheduling problems (JSSP) focus on construction heuristics. However, their performance is still far from optimality, mainly because the underlying graph representation scheme is unsuitable for modeling partial solutions at each construction step. This paper proposes a novel DRL-based method to learn improvement heuristics for JSSP, where graph representation is employed to encode complete solutions. We design a Graph Neural Network based representation scheme, consisting of two modules to effectively capture the information of dynamic topology and different types of nodes in graphs encountered during the improvement process. To speed up solution evaluation during improvement, we design a novel message-passing mechanism that can evaluate multiple solutions simultaneously. Extensive experiments on classic benchmarks show that the improvement policy learned by our method outperforms state-of-the-art DRL-based methods by a large margin.

Learning to Solve Multiple-TSP with Time Window and Rejections via Deep Reinforcement Learning

We propose a manager-worker framework based on deep reinforcement learning to tackle a hard yet nontrivial variant of Travelling Salesman Problem (TSP), \ie~multiple-vehicle TSP with time window and rejections (mTSPTWR), where customers who cannot be served before the deadline are subject to rejections. Particularly, in the proposed framework, a manager agent learns to divide mTSPTWR into sub-routing tasks by assigning customers to each vehicle via a Graph Isomorphism Network (GIN) based policy network. A worker agent learns to solve sub-routing tasks by minimizing the cost in terms of both tour length and rejection rate for each vehicle, the maximum of which is then fed back to the manager agent to learn better assignments. Experimental results demonstrate that the proposed framework outperforms strong baselines in terms of higher solution quality and shorter computation time. More importantly, the trained agents also achieve competitive performance for solving unseen larger instances.

Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learning

Priority dispatching rule (PDR) is widely used for solving real-world Job-shop scheduling problem (JSSP). However, the design of effective PDRs is a tedious task, requiring a myriad of specialized knowledge and often delivering limited performance. In this paper, we propose to automatically learn PDRs via an end-to-end deep reinforcement learning agent. We exploit the disjunctive graph representation of JSSP, and propose a Graph Neural Network based scheme to embed the states encountered during solving. The resulting policy network is size-agnostic, effectively enabling generalization on large-scale instances. Experiments show that the agent can learn high-quality PDRs from scratch with elementary raw features, and demonstrates strong performance against the best existing PDRs. The learned policies also perform well on much larger instances that are unseen in training.

Re-route Package Pickup and Delivery Planning with Random Demands

Recently, a higher competition in logistics business introduces new challenges to the vehicle routing problem (VRP). Re-route planning, also known as dynamic VRP, is one of the important challenges. The re-route planning has to be performed when new customers request for deliveries while the delivery vehicles, i.e., trucks, are serving other customers. While the re-route planning has been studied in the literature, most of the existing works do not consider different uncertainties. Therefore, in this paper, we propose two systems, i.e., (i) an offline package pickup and delivery planning with stochastic demands (PDPSD) and (ii) a re-route package pickup and delivery planning with stochastic demands (Re-route PDPSD). Accordingly, we formulate the PDPSD system as a two-stage stochastic optimization. We then extend the PDPSD system to the Re-route PDPSD system with a re-route algorithm. Furthermore, we evaluate performance of the proposed systems by using the dataset from Solomon Benchmark suite and a real data from a Singapore logistics 1company. The results show that the PDPSD system can achieve the lower cost than that of the baseline model. In addition, the Re-route PDPSD system can help the supplier efficiently and successfully to serve more customers while the trucks are already on the road.