With the booming proliferation of user requests in the Internet of Things (IoT) network, Edge Computing (EC) is emerging as a promising paradigm to provide flexible and reliable services. Considering the resource constraints of IoT devices, for some delay-aware user requests, a heavy workload IoT device may not respond on time. EC spurs a popular wave of offloading user requests to edge servers at the edge of the network. The orchestration for user-requested offloading schemes is a remarkable challenge for the execution time and energy consumption for IoT devices in edge networks. To address this challenge, we propose a dynamic computation offloading strategy consisting of the following two parts: (i) we propose the concept of intermediate nodes, which can minimize the time and energy cost to process the current task by dynamically combining task offloading and service migration strategies; and (ii) based in the current network’s workload state, we model such intermediate node selection problem as multiple dimensional Markov Decision Process (MDP) spaces and implement deep reinforcement learning algorithms to reduce the large MDP space and achieve fast decision-making. Experimental results show that this strategy performs better than existing baseline approaches on the reduction of delay and energy consumption.