Hello.
I am running your code in the atari game Breakout-v0.
the settings are simple DQN(nips), DQN(nature), DDQN, dueling DQN, dueling DDQN.

now, each processes are running almost 6M(6,000,000) frames.
but all output of processes are very low.
-> avg_p ~ [0.3 ~ 0.5]
-> avg_ep_r ~ [0.25 ~ 0.45]

on the paper (Mihn et al, 2013),
[avg_p ~ at least 2.5] and [avg_ep_r ~ at least 50] in Breakout, running time ~ 6M

I didn't adjust any code about DQN.
Has anyone seen a breakout example run? was i something wrong?