Differences

This shows you the differences between two versions of the page.

--- brain:arcade_games [2017/01/04 23:44] – created peter
+++ brain:arcade_games [2020/07/15 09:30] (current) – external edit 127.0.0.1
@@ Line 2: / Line 2: @@
 Download the most recent version of ALE (The Arcade Learning Environment) from http://www.arcadelearningenvironment.org/downloads/.
+Extract the download ALE file into a directory, such as **/ale**.
+Install dependencies
+<code bash>
+sudo apt-get install cmake
+sudo apt-get install libsdl1.2-dev
+</code>
+Assuming ALE was extracted to /ale. Then, compile it with CMake.
+<code bash>
+cd /ale
+ale> cmake -DUSE_SDL=ON -DUSE_RLGLUE=OFF -DBUILD_EXAMPLES=ON .
+ale> make -j 4
+</code>
+Create a directory for the ROM files.
+<code bash>
+mkdir /ale/roms
+</code>
+and copy the binary rom files into that directory.
+<WRAP info>
+The DeepMind-Atari-Deep-Q-Learner-master.zip file at https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner contains **breakout.bin**.
+Download this file too; or obtain roms from another source.
+</WRAP>
+Check that ALE has compiled correctly.
+<code bash>
+cd /ale
+ale -help
+</code>
+<WRAP info>
+This should display help tips
+</WRAP>
+Check the rom is able to be processed by ALE.
+<code bash>
+./ale rom/breakout.bin
+</code>
+This should display some details about the game.
+READ the manual at /ale/doc/manual/manual.pdf
+<code bash>
+cd /ale/doc/examples
+./sharedLibraryInterfaceExample
+./sharedLibraryInterfaceExample breakout.bin
+</code>
+<code bash>
+./ale -display_screen true roms/breakout.bin
+./ale -display_screen true -game_controller fifo roms/breakout.bin
+</code>
+Create a test program.
+<code bash>
+vi test.cpp
+g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale
+./test
+ls /etc/ld.so.conf
+cat /etc/ld.so.conf
+cat /etc/ld.so.conf.d/libc.conf
+ls /usr/local/lib
+cp libale.so /usr/local/lib
+g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale
+./test
+ldconfig
+./test
+./test roms/breakout.bin
+g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale -D__USE_SDL
+locate SDL.h
+g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale -D__USE_SDL -I/usr/include/SDL
+./test roms/breakout.bin
+./ale -display_screen true -game_controller fifo roms/breakout.bin
+</code>
+===== Install ImageMagick and supporting libs =====
+apt-get install imagemagick
+apt-get install libmagick++-dev
+===== Ideas =====
+Brain learns from images, i.e. image snapshots, or differences between one image and another.
+Images therefore need to be readable by the Brain.  Use jpg, png, bmp, own format libraries to do this.
+Standardise all images to a standard size to standardise learning.  Do we therefore assume X*Y is pre-set for any image.  The larger that X and Y is will result in more pixels, so more training / longer training needed to process, but more accurate.
+Perhaps different type neurons for different size images with some sort of standardisation comparison between unequal image sizes.
+Images - Convolutions
+update resolver's epsilon (chance of random action instead of optimal one)
+# epsilon decreases over time
+current_epsilon = 0.05 + 0.95 * np.exp(-epoch / 2500.)
+resolver.epsilon.set_value(np.float32(current_epsilon))
+# gamma - delayed reward coefficient - what fraction of reward is retained if it is obtained one tick later
+gamma = theano.shared(np.float32(1), name='q_learning_gamma')
+===== References =====
+http://www.arcadelearningenvironment.org/
+https://www.nervanasys.com/deep-reinforcement-learning-with-neon/
+http://www.imagemagick.org/Magick++/tutorial/Magick++_tutorial.pdf