Add automatic allocation of copies of the share library on demand for multiple copies of players. #77

marcharper · 2018-08-17T05:58:51Z

This PR adds a class to manage multiple copies of the shared library so that multiple copies of a given player can be used without the user having to maintain multiple copies of the shared library themselves. It does this by literally copying the shared library file and reloading it. (Loading the same file repeatedly doesn't work.)

It works as is and I was able to run the Moran Process for several copies of players, but there are two issues:

I think it's not thread safe so tournaments don't run correctly with multiprocessing
I haven't been able to find a way to locate the shared library file's absolute path so I've hard-coded it for now (which causes the build to fail here)

from axelrod_fortran import Player
import axelrod as axl
from matplotlib import pyplot as plt


if __name__ == "__main__":
    # Run a match
    players = (Player('ktitfortatc'),
               Player('ktitfortatc'))
    match = axl.Match(players, 5)
    print(match.play())

    # Run a Population Match
    players = [Player('ktitfortatc') for _ in range(5)] + [
        Player('k42r') for _ in range(5)]
    mp = axl.MoranProcess(players)
    mp.play()
    mp.populations_plot()
    plt.show()

… multiple copies of players.

marcharper · 2018-08-17T06:00:23Z

drvinceknight · 2018-08-17T07:43:47Z

Awesome, this would be great to fully get working as it would make things a lot simpler for the revisiting tournament too :)

meatballs · 2018-08-17T08:26:56Z

src/axelrod_fortran/player.py

+        self.player_indices = defaultdict(set)
+        self.player_next = defaultdict(set)
+        # Generate a random prefix for tempfile generation
+        self.prefix = str(uuid.uuid4())


I can't remembery why, but I seem to recall it's safer to use uuid4().hex here instead of str

Based on the docs here it looks like str(...) gives the hex value (perhaps previously it was different?). I just need a unique value so we can change to hex if there are any concerns.

meatballs · 2018-08-17T08:29:31Z

src/axelrod_fortran/player.py

+        self.library_path = self.find_shared_library(shared_library_name)
+
+    def find_shared_library(self, shared_library_name):
+        ## This finds only the relative path to the library, unfortunately.


I wonder if the absolute method of the Pathlib library might be able to help us out here

I'll look into it. I think the issue is that the library loader doesn't return the absolute path for some reason (though it clearly knows it) so we may need to either look for the file somehow (in a cross platform way) or instruct the user to symlink the file (not ideal).

Maybe if we have an install script for the shared library here we can capture the path at installation time and write to a config file.

marcharper · 2018-08-19T03:15:36Z

Ok, I've got a thread-safe implementation now and a proper tournament runs (see example below). There's also a function to find the absolute path on Linux (which works locally) and various documentation leads me to believe that the absolute path is returned by ctypes.util.find_library on other platforms (which I haven't tested).

However, the travis build still can't find the library, and the local tests never finish for me, using "python -m pytest" locally. I'm not as familiar with pytest or the travis setup so if either of you see anything please chime in!

from matplotlib import pyplot as plt
import axelrod as axl
from axelrod.action import Action

from axelrod_fortran import Player
from axelrod_fortran.strategies import all_strategies


C, D = Action.C, Action.D


if __name__ == "__main__":
    players = (Player('ktitfortatc'),
               Player('ktitfortatc'))
    match = axl.Match(players, 5)
    print(match.play())

    players = [Player('ktitfortatc') for _ in range(5)] + [
        Player('k42r') for _ in range(5)]
    mp = axl.MoranProcess(players)
    mp.play()
    mp.populations_plot()
    print(mp.populations[-1])

    players = [Player(name) for name in all_strategies]
    tournament = axl.Tournament(players, repetitions=2)
    results = tournament.play(processes=4)
    print(results.ranked_names)
    plot = axl.Plot(results)
    plot.save_all_plots("plots")

drvinceknight · 2018-08-20T08:33:07Z

Nice work @marcharper!

I've taken a quick look at running things locally. Here are my steps:

I have not done anything at all in TourExec.

To double check things I uninstalled whatever version of the axelrod_fortran library I had by repeatedly rerunning:

$ pip uninstall axelrod_fortran

Then I installed a develop version of the library to be able to test (this did not actually seem necessary, there might be something that takes care of this in the other files in the repo):

$ python setup.py develop

Then I ran pytest just be typing:

$ pytest

From the stacktrace I believe some things are working but I'm not entirely sure, I've attached the output to this comment: test.log

I'm not sure if that's helpful, when I get a moment I can look in to things a bit more.

meatballs · 2018-08-20T09:23:10Z

tests/test_player.py

@@ -26,6 +28,7 @@ def test_init():
        assert type(player.shared_library) is CDLL
        assert "libstrategies.so" in str(player.shared_library)

+
 def test_init_with_shared():
    player = Player("k42r", shared_library_name="libstrategies.so")


I think that passing the shared_library_name argument here will throw an error as it's no longer in the signature for player.init

Good catch. I'll use Vince's example above to get the tests run locally and fix them up.

marcharper · 2018-08-21T06:05:59Z

pytest still doesn't work locally for me but it seems to be some shenanigans with various library versions that I can't seem to sort. In any case it looks like we're in business!

drvinceknight · 2018-08-21T09:00:00Z

Nice work @marcharper! Looks good to me, when I get a moment I'll give it a whirl on the reproducing paper repo, should clear up the analysis code nicely :)

meatballs

Just one minor typo and a suggestion/debate topic from me. Looking really good!

meatballs · 2018-08-21T09:48:48Z

src/axelrod_fortran/player.py

@@ -106,6 +113,25 @@ def strategy(self, opponent):
            my_last_move)
        return actions[original_action]

+    def _release_shared_library(self):
+        # While this looks like we're checking that the shared library file
+        # isn't deleted, the exception is actually thrown in the manager


typo: in should be if

meatballs · 2018-08-21T09:56:26Z