METIS file format parser

[clstaudt]

The parser implements basic features of the METIS graph file format (unweighted, undirected graphs)

[akyrola]

I have merged your changes, thanks for the contribution!

[clstaudt]

Tried the community detection algorithm successfully on several graphs from the DIMACS collection, but running into an open file limit for the uk-2007 web graph:

INFO: graphchi_engine.hpp(print_config:125): Engine configuration:
INFO: graphchi_engine.hpp(print_config:126): exec_threads = 4
INFO: graphchi_engine.hpp(print_config:127): load_threads = 4
INFO: graphchi_engine.hpp(print_config:128): membudget_mb = 800
INFO: graphchi_engine.hpp(print_config:129): blocksize = 4194304
INFO: graphchi_engine.hpp(print_config:130): scheduler = 1
INFO: graphchi_engine.hpp(run:706): Start iteration: 0
INFO: graphchi_engine.hpp(run:760): 0.232941s: Starting: 0 -- 591138
INFO: graphchi_engine.hpp(run:773): Iteration 0/9, subinterval: 0 - 591138
DEBUG: memoryshard.hpp(load_edata:249): Compressed/full size: 0.00436574 number of blocks: 25
ERROR: stripedio.hpp(open_session:318): Could not open: /home/i11/staudt/Graphs/Prototypes/graphchi/real/uk-2007-05.graph.edata.e8B.60_254_blockdir_4194304/0 session: 340 error: Too many open files
ERROR: stripedio.hpp(open_session:318): Could not open: /home/i11/staudt/Graphs/Prototypes/graphchi/real/uk-2007-05.graph.edata.e8B.58_254_blockdir_4194304/0 session: 338 error: Too many open files
ERROR: stripedio.hpp(open_session:318): Could not open: /home/i11/staudt/Graphs/Prototypes/graphchi/real/uk-2007-05.graph.edata.e8B.59_254_blockdir_4194304/0 session: 339 error: Too many open files
communitydetection: ./src/io/stripedio.hpp:319: int graphchi::stripedio::open_session(std::string, bool, bool): Assertion rddesc>=0' failed. communitydetection: ./src/io/stripedio.hpp:319: int graphchi::stripedio::open_session(std::string, bool, bool): Assertionrddesc>=0' failed.
communitydetection: ./src/io/stripedio.hpp:319: int graphchi::stripedio::open_session(std::string, bool, bool): Assertion `rddesc>=0' failed.

Reading your publication on GraphChi, I see that you have already worked successfully with the same graph. So I guess I am doing something wrong. Tried to increase the operating system limit with: sudo sysctl -w fs.file-max=100000000, but the error remains.

What could be the problem here? If it is the limit set by the OS, could GraphChi read and check it before attempting a 30 minute computation which will then fail?

Kind regards
Chris

Am 05.08.2013 um 21:38 schrieb Aapo Kyrola [email protected]:

I have merged your changes, thanks for the contribution!

—
Reply to this email directly or view it on GitHub.

[akyrola]

Hi Chris,

have a look here:
https://github.com/GraphChi/graphchi-cpp/wiki/Troubleshooting

Aapo

On Aug 13, 2013, at 10:28 AM, clstaudt wrote:

Tried the community detection algorithm successfully on several graphs from the DIMACS collection, but running into an open file limit for the uk-2007 web graph:

INFO: graphchi_engine.hpp(print_config:125): Engine configuration:
INFO: graphchi_engine.hpp(print_config:126): exec_threads = 4
INFO: graphchi_engine.hpp(print_config:127): load_threads = 4
INFO: graphchi_engine.hpp(print_config:128): membudget_mb = 800
INFO: graphchi_engine.hpp(print_config:129): blocksize = 4194304
INFO: graphchi_engine.hpp(print_config:130): scheduler = 1
INFO: graphchi_engine.hpp(run:706): Start iteration: 0
INFO: graphchi_engine.hpp(run:760): 0.232941s: Starting: 0 -- 591138
INFO: graphchi_engine.hpp(run:773): Iteration 0/9, subinterval: 0 - 591138
DEBUG: memoryshard.hpp(load_edata:249): Compressed/full size: 0.00436574 number of blocks: 25
ERROR: stripedio.hpp(open_session:318): Could not open: /home/i11/staudt/Graphs/Prototypes/graphchi/real/uk-2007-05.graph.edata.e8B.60_254_blockdir_4194304/0 session: 340 error: Too many open files
ERROR: stripedio.hpp(open_session:318): Could not open: /home/i11/staudt/Graphs/Prototypes/graphchi/real/uk-2007-05.graph.edata.e8B.58_254_blockdir_4194304/0 session: 338 error: Too many open files
ERROR: stripedio.hpp(open_session:318): Could not open: /home/i11/staudt/Graphs/Prototypes/graphchi/real/uk-2007-05.graph.edata.e8B.59_254_blockdir_4194304/0 session: 339 error: Too many open files
communitydetection: ./src/io/stripedio.hpp:319: int graphchi::stripedio::open_session(std::string, bool, bool): Assertion rddesc>=0' failed. communitydetection: ./src/io/stripedio.hpp:319: int graphchi::stripedio::open_session(std::string, bool, bool): Assertionrddesc>=0' failed.
communitydetection: ./src/io/stripedio.hpp:319: int graphchi::stripedio::open_session(std::string, bool, bool): Assertion `rddesc>=0' failed.

Reading your publication on GraphChi, I see that you have already worked successfully with the same graph. So I guess I am doing something wrong. Tried to increase the operating system limit with: sudo sysctl -w fs.file-max=100000000, but the error remains.

What could be the problem here? If it is the limit set by the OS, could GraphChi read and check it before attempting a 30 minute computation which will then fail?

Kind regards
Chris

Am 05.08.2013 um 21:38 schrieb Aapo Kyrola [email protected]:

I have merged your changes, thanks for the contribution!

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.

Aapo Kyrola
Ph.D. student, http://www.cs.cmu.edu/~akyrola
GraphChi -- Big Data, small machine: http://graphchi.org