Build issue: /bin/sh: ./transactiondb.sh: Permission denied

[MichaelSchoenfelder-SiV]

Occasionally when building map and sparta, I get /bin/sh: ./transactiondb.sh: Permission denied.
I think this is a race condition because transactiondb.sh is a dependency.

It is produced like this:

helios/pipeViewer/transactiondb/CMakeLists.txt:  python3 ${CMAKE_CURRENT_SOURCE_DIR}/setup.py build_ext --inplace;' > transactiondb.sh && chmod +x transactiondb.sh

If make is looking for transactiondb.sh, it might see it and try to use it before the chmod is completed on the filesystem.

This would be safer:

python3 ${CMAKE_CURRENT_SOURCE_DIR}/setup.py build_ext --inplace;' > transactiondb.sh.tmp && chmod +x transactiondb.sh.tmp && mv transactiondb.sh.tmp transactiondb.sh

[MichaelSchoenfelder-SiV]

My suggestion didn't work and I don't know why.

Also, the race condition may be coming from some place else. Both of these rules generate the same target of transactiondb.sh. I don't know why, but clearly that's another race.

17:45:42 Scanning dependencies of target transactiondb_driver
17:45:42 Scanning dependencies of target transactiondb_sh_driver
17:45:42 [ 92%] Generating transactiondb.sh
17:45:42 [ 92%] Generating transactiondb.sh

[MichaelSchoenfelder-SiV]

On second thought, my idea may also be failing due to the exact same race condition. Here's what happened:

Scanning dependencies of target transactiondb_driver
Scanning dependencies of target transactiondb_sh_driver
Scanning dependencies of target transactionsearch
[ 92%] Generating transactiondb.sh
[ 92%] Generating transactiondb.sh
[ 92%] Building CXX object helios/pipeViewer/transactionsearch/CMakeFiles/transactionsearch.dir/src/transaction_search.cpp.o
mv: cannot stat 'transactiondb.sh.tmp': No such file or directory

If both of these targets are happen to fire at the same time, then transactiondb.sh.tmp will get moved out from underneath the 2nd target.

If I make -j 1, my changes work. I still think my suggestion is important for a parallel make, but the root cause of the race condition is making the same target twice. I don't understand the intent of having two rules for the same target, so will stop pursuing this for now.

[ghost]

Looking at the CMakeLists.txt for this, it's a wonder it works at all. The dependency chain is a mess, and I might not be coming across the issue 'cause I use ninja. I'll see if I can play with the make list, but I'll admit this will be hard to test.

[MichaelSchoenfelder-SiV]

I see a clue, but I don't understand how cmake works.

There's a custom target for both transactiondb_driver and transactiondb_sh_driver. The resulting makefiles are:

1. helios/pipeViewer/transactiondb/CMakeFiles/transactiondb_driver.dir/build.make
2. helios/pipeViewer/transactiondb/CMakeFiles/transactiondb_sh_driver.dir/build.make

Both have the make build step of:

python3 <my path>/map/helios/pipeViewer/transactiondb/setup.py build_ext --inplace ' > transactiondb.sh && chmod +x transactiondb.sh

I agree that there is a dependency problem. I assume it is there to solve some other problem. As long as you can get this all to build and the pipeviewer to work and avoid duplicate transactiondb.sh && chmod +x transactiondb.sh, this should work better. I don't see a need to recreate the race condition. I am certain we have found it.