bashrc.d
Folders and files
Name | Name | Last commit date | ||
---|---|---|---|---|
parent directory.. | ||||
(C) Martin Väth <[email protected]> The license of this project is the GNU Public License GPL-2. The portage-bashrc-mv project is meant to be used with the gentoo portage system and serves two purposes. 1. It provides support for an /etc/portage/bashrc.d directory in which you can define functions which are executed during emerge phases. 2. It provides the following functionality in /etc/portage/bashrc.d (a) Support for an /etc/portage/package.cflags file (or directory) in which you can easily execute tasks or modify variables like CFLAGS on a per-package basis. This is similar to using /etc/portage/env but has a more convenient syntax for e.g. modifying CFLAGS. There is also a special flag filtering for non-GNU compilers (mainly clang). Moreover, pgo (profile-guided optimization) is supported. (b) Support for removing undesired .la files before installation (c) Support for removing undesired locales before installation (d) Support for CCACHE_* variables. (e) Output of time information and title bar if app-portage/portage-utils and "app-shells/runtitle" (the latter from the mv overlay) are installed For 1: It is attempted to keep sourcing of files as minimal as possible to not decrease the efficiency of portage more than absolutely necessary. This works as follows: When portage scans /etc/portage/bashrc for the first time, it sources once all files of the form $PORTAGE_CONFIGROOT/etc/portage/bashrc.d/*.sh in alphabetical order (the exceptional file bashrc.sh contains the main code for this and is therefore sourced first in any case; to fix the order, I recommend to keep all other filenames in the form [0-9][0-9]-*.sh). These files should only define functions and contain calls to BashrcdPhase. The function BashrcdPhase defines what should happen at which $EBUILD_PHASE. The call convention for this function is: BashrcdPhase name_of_the_phase 'action to be done' Here, name_of_the_phase is the name of the portage phase, and 'action to be done' will be executed with bash's exec in that phase, the order depending on the order of the calls to BashrcdPhase. Typical examples for name_of_the_phase are setup preinst pretend The special name all means that the corresponding action is executed in all phases and before the other actions for the ordinary phases are executed. The variable $EBUILD_PHASE contains the currently executed phase. You can find out which phases are called by portage by setting the variable $BASHCRD_DEBUG to true (Boolean values are handled in shell variables as follows: False means unset, empty value, -, 0, starting with f(alse), n(o), of(f) (case being ignored); everything else means true). Since there is global variable scope for which we must avoid collision, the following name conventions are used: All variables and function names are prefixed by the (letter parts of the) file names. The variable $ED is also set in EAPI=0: This is to simplify writing of functions which work for all ebuilds without taking care of the EAPI in this respect. For convenience, there are also some functions provided: BashrcdTrue $variable tests if $variable value is "true". BashrcdLog "output line 1" "output line 2" ... BashrcdEcho "output line 1" "output line 2" ... is as "elog" or "einfo", but prints each argument in a separate line. Moreover, if BASHRCDNOLOG or BASHRCDLOG is true, the output is like "einfo" or "elog", respectively. For 2: (a) Support for /etc/portage/package.cflags This is done in /etc/portage/bashrc.d/*-flag.sh: You can have a file or directory $PORTAGE_CONFIGROOT/etc/portage/package.cflags where you can store per-packet CFLAGS, LDFLAGS, and other variables. If it is a directory, all non-hidden files/dir (and those not ending in ~) in this directory and all its subdirectories are read (analogously as >=portage-2.1 reads /etc/portage/package.keywords) You temporarily add lines to the above file by putting them into the variable FLAG_ADDLINES The format of the lines (in the above file(s) or FLAG_ADDLINES) is as follows: Comment lines contain only space or start with "#" all other lines should have the following form: PACKAGEMASK DATA If >=app-portage/eix-0.26 (with the "masked-packages" tool) is installed, then PACKAGEMASK can have the form as in /etc/portage/package.mask (see e.g. man 5 portage). The DATA applies to the matching packages. For instance, PACKAGEMASK can be some of foo/bar (DATA applies to all versions of the package) =foo/bar-1 (DATA applies only to version 1) >foo/bar-1 (DATA applies only to newer versions than 1) ~foo/bar-1 (DATA applies to all revisions of version 1) =foo/bar-1* (DATA applies also to e.g. version 1.1 but not to version 2) foo/bar:x (DATA applies only to slot x) */*::gentoo (DATA applies only for packages from the gentoo repository) If you do not have the mentioned "masked-packages" tool installed, a heuristic is used to emulate the above behaviour, but e.g. the operators '<' '>' '<=' '>=' are not supported and wildcards are handled as "text" wildcards. All matching lines take effect, i.e. you can e.g. "accumulate" DATA. You can also add DATA (independent of the above matches) by putting it into the variable FLAG_ADD Summarizing, DATA is a concatenation of the following data (in this order): Matching lines from $PORTAGE_CONFIGROOT/etc/portage/package.cflags Matching lines from $FLAG_ADDLINES $FLAG_ADD The DATA is a list of commands, separated by spaces (the DATA is evaluated, so you can group a command using '...' or "..."; recall that unquoted variables are expanded in evaluation. Fileglobbing is not performed during evaluation.) A command can have one of these three forms: COMMAND [this executes COMMAND, see below] !COMMAND [this executes COMMAND if $HOSTTYPE = i686 ] ~COMMAND [this executes COMMAND if $HOSTTYPE = x86_64 ] COMMAND in turn can be any shell command which will be executed (e.g. setting or unsetting some variable). In addition, the following special COMMANDs are supported: -something This adds -something to the CFLAGS and CXXFLAGS variables if this flags was not included in these variables anyway. This is equivalent to C*FLAGS+=-something +something This removes -something from the variables CFLAGS, CXXFLAGS, CPPFLAGS, OPTCFLAGS, OPTCXXFLAGS, OPTCPPFLAGS, LDFLAGS, OPTLDFLAGS, FFLAGS, FCFLAGS, F77FLAGS. This is equivalent to *FLAGS-=-something Note that "something" can contain patterns, e.g. +O? removes all flags of type -O0, -O1, -O2, ... +flto* This is a special case of +something in the sense that it also implies +fuse-linker-plugin +femit-llvm +fwhole-program /-foo/-bar This is somewhat similar too +foo -bar with the following difference that the place of -bar is at the place of the first matching -foo, and that -bar is not added if there is no matching +foo (use -bar afterwards if you want to add -bar in such a case). For instance: /-O?/-O2 will replace the first -O? by -O2 and remove all other -O? /-march*/-march=native will transfer your -march setting to -march=native ATHLON32 This is somewhat special: It attempts to setup the compiler for 32-bit-mode cross-compiling (e.g. by setting CC, CXX) and by modifying -march. This may or may not work. VAR+=WORDS Add each word in WORDS to variable (with a separating space if necessary) unless VAR already contained this word. The individuals words are separated by spaces, but WORDS is evaluated so that you can use quoting. Args from the array FLAG_VAR_ARGS_WITH_PARAMS (defaults to empty array) are treated as quoted with the subsequent parameter. VAR-=WORDS Analogously to VAR+=WORDS, only that the words are removed. VAR/=/foo/WORDS Replace foo by WORDS The following names for VAR are treated specially in the above cases: RUSTFLAGS As above, but use the array FLAG_RUSTFLAGS_ARGS_WITH_PARAMS (default contains -C) instead of FLAG_VAR_ARGS_WITH_PARAMS. All of the following use FLAG_ARGS_WITH_PARAMS (default contains -Xclang) instead of FLAG_VAR_ARGS_WITH_PARAMS. CFLAGS... CXXFLAGS... LDFLAGS... C*FLAGS=... This sets CFLAGS, CXXFLAGS simultaneously and clears CPPFLAGS, OPTCFLAG, OPTCXXFLAGS, OPTCPPFLAGS C*FLAGS+=... Adds to CFLAGS, CXXFLAGS C*FLAGS-=... Removes from CFLAGS, CXXFLAGS, CPPFLAGS, OPTCFLAG, OPTCXXFLAGS, OPTCPPFLAGS C*FLAGS/=/foo/bar Similar to C*FLAGS+=foo C*FLAGS-=bar F*FLAGS=... This sets FFLAGS. FCFLAGS, F77FLAGS simultaneously. F*FLAGS+=... Adds to FFLAGS. FCFLAGS, F77FLAGS. F*FLAGS-=... Removes from FFLAGS. FCFLAGS, F77FLAGS. F*FLAGS/=/foo/bar Similar to F*FLAGS+=foo F*FLAGS-=bar *FLAGS=... This sets CFLAGS, CXXFLAGS, FFLAGS, FCFLAGS, F77FLAGS, and clears CPPFLAGS, LDFLAGS, OPTCFLAGS, OPTCXXFLAGS, OPTCPPFLAGS, OPTLDFLAGS *FLAGS+=... Adds to CFLAGS, CXXFLAGS, FFLAGS, FCFLAGS, F77FLAGS (currently the same as C*FLAGS+=... F*FLAGS+=...) *FLAGS-=... Removes from CPPFLAGS, CFLAGS, CXXFLAGS, LDFLAGS, OPTCPPFLAGS, OPTCFLAGS, OPTCXXFLAGS, OPTLDFLAGS, FFLAGS, FCFLAGS, F77FLAGS *FLAGS/=/foo/bar Similar to *FLAGS+=foo *FLAGS-=bar NOC*=... means the same as NOC*OPT=... This sets simultaneously the special variables NOCOPT, NOCXXOPT, NOCPPOPT (see below). NO*OPT=... This sets simultaneously the special variables NOCOPT, NOCXXOPT, NOCPPOPT, NOLDOPT (see below). NOLD*=... This sets simultaneously the special variables NOLDOPT, NOLDADD (see below). NO*=... This sets simultaneously all of the special variables NOCOPT, NOCXXOPT, NOCPPOPT, NOLDOPT, NOLDADD, NOFFLAGS, NOFCFLAGS, NOF77FLAGS (see below). SAFE This is a shortcut for NO*OPT=1 MESONDEDUP=1 LDFLAGS='' CONFIG_SITE='' NOLAFILEREMOVE=1 'unset CMAKE_MAKEFILE_GENERATOR' Note that this is the only command which modifies the variables CONFIG_SITE NOLAFILEREMOVE CMAKE_MAKEFILE_GENERATOR Some arguments take parameters: If supported by the compiler, try to use the -arg=parameter version; otherwise, sometimes quoting may help, though that might be tricky. For instance, +'Xclang\ foo' will filter '-Xclang foo' but not '-Xclang bar' After all COMMANDs have been executed, some additional actions are taken. You can prevent this by setting the corresponding variables to false (either by a COMMAND or in the environment): PGO If this variable is true, and the directory $PGO_DIR (usually /var/cache/pgo/$CATEGORY:$P, see below) does not exist, it is created during the preinst phase and equipped with world-writable permissions. Moreover, *FLAGS are added to write pgo profile information into that directory, i.e. if you use the resulting program after emerge, it will write profile data. Afterwards, you can reemerge the package and then presumably it will run faster. For more details see the description below. Note that the world-writable directory may be a security issue. An existing directory $PGO_DIR will cause PGO to be unset. If the variable PGO_DIR is not explicitly set, it defaults to $PGO_PARENT/$CATEGORY:$P If PGO_PARENT is not explicitly set, it defaults to /var/cache/pgo NOPGO If this variable is not true, it is attempted to set *FLAGS to read profile information from the directory $PGO_DIR (see above) if it exists. KEEPPGO If this variable is not true (and if PGO is not true either), and if the directory $PGO_DIR (see above) exists, that directory is deleted completely (and possibly empty parent directories removed). Usually, this is what you want unless you have set NOPGO. If you do not explicitly set this variable it defaults to false, but if you have set NOPGO (and $PGO_DIR exists) this variable defaults to true. USE_NONGNU If this variable is false, the flags from the bash array FLAG_FILTER_GNU are filtered: These are known to be not supported by GNU compilers. If this variable is true, the following happens: 1. The flags from the bash array FLAG_FILTER_NONGNU are filtered (these are known to be unsupported by sys-devel/clang). 2. The variable USE_NONGNU is exported. 3. NOLDADD (see below) defaults to true (unless you have explicitly defined NOLDADD). NOLDOPT If this variable is false, then the OPTLDFLAGS will be added to the LDFLAGS NOCOPT If this variable is false, then the OPTCFLAGS will be added to the CFLAGS NOCXXOPT If this variable is false, then the OPTCXXFLAGS will be added to the CFLAGS NOCPPOPT If this variable is false, then the OPTCPPFLAGS will be added to the CPPFLAGS NOLDADD If this variable is false, the LDFLAGS variable finally calculated will be added to the CFLAGS and CXXFLAGS. This variable is considered as true if MESONDEDUP is true. NOCADD If this variable is false and some of CFLAGS, CXXFLAGS, or LDFLAGS contains -flto* then the CFLAGS and CXXFLAGS variables finally calculated will be added to the LDFLAGS. This variable is considered as true if MESONDEDUP is true. MESONDEDUP If this variable is true, one has the behaviour of NOLDADD and NOCADD, and moreover, all entries of the finally calculated LDFLAGS which occur already in one of CPPFLAGS, CFLAGS, CXXFLAGS, FFLAGS, FCFLAGS, F77FLAGS are removed. This is to avoid the >=meson-0.47 insanity to know better than the user when some values should be duplicated for e.g. -flto* NOFFLAGS If this variable is false, then the CFLAGS finally calculated will also be used as FFLAGS NOFCFLAGS If this variable is false, then the FFLAGS finally calculated will also be used as FCFLAGS NOF77FLAGS If this variable is false, then the FFLAGS finally calculated will also be used as F77FLAGS NOFILTER_CXXFLAGS If this variable is false, then CXXFLAGS from the bash arrays FLAG_FILTER_C_CXX FLAG_FILTER_CXX_FORTRAN FLAG_FILTER_CXXFLAGS are removed. NOFILTER_CFLAGS If this variable is false, then CFLAGS from the bash arrays FLAG_FILTER_C_CXX FLAG_FILTER_C_FORTRAN FLAG_FILTER_CFLAGS are removed. NOFILTER_FFLAGS If this variable is false, then FFLAGS from the bash arrays FLAG_FILTER_C_FORTRAN FLAG_FILTER_CXX_FORTRAN FLAG_FILTER_FORTRAN FLAG_FILTER_FFLAGS are removed. NOFILTER_FCFLAGS If this variable is false, then FFLAGS from the bash arrays FLAG_FILTER_C_FORTRAN FLAG_FILTER_CXX_FORTRAN FLAG_FILTER_FORTRAN FLAG_FILTER_FCLAGS are removed. NOFILTER_F77FLAGS If this variable is false, then FFLAGS from the bash arrays FLAG_FILTER_C_FORTRAN FLAG_FILTER_CXX_FORTRAN FLAG_FILTER_FORTRAN FLAG_FILTER_F77FLAGS are removed. There are several bash arrays used for filtering flags of some *FLAGS variables. The entries of these arrays can contain wildcards. The arrays are initialized to values which are known to be invalid for the corresponding *FLAGS variables. FLAG_FILTER_C_CXX Flags from this variable are filtered from CFLAGS and CXXFLAGS unless the corresponding NOFILTER_* variable is true FLAG_FILTER_CXX_FORTRAN Flags from this variable are filtered from CXXFLAGS, FFLAGS, FCFLAGS, and F77FLAGS unless the corresponding NOFILTER_* variable is true FLAG_FILTER_C_FORTRAN Flags from this variable are filtered from CFLAGS, FFLAGS, FCFLAGS, and F77FLAGS unless the corresponding NOFILTER_* variable is true FLAG_FILTER_CFLAGS Flags from this variable are filtered from CFLAGS unless NOFILTER_CFLAGS is true FLAG_FILTER_CXXFLAGS Flags from this variable are filtered from CXXFLAGS unless NOFILTER_CXXFLAGS is true FLAG_FILTER_FORTRAN Flags from this variable are filtered from FFLAGS, FCFLAGS, and F77FLAGS unless the corresponding NOFILTER_* variable is set FLAG_FILTER_FFLAGS Flags from this variable are filtered from FFLAGS unless NOFILTER_FFLAGS is true FLAG_FILTER_FCFLAGS Flags from this variable are filtered from FCFLAGS unless NOFILTER_FCFLAGS is true FLAG_FILTER_F77FLAGS Flags from this variable are filtered from F77FLAGS unless NOFILTER_F77FLAGS is true FLAG_FILTER_NONGNU Flags from this variable are filtered if USE_NONGNU is true FLAG_FILTER_GNU Flags from this variable are filtered if USE_NONGNU is false Some motivation for the above commands/variables: *PGO*: To use PGO for some package(s), set PGO=true in your environment or in /etc/portage/package.cflags and emerge the package(s). Then use the package(s) normally, and after a while reemerge it/them. Then usually the package(s) will be faster. Unless you have a special setup, you need not set any other *PGO* variables. (Note that the second emerge automatically happens with unset PGO, because PGO will be automatically unset if the directory $PGO_DIR exists.) Be aware that between the two emerges the world-writable directory PGO_DIR exists into which a malevolent user might write bad information: in case of a compiler bug this bad information might even be used to inject code into the package(s) for the second emerge. NOLDADD: Usually, packages take care of LDFLAGS automatically, so adding them to CFLAGS/CXXFLAGS causes lot of warnings and even breaks some compilations. Therefore, you will often want to set NOLDADD=1. However, some buggy packages ignore LDFLAGS which might be a securitry issue, e.g. if -Wl,-z,now is ignored. Therefore, the default of this script is to add LDFLAGS unless you explicitly forbid it. If you have compilation problems, setting NOLDADD=1 is what you should try first - especially before you send a bug report to bugs.gentoo.org. (But before you do the latter, you should try with SAFE anyway). NOCADD: For -flto, it is necessary to add CFLAGS and CXXFLAGS to LDFLAGS. In rare circumstances, this can cause trouble, so we allow optionally to deactivate this feature. NO*OPT / SAFE: For most packages, you will only need one of these two CFLAG settings: 1. A (semi-)optimized setting with flags like -D_FILE_OFFSET_BITS=64 which should apply to most packages although they are not necessarily safe. 2. A "simple" setting for testing (e.g. before you file a bug) or for packages for which the optimized setting causes problems. With this script, you can write in your make.conf the safe settings into the usual CFLAGS/CXXFLAGS/LDFLAGS variables. Moreover, in make.conf the variables OPTCFLAGS/OPTCXXFLAGS/OPTLDFLAGS can be set to the additional flags needed for the optimized setting. Then by default, this script will use your optimized settings except when you set NO*OPT=1. In particular, If you want to test (e.g. before you file a bug) you can quickly eliminate all optimizations by emerging with the command FLAG_ADD=SAFE emerge -1 relevant_packages Then (since SAFE implies NO*OPT=1) the relevant_packages will be built without "optimized" flags and without fancy things like removed la-files. Typical examples of lines in /etc/portage/package.cflags: sci-mathematics/scilab "export PVM_ARCH='LINUX'" # observe the quoting media-libs/freetype +ftree-vectorize* # Filter -ftree-vectorize* media-video/ffmpeg EXTRA_ECONF='' +combine media-video/kaffeine !/-O?/-O2 !+finline-functions # filter if in 32bit-mode media-video/mpeg4ip MAKEOPTS=-j1 sys-boot/grub SAFE # Use "safe" settings sys-fs/sysfsutils NOLDADD=1 sci-libs/*-atlas FEATURES-=ccache # Bad, see below... Important note concerning the FEATURES-Variable: For >=sys-apps/portage-2.2_rc68 you should better modify FEATURES by using /etc/portage/package.env than by using package.cflags: Modifying FEATURES with package.cflags need not have the desired effect. Anyway, the above example FEATURES-=ccache might work (as an exception); however, it is better to use /etc/portage/package.env anyway. For the above example, put into /etc/portage/package.env better the line sci-libs/*-atlas noccache.conf and create a file /etc/portage/env/noccache.conf with the line FEATURES=-ccache In a similar manner, you should handle modifications of other FEATURES. Support for Non-GNU compilers: If your CC or CXX contains "clang" it is assumed that you use sys-devel/clang. In this case, the variable USE_NONGNU=1 is exported. You can avoid this by setting this variable explicitly to the empty value, e.g. in make.conf or package.cflags. The effect of USE_NONGNU was described earlier. (b) Support for removing undesired .la files before installation. This is done in /etc/portage/bashrc.d/*-remove_la.sh: If the variable NOLAFILEREMOVE is false, all those *.la files are deleted (before merging to the filesystem) which appear to be unnecessary at runtime. It is recommended anyway to use >=portage-2.2_rc68 with FEATURES=fixlafiles. Note that the variable NOLAFILEREMOVE can be set per package by using (a). (c) Support for removing undesired locales before installation. This is done in /etc/portage/bashrc.d/*-localepurge.sh: If NOLOCALEPURGE is false and one of LOCALEPURGE MANPURGE is true and KEEP_LOCALES is defined, then the latter variable defines a list of locales or manpages to keep: All other locales/manpages are removed before installation (so that the removed files are not recorded in portage's database). The variable KEEP_LOCALES supports wildcards, e.g. KEEP_LOCALES="en*". In particular, setting KEEP_LOCALES="*" is the same as not setting it since all locales and manpages will be kept. In contrary, setting KEEP_LOCALES="" removes all locales and manpages (if LOCALEPURGE and MANNPURGE are true, respectively). If you observe that a language is not filtered, you can add it to the variable ALL_LOCALES_ADD (and perhaps file a bug to add the language to the defaults in future). Note that all of the above variables can be set per package by using (a). (d) Support for CCACHE_* variables: 1. This sets the following defaults for CCACHE_* variables: CCACHE_BASEDIR=$PORTAGE_TMPDIR/portage" (This saves lot of time without risk unless you have special debugging needs) CCACHE_SLOPPINESS='file_macro,time_macros,include_file_mtime,include_file_ctime,file_stat_matches,pch_defines' (This saves also a lot of time, and problems with it are very unlikely) CCACHE_COMPRESS=true (This is reasonable unless you reserve an incredible amount of cache. Moreover, it can speed up things; certainly on large machines, but sometimes even on slow machines.) CCACHE_CPP2=true # Only if USE_NONGNU and CCACHE_CPP2_OPTIONAL are true (The reason is that older versions of ccache had problems with clang without this setting. Since current versions have no such problem and CCACHE_CPP2 costs time, this is now done only if CCACHE_CPP2_OPTIONAL is set). If you do not agree with the defaults chosen above, just set these variables to some different value e.g. in your make.conf (globally) or package.cflags (per package). In this connection the following is important: 2. For all CCACHE_* variables (not only for the ones listed in 1.) the following happens: If the variable is false (in the Boolean sense described earlier), it is unset during emerge - otherwise, it is exported. For instance, you can put into make.conf the definition CCACHE_NOSTATS=true if you normally want to save the time to update the statics. Then you can still override this value by the environment for a particular emerge by calling: CCACHE_NOSTATS=false emerge -1 foo Due to the mechanism described above, CCACHE_NOSTATS will then actually be unset during the emerge call (note that if CCACHE_NOSTATS would not be unset, ccache would just recognize this variable as "set", not as "false"). (e) Output of time information and title bar: This is done in /etc/portage/bashrc.d/*-title.sh: If the "qlop" command of app-portage/portage-utils is available, it is used to output the expected emerge time of the package, whenever a package starts to emerge. Moreover, if in addition a "title" command is available (e.g. provided by app-shells/runtitle from the mv overlay), then this command is used to display the expected emerge time of the package as the windows title and/or in the hardstatus line, overriding portage's original output. The variable QLOPCOUNT determines in this display the number of last emerges taken into account for the average calculation. If this is not a number, then all previous emerges are taken into account. If unset or empty, the default is QLOPCOUNT=3. To skip this phase, you can set NOQLOP to a true value. Recommendations and notes concerning earlier versions of this project: (1) Some earlier versions of this project removed bash-completion files by default (if you did not want them explicitly). This is no longer the case, since this is better done by setting INSTALL_MASK in make.conf. For instance, to avoid installing bash-completion support, set INSTALL_MASK=/usr/share/bash-completion (2) Some earlier versions of this project have set the variables LANG or LC_ALL to sane values. This is no longer the case, since it is better to do this in make.conf. For instance, I recommend to put something like LANG=C LC_CTYPE=$LANG LC_COLLATE=$LANG LC_MESSAGES=$LANG into make.conf to avoid triggering bugs with not well-tested settings. Using package.cflags, you can still change these settings per package. (3) In some earlier versions of this project, lafilefixer was called by default. This is not longer the case, since this happens automatically if you use >=portage-2.2_rc68 and FEATURES=fixlafiles I strongly recommend to use that portage feature. (4) Some earlier versions of this project made an attempt to fix portage's behaviour concerning FEATURES="nodoc noinfo noman" with binary packages. Since this is fixed in current portage versions, this support was removed.